Questionnaire

In order to improve software, it is essential to define and frame the limits, flaws, and challenges that users face when using them in the industry. This kind of software feedback is related to architecture and engineering practice, none or very few sources that determine these issues are available. Therefore, the users were given a questionnaire at the end of the usability study (see Figure 3.9), which was an essential tool for gathering critical information and feedback.

The questionnaire was divided into three sets of questions given and explained below. The first set asked questions about the user‟s background. The second set aims at probing the user‟s satisfaction with current software. Finally, the third set asked the user to provide feedback on StructureFIT to evaluate its potential input.

Figure 3.9: The questionnaire filled by users and collected at the end of the usability study On the left, the students are filling the questionnaire at the end of the sessions. On the right, StructureFIT

programmer, Caitlin Mueller, is collecting them.

3.2.1. Individual background

To start with, the first questions interrogate the user on his background, as shown in Figure 3.10. His field of study and his professional experience are determinant for our study as it tells how familiar the user is with design software. Depending on the academic or professional use of tools, depending on knowledge of different software, the users‟ results and opinions will differ and be more or less relevant. In other words, the influence of experience has to be taken into account in the results to a certain extent. As an example, a user already familiar to computer-based engineering tools is used to certain conventions, graphical user interface codes and terminology, would expect that all software would follow specific rules and designs (Glenford, 2012). Therefore, it would take him more time to learn the tool. Thus, if new frameworks differ too much from industry standards, it can limit its success.

Q1a. What is your academic background (i.e. degrees and majors)? What program are you in now?

Q2a. Do you have industry experience? If yes, for how many years did you work? Where did you work?

Figure 3.10: Sample of the questionnaire. Questions on the user’s background

3.2.2. Current software assessment

To answer the research question Q1 (i.e., how satisfied are architects and engineers are with today‟s structural design tools?), several questions (see Figure 3.11) focus on performance and limits encountered during the design process. Not only does this part of the questionnaire assess current software but it also enables the practitioner to suggest potential improvements based on his personal experience as a software user.

Q1b. What software programs do you usually use for design and analysis in your discipline?

Q2b. Which of these is your favorite? Why?

Q3b. Do you find these programs lacking in any aspects? If so, which?

Q4b. Please rate your satisfaction with these programs on a 5-point scale where 0 is Poor and 5 is Great.

Q5b. Do you think current software encourages you to explore design alternatives?

Why or why not?

Q6b. Give up to four features or functionalities you would like in an ideal design and analysis software program.

Figure 3.11: Sample of the questionnaire. Questions on the user’s satisfaction with current software.

3.2.3. StructureFIT assessment

To answer the research question #3 in a more general way (i.e., whether StructureFIT does provide solutions and improvements compared to existing frameworks), the second part of the questionnaire was written. After the user had become familiar with StructureFIT functionalities and had solved several problem sets, he has enough knowledge to assess this new framework performance. Even if it offers unique and unprecedented abilities such as design explorations and generated models to combine or modify, the full and efficient use of them totally depends on the user. Non-intuitive interface, difficulties to navigate, complexity to get results, and unnecessary data and functions are crucial elements that can dramatically lower the framework quality although it could make a breakthrough. Thus, this third set of questions in Figure 3.12, aims at identifying these matters and assessing whether or not this tool surmounted challenges by offering solutions to the previously defined current frameworks limits.

Q1c. How much do you like the designs you submitted?

Q2c. Did you enjoy using StructureFIT? Why or why not?

Q3c. Did you have any issues with the graphical user interface while using the tool? If so, which parts in particular?

Q4c. Do you think the provided numerical data (structural performance feedback or other information) is useful?

Q5c. Do you think StructureFIT helped you to explore good solutions for the three design problems?

Q6c. Please rate your satisfaction with StructureFIT on a 5-point scale where 0 is Poor and 5 is Great.

Q7c. Is there any way that StructureFIT could be improved?

Q8c. Any additional comments on StructureFIT?

Figure 3.12: Sample of the questionnaire. Questions on the user’s satisfaction with StructureFIT

Several questions presented in Sections 3.1.2, 3.1.3, aim as well at studying the perception of users. As Frekjmr et. al. (2000) mentioned: the correlation between effectiveness, efficiency and

satisfaction is not well understood and complex. User are asked to assess their performance based on their perception. As everyone uses different criteria, it is therefore interesting to question about what the users think of their own experience using the software and then compare with the actual result quality they provided during the usability study.