This study employed mixed methods to learn more about the experiences of students who participate in internships. Mixed methods inquiry involves both qualitative and quantitative methods, as both types of methods have strengths and weaknesses. Using both types of methods in the same inquiry helps to mitigate the weaknesses of each. According to Greene (2007), “a mixed methods way of thinking rests on assumptions that there are multiple legitimate
approaches to social inquiry and that any given approach to social inquiry is inevitably partial” (p. 20). Therefore, incorporating both types of methods results in a better understanding of a phenomenon through exploring related variables but also incorporating rich data about lived experiences. But the true potential of the methodology involves more than just using both qualitative and quantitative methods. The value of mixed methods is the opportunity to integrate different types of data at some point and/or throughout the study, providing a richer and more complete picture of what is happening.
Greene (2007) provides a brief history of the emergence of mixed methods. In the first half of the twentieth century, researchers learned quantitative methods to study phenomena. As other frameworks for inquiry developed, researchers began to adhere to a specific methodology and disagreements surfaced. The War on Poverty in the United States in the mid-1960s
instigated a number of programs to improve social welfare, and these programs needed to be evaluated. At the time, the default method for studying or evaluating government programs involved classic experiments with a control and an experimental group. But a number of issues surfaced that made using experiments to assess the impact and quality of these programs
challenging. First, ethical issues arose in assigning subjects to a control group and denying them the opportunity to realize the benefits of government programs. Second, there was a need to modify programs over time, resulting in suboptimal conditions for experiments. Further, many researchers believed that experiments did not take into account the processes used in the
programs or the experiences of the people who participated, leaving out critical data to evaluate the impact of the programs. These issues grew into a significant philosophical debate about the value of different paradigms or stances involved in research methods and how researchers view the world. Some researchers heavily rooted in either the qualitative or quantitative philosophies felt that the characteristics of the two prohibited mixing them because the inherent belief systems were too different, or incommensurable. However, others felt that mixing methods would
enhance the understanding gained by leveraging the strengths of each and minimizing the weaknesses. Mixed methods gained popularity with the emergence of triangulation in the field of sociology in the late 1970s. Over the next couple of decades, support for mixed methods increased and today it has become widely accepted practice in some fields. However, not all studies that claim to use mixed methods are conducted as effectively as others, and some still feel that the qualitative and quantitative philosophies are too fundamentally opposed to allow them to be mixed. While in most cases, a truce was reached, some debate still remains (Greene, 2007).
In this study, I aimed to better understand the student internship experience as well as any differences in student experiences at startups compared to that of established companies or due to inherent student characteristics. Qualitative methods helped me to learn more about students’ lived experiences during internships. However, I also wanted to develop a sense for how consistent student experiences are, which lends itself to quantitative data. I selected a mixed
methods approach because it draws on the strengths of both qualitative and quantitative methods and mitigates the limitations of each (Creswell, 2014). Using both methods provided me with a deeper, more comprehensive understanding of how responsibilities and tasks assigned to student interns differ by organization (if at all) as well as an opportunity to talk in depth with students who have participated in an internship at a startup or established company.
Research Design
This study used an exploratory sequential mixed methods design in which quantitative data was collected and analyzed in order to assess respondents on a set of variables related to internship experiences and to identify a pool of participants for a qualitative phase (Creswell, 2014). Two mixed methods purposes were employed through this research design:
complementarity and development. Complementarity uses multiple methods to understand different facets of a single complex phenomenon with the intention of enhancing or broadening the interpretations from the study (Greene, 2007). With the development purpose, one method provides results that are used to inform the subsequent method (Greene, 2007). In this study, the initial phase included a survey that was administered to business and engineering students who interned at companies during Summer 2015. The survey included questions about the job responsibilities and tasks that students are asked to perform in internships. Other questions inquired about the work environment, such as how much guidance coworkers and/or supervisors provide to the intern and the climate of the workplace, and how these factors impact the
experience the student has and what he or she learns as a result. This information was used to identify a pool of students to invite to participate in semi-structured in-person interviews. I planned to identify ten to twelve students who interned at a startup and the same number of students who interned at an established company by purposefully selecting them from those who
indicated they were willing to participate in the interview phase. I chose ten to twelve because in my experience conducting ten or more interviews of a certain group typically reaches a point of saturation where you learn minimally more with additional interviews. However, I planned to reassess whether the number of interviews was sufficient once I reached twenty or more (Baker & Edwards, 2012). Analysis of the survey data was used in the selection of interviewees, with the intent of maximizing information richness, though some efforts were made to include diversity in the interview sample (representation of both genders, socioeconomic status, race, etc.). The interview protocol included a core set of questions, and some additional or clarifying questions were sometimes incorporated based on the results of the survey. While the survey focused on job tasks and the workplace environment, the interviews aimed to gain a deeper understanding of student experiences related to how they interact with others in the workplace, the knowledge and skills they employ or develop in their role, and the level of guidance they receive in completing their assigned responsibilities. In addition, the rich data collected from the interviews supplemented the data from the survey so that I could explore general differences between startup and established company environments and social relationships, which in turn impact learning. Please see Appendix D for a copy of the approval letter from the University of Illinois Institutional Review Board.
The constructs that I planned to measure with the two methods include:
• Job tasks and characteristics: characteristics of the tasks students are assigned, the variety of tasks, how much control the student has over his or her work, etc., to address the first set of research questions (RQ1, RQ1a, RQ1b)
• Social interaction: how much of the student’s work involves or relies on working with others and how interactions with others may affect learning opportunities (training opportunities,
interest in the student’s development, etc.) to address the second set of research questions (RQ2, RQ2b)
• Level of guidance: how much the student learns how to do their work from demonstrations by others and mimicking what they do versus figuring things out on their own to address the second set of research questions (RQ2, RQ2a, RQ2c)
• Skills and knowledge: how much the student is able to understand or recognize connections between their work and what they learn in school, and the extent to which what they learn helps them to complete their assigned tasks to address the second set of research questions (RQ2)
• Organizational culture: the characteristics of the work environment, such as how much people work together, help each other, encourage discussion or ideas, etc. to address the second set of research questions (RQ2, RQ2a, RQ2b, RQ2c)
Please see Tables 2 and 3 in Appendix C for tables that link each survey and interview question to these constructs. In Table 2, I list each survey question, the construct that it measured, the research question it addressed, the source of the question if it was from an existing instrument, and the construct it measured in the source instrument, if applicable. Table 3 is similar in
concept but includes fewer columns because none of the questions came from an existing source. In this table, I list each interview question, the construct it measured, and the research question it addressed.
Data Collection Sample
The sample for this study included business and engineering students enrolled in four- year higher education institutions who participated in an internship during Summer 2015.
Because a number of startups that work with universities seek technical talent (computer science, engineering, or science students) and business skills, the sample focused on students in STEM (science, technology, engineering, and mathematics) and business majors (Lowrey, 2009). As mentioned in the research questions, the sample included students who interned at startups and established companies (startups and established companies are determined mostly by size, but to the extent possible were also based on the criteria listed in the introduction, including ability to grow, the number of employees, the number of board members, and annual revenues). As many students as possible were recruited to participate in the survey, and twenty-one students were selected from the survey respondents to participate in student interviews.
Sample recruitment
In the first phase of the study, students interning at startups were recruited through two sources: the University of Illinois Research Park/EnterpriseWorks, which provides startup incubator services in Champaign, IL, and three firms that provides recruiting services
specifically for startups: 1871 in Chicago, IL; the Illinois Technology Association in Chicago, IL; and Think B1g, which recruits nationally. Many of the startups that these companies work with are digitally focused startups.
Students interning at established companies were recruited through career services offices at public/state universities in several cities. Cities were selected based on identification as a startup-friendly city, defined as cities with high community engagement and better access to relevant resources for small businesses (Henry, 2015; Post, 2014). These cities included: San Diego, CA; Denver, CO; Austin, TX; Seattle, WA; Portland, OR; San Francisco, CA; Dallas, TX; Boston, MA; New York City, NY; and Chicago, IL. Eighteen universities representing these ten cities were contacted with a request to send out the survey, and six agreed to
participate, or approximately 33%. Fincham (2008) suggested that researchers aim for a minimum 60% response rate. While participation is different from response rate, 33%
participation is very low based on this criteria and unfortunately leaves 67% of the institutions unrepresented in the survey. I was disappointed with this participation rate because it was low, but also because several areas with startup activity were not represented in the survey, which could impact the results. In addition, the universities that participated in the study only
represented four of these cities: Chicago, IL; Denver, CO; Austin, TX; and San Francisco, CA. However, despite attempts to convince career services representatives to reconsider, no
additional schools agreed to participate. The most common reason cited for not participating was concern over sending too many communications to students, particularly communications that include surveys, and some institutions did not have a method to identify students who had interned during Summer 2015 and did not want to send the survey to all students. For those who did agree to participate, a recruitment letter was provided to recruiting entities that could be sent to students via e-mail with a link to the survey. The message explained the project and clearly indicated that participation was voluntary and could be discontinued at any time. Students were asked to provide informed consent before participating in the survey. Please see Appendix E for a copy of the recruitment message and Appendix F for a copy of the informed consent
documentation.
Students were encouraged to participate in the survey through an incentive. Students who chose to do so were entered into a drawing for one of two $100 gift certificates to
Amazon.com. The drawing took place in December 2015 once the survey data collection phase was completed and certificates were sent to two students via e-mail.
The sample for the student interviews was drawn from the respondents to the survey. Students were selected based on quantitative and qualitative data from the survey (more detailed information follows in the Data Analysis section). Three students who interned at startups were invited to participate in an interview, which represented the total number of students who interned at a startup and volunteered to participate in an interview. In addition, twenty-five students who interned at established companies were invited and eighteen of them accepted, for a total of twenty-one students representing established companies. Students who interned at startups were slightly overrepresented in the interview sample based on the survey sample (three of twenty-one interview subjects, or approximately 14% vs. fifteen of 131 survey subjects, or approximately 11.5%; please refer to Table 4 in Appendix C for a table that compares descriptive statistics for the students in the survey and interviews). Similar to the survey procedures,
students were asked to provide informed consent prior to participating in the interviews; they also received access to an explanation of the research study and were informed that participation was voluntary and could be discontinued at any time with no penalty. Students who participated in an interview received a $10 gift certificate to Amazon.com sent via e-mail after the interview was complete.
Student internship survey
Students who qualified for the study (those who interned at a startup or established company during Summer 2015) were invited to complete an online survey which includes
questions about their work tasks and responsibilities as well as the work environment. Tabanelli, Depolo, Cooke, Sarchielli, Bongiflioli, Mattioli, and Violante (2008) identified and reviewed 33 instruments used to assess psychosocial factors in the workplace. After reviewing several instruments identified in the article, the General Nordic Questionnaire and the Job
Characteristics Inventory provided the best fit to gather data to address the research questions. Twenty-three questions for the survey were drawn from the General Nordic Questionnaire, an instrument developed in 2000 for the Nordic Council of Ministers for the purpose of measuring psychosocial factors in the workplace. Measures in the survey include job demands and control, social interaction, organizational culture, work group, and several others (Lindstrom, Elo, Skogstad, Dallner, Gamberale, Hottinen, Knardahl, & Orhede, 2000). These questions explore the characteristics of work tasks and the work environment, and the intent is to use these questions to compare how the work tasks and environment are similar or different at startups compared to established companies. Eight questions were drawn from the Job Characteristics Inventory, which measures skill variety, autonomy, feedback, coworker interactions, etc.
(Taanelli et al., 2008; Sims, Szilagyi, & Keller, 1976). This instrument was developed to assess the characteristics of work tasks and can also help to understand how work tasks are similar or different at different types of companies. Both of these instruments will assist in answering the first set of research questions about the types of tasks that students are assigned in internships
While an existing instrument provides less flexibility in specifically addressing the research questions, its reliability and validity have been tested extensively and it provides for a uniform experience for all respondents (Lindstrom et al., 2000). In order to specifically address the research questions in the study, additional questions were added, including some qualitative or open-ended response questions. The survey included a total of 71 questions, including 24 questions about general student and internship characteristics, 37 questions with closed-ended responses that the respondent rates on a 5-point Likert scale, 4 questions with closed-ended multiple choice responses, 4 open-ended questions, and 2 questions asking about the respondent’s willingness to participate in an interview related to the study. The survey was
distributed in mid-August 2015, after students had completed most or all of their summer internships. The survey deadline was the end of September, to provide adequate time for students to respond but also to allow time to analyze the survey data for the qualitative phase. The deadline was originally early September, but was moved back because some institutions started the fall term later than others and the later deadline allowed contacts at these institutions to send additional reminders to students about the survey. Please see Appendix G for a copy of the survey instrument and Appendix F for a copy of the informed consent documentation.
The questions from the General Nordic Questionnaire and the Job Characteristics Inventory were tested for validity and reliability. The General Nordic Questionnaire was tested with employees representing several industries in four Nordic countries: Denmark, Finland, Norway, and Sweden; the survey was validated through a two-stage process in which the first data set was used to study the factor structure and develop the scales and the second was used to test the construct and predictive validity (Lindstrom et al., 2000). The Job Characteristics Inventory was tested with two populations, one at a medical center and one in a manufacturing facility (Sims et al., 1976). The researchers found that the instrument had “validity and
reliability characteristics acceptable for research on the relationship between job characteristics and employee attitudes and behavior” (Sims et al., 1976, p. 210).
The populations involved in the testing of the two instruments included full-time employees who likely had more experience than the interns who participated in this study. Therefore, all survey questions were tested for reliability, validity, and clarity with the target audience through a pilot of the survey with eight STEM and business college students at the University of Illinois to ensure that the meaning of the questions was clear and to estimate how long the survey would typically take students to complete. The pilot took place at the end of the
spring semester in mid-May 2015 to ensure that any changes could be made before the survey was administered during the Summer 2015 term. The pilot informed further development of the survey based on the feedback from students who participated in the pilot. Minor changes to questions, including wording and order, were made and the revisions were shared and an
amendment was filed with the University of Illinois Institutional Review Board (IRB). The IRB provided approval for the pilot survey along with approval for the overall study, as well as approval for the minor changes made in early May 2015. Please refer to Appendix D for copies of the original approval letter as well as two e-mails documenting approval of minor changes after the pilot and prior to the start of interviews.
The questions in the survey focused on several areas that were intended to provide
information about the context of the work environment. The survey included questions about the