determined to be reliable, and the sub-construct development was considered to be valid, it was necessary to determine the overall validity of the construct of School STEM Culture. Construct validity was determined by administering the School STEM Culture Questionnaire to the stakeholders in eight different school communities.
Recruitment of participating schools. Schools were originally recruited to participate in the study through email messages that were sent to principals and assistant principals in February of 2014. A message was sent to a random selection of three schools from each of the 50 United States. Schools chosen had a range of 900-2,000 students. The original recruitment email was titled “STEM Education at XXX High School” and included information about the study, along with some information about what would be asked of each school. Of the 150 schools originally recruited, 24 schools responded to the original recruitment email. Of the 24 schools that responded, 18 denied the request to complete research in their building, two schools agreed, and the other four schools gave contact information to request a research project within the district. For the four schools that gave district level contact information, after completing the research application, there was no response either at the district level or the school level. A second recruitment email was sent to many of these schools in April of 2014, with a
response from 22 of the schools. All of the responses to the follow-up recruitment email indicated that it was too late in the school year, or that their students were surveyed too much. Three of the schools made the comment that they did not have a STEM program, which is why they would decline participation. Because this step was completed in April of 2014, it was decided to wait until the fall of 2014 to continue trying to recruit schools.
In the Fall of 2014, several phone calls were made to schools on the recruitment list. When calling a school, a request was made to speak to the curriculum coordinator or an assistant principal in charge of science and mathematics. After calling more than 70 schools, I spoke with either a curriculum coordinator or assistant principal at 27 schools. For six of the contacted schools, the person contacted indicated some interest in the study, but all six of the schools decided not to participate citing a lack of a STEM program or not enough time to complete the study.
After little success with sending emails and calling the schools on the original list of 150 schools, a decision was made to create a short video for recruitment into the study and email that video to a larger sample of high schools. The video was created citing statistics on spending for scientific research and the need for STEM workers in the near future, and was sent to a list of over 300 school principals. The recruitment email was shortened to one paragraph identifying the main purpose of the study and asking the principal to contact the investigator if interested in participating. The language within the recruitment email also changed from “STEM School Culture” to “School culture in science and mathematics” to ensure that principals did not rule out participation because they did not have a STEM program. In addition, a sample report was sent to schools to show the type of data the school would receive upon participation in the study. A copy of a portion of the sample report indicating strengths and areas of improvement for schools
can be found in Appendix L. Responses were received from 42 of those emails, with 5 of those contacted manifesting interest in participation.
During the spring of 2014, two schools agreed to participate in the study, and completed their senior survey in that time period. Those schools were Fisk High School in the Western part of the US and Rice High School in the Midwestern part of the US, with all school names being pseudonyms. Data were collected from these schools with the intent to complete the School STEM Culture survey in the Fall of 2014. In the Fall of 2015, two more schools agreed to participate in the study: Williams High School in the northeastern part of the US, and Varitek High School in the southeastern US. An effort was made to contact more local schools with the offer that the author would come to the school to collect data from students and teachers to make the process easier on the school. After these contacts to 15 more schools, three more schools in the southeastern part of the US agreed to participate: Boggs High School, Ortiz High School, and Martinez High School. Data were retained from Evans High School from the pilot study, which made eight total high schools participating in the study.
Data collection. A separate survey was given to a sample of seniors of each participating school requesting information on their plans after high school. The students were asked to identify whether they planned to attend a 2/4 year college, enter the
military, or go straight into the work force. The survey then asked students to select from a list of 12 possible fields of study that they were most interested in pursuing. Students were also asked to complete an open- ended question regarding which college major or field of work they planned to pursue. Once the senior surveys were completed, results were analyzed and a percentage of seniors intending to pursue STEM fields was
STEM based on their choice of fields of study and their identified major or field of work. These results were used to calculate the percentage of seniors pursuing STEM fields.
The instrument was administered to a sample of stakeholders at the school determined by the administration of the school along with the survey of post-secondary plans. An attempt was made to have a sample of 30 teachers, 50 students taking the culture instrument, 50-100 seniors taking the post-secondary plans survey, all
administrators and counselors, and 30-40 parents for each school. Some schools had difficulty recruiting parents to participate, so two of the eight schools had very small samples of parents.
Data analysis. Once the data were collected from the STEM-CAT and the Senior Survey, the percentage of seniors pursuing STEM fields was calculated using the data collected from the Post-Secondary Plans survey completed by current seniors at each participating school. The seniors selected their intended cluster of study from a list of 14 clusters as defined by the South Carolina Economic Development Act. Students were also given an opportunity to identify their intended field of study or work in a text box. The researcher sorted responses by cluster, and identified students as STEM or NOT STEM according to their cluster and field of study/work. The percentage of students pursuing STEM fields was then calculated by dividing the number of students pursing STEM by the total. The author was conservative when identifying STEM fields if it was not obvious. For example, if the student indicated the intent to go into auto-body work, this was not defined as a STEM field even though many STEM concepts are present in auto-body work.
After collecting data from the administration of the STEM-CAT to stakeholder groups for each of the eight school communities, the issue of weighting for each
stakeholder group was addressed. Because there is no precedent set in the literature regarding how to weight each stakeholder group, two analyses were done. The first analysis, referenced as the Item Response Equality Analysis, maintained the equality of every response in the completion of the STEM-CAT. The second analysis, referenced as the Stakeholder-Domain Equality Method, maintained the equality of domains within stakeholder groups when calculating the total Positive Response Rate of the school community.
Item response equality method. In the Item Response Equality Method, each response to each item was considered to have equal value in the determination of the overall culture of the school. This provided each individual equal contribution to the overall Positive Response Rate score for that school, allowing for each individual student to have as much weight as the principal of the school. This method allowed for more input from individuals, but also might bias the results in favor of larger groups such as students or parents.
The calculation of the Total Positive Response Rate for each school using the Item Response Equality Method took the number of positive responses from all items, and divided that value by the total number of responses. This method allowed each respondent to have equal weight in the calculation of school-wide Positive Response Rate.
Stakeholder-domain equality method. Each stakeholder group makes a
significant contribution to the culture of the school community (Franco, Patel, & Lindsey, 2012), Although the student and parent groups are much larger than the others, the sphere of influence of the individual teachers and members of school leadership may be larger, which could conceivably account for the smaller N for these groups due to the stability of
these groups over a longer period of time. For example, a student or parent often interacts with the community for a time period of up to four years, while the teacher, counselor, or administrator is often a part of the culture for a longer period of time. This gives the teachers and school leadership a longer time period to affect the culture of the school. Considering the gap in the literature to quantify the amount each group
contributes to the culture, it was assumed that each stakeholder group would contribute an equal amount to the culture.
The calculation of the Total Positive Response Rate for each school using the Stakeholder-Domain Equality Method took the average PRR for each stakeholder group from all domains. The average PRR between stakeholder groups was then calculated and used as the Total PRR for each school. This method allowed for each stakeholder-
domain combination to have equal weight in the calculation of school-wide Positive Response Rate.
Correlation analysis. Using both methods of calculating the Total PRR for each school, the results from the STEM-CAT were used to correlate the percentage of seniors pursuing STEM fields to the total PRR for each school. Neutral responses were not counted as positive responses. This positive response rate (PRR), both for individual stakeholder groups and for the total school population, was correlated to the percentage of seniors planning to pursue STEM fields. A positive correlation would indicate that the construct of STEM culture was a valid construct and that the survey accurately measures that construct.
Using the data collected through the STEM-CAT and the Senior Survey, a scatterplot was created showing the overall Positive Response Rate for each school plotted against the percentage of seniors pursuing STEM fields. A bivariate correlation
was run using SPSS (v 22.0) using Pearson’s Correlation Coefficient (Hinkle, Wiersma, & Jurs, 2003). A p level of less than 0.05 was considered a significant result. After running the bivariate correlation between the total PRR and Percentage of seniors pursuing STEM fields using the Item Response Equality Method, bivariate correlations were also run using the PRR of individual stakeholder groups including parents, teachers and students. The school leadership group was not run because of the small N for this group for each school.
Multiple linear regression. The purpose of a bivariate correlation analysis is to determine if two variables have some relationship. The use of a multiple linear
regression analysis determines if several factors might predict an dependent variable. For further analysis of the data, a multiple linear regression was attempted using both the Item Response Equality Method and the Stakeholder-Domain Equality Method of calculating the total PRR. A stepwise multiple linear regression was attempted for each set of data.
CHAPTER FOUR: RESULTS
The purpose of this investigation was to develop an instrument to measure School STEM Culture as defined as the beliefs, values, practices, resources and challenges within a school community as perceived by the parents, students, teachers, and school leadership. Once an instrument is created to measure school STEM culture, schools may use the tool to affect change within the culture, while always keeping in mind that this measurement is based on the perceptions of the stakeholders within the school. The STEM-CAT is an instrument that was designed to measure the STEM culture within a school community. This instrument was created via focus group review of initial items, an expert review of items, and a pilot reliability study of the initial items done within local high school. In the final phase of this study, the concept of school STEM culture construct was supported using eight high schools that agreed to participate in the study, and correlating the results of the STEM Culture Survey to the percentage of seniors at each school reporting an intention to pursue STEM fields upon graduation.
The results of the study align with each of the four phases of the study: • Phase 1- Initial Item Design/Focus Group
• Phase 2- Expert Review • Phase 3- Pilot Study
• Phase 4- Construct Validity