Syllabus Content Analysis

At the beginning of the semester the sample of publicly available design studio syllabi was collected for studio courses taught at A&M, UKY, and KU.

A number of books and articles that detail the review of syllabi and how to describe the information gathered in this process were useful in analyzing this data. The

discussed the various types of data that might be generated from a comprehensive review of syllabi including student learning outcomes, descriptions of learning activities, and the methods of assessment (Stanny, Gonzalez, & McGowan, 2015). This approach was coordinated to literature obtained from the Association of American Colleges and Universities (AAC&U) and to personal conversations with Tara Rose and Brandon Combs at the University of Kentucky Office of Institutional Effectiveness and the Office of Assessment. Together, these sources provided detailed and comprehensive

information related to the AAC&U rubrics, methods for developing an assessment rubric, and for analyzing and interpreting this information.

These approaches further articulate the higher levels of learning (Bloom, 1956) and the high-impact educational practices that are advocated by the Association of American Colleges and Universities (https://www.aacu.org/resources/high-impact- practices). High-impact learning is an alignment between learner-centered activities and university/institutional priorities that meaningfully contributes to cumulative learning experiences (Kilgo et al, 2015). This type of educational model enables the students to translate knowledge gained in one context and apply it to problems outside of the original context. Research demonstrates that “First-Year Experiences, Common Intellectual Experiences, Learning Communities, Writing-Intensive Courses,

Collaborative Assignments and Projects, Undergraduate Research, Diversity/Global Learning, Service Learning, Community-Based Learning, Internships, Capstone Courses and Projects” result in deeper learning, student engagement, and academic retention (AAC&U, 2016).

III.3.1.1 Justification

Completing the content analysis of syllabi enabled the researcher:

• to better understand the types of materials that were distributed to students at the beginning of the semester,

• to identify the common materials, themes, and information,

• to determine the language used to frame the methods of working in the studio—individually or in teams,

• to achieve a better understanding of how the problems were situated— hypothetical or real,

• to determine critical intersections or tangents in design thinking at each of the centers, and

• to examine the parallels between student learning outcomes (SLO) and NAAB accreditation student performance criteria (SPC) (NAAB, 2014). III.3.1.2 Population and Sample

The sample included all of the syllabi for design studio courses offered in the spring 2016 semester at TAMU, UKY, and KU. These are publicly available by law. III.3.1.3 Qualitative Data Analysis Tools

The public documents were analyzed using Atlas.ti, content analysis software. The research followed the coding procedures outlined by Saldaña (Saldaña, 2009). Using the syllabi as raw data, coding proceeded in four phases: an initial coding (to discover common themes based upon first impressions of the materials), a process coding (that

organizational coding (that developed explicit categorization of coded data and then refined the coded data using sub-categorization), and a thematic coding (that brought meaning to the content analysis) (Saldaña, 2009). Content analysis of the syllabi produced data to support interpretation. This interpretation defined the educational methods and articulated common studio themes related to categorization of studio type, project type, and student learning objectives.

III.3.1.4 Procedure

With permission from the Department Head/Director/Chair of each program, the offices of student services at each institution were contacted. During this contact, architectural design studio syllabi were requested for studios that were being offered in the spring 2016 semester. These materials were conveyed through email and saved to both a password protected local computer and a secure hard drive server.

A content analysis of the syllabi was conducted to produce data and evidence that define the educational methods employed. A syllabus review rubric outlined by Stanny (Stanny, Gonzalez, & McGowan, 2015) was used to outline the required components and best practice components. A supplemental measure that identified learner-centered approaches (O’Brien, Millis, Cohen, 2008) for active and engaged learning strategies that promote student success was also included. The items that were identified as being the required components, best practice components, or learner-centered components are provided (Appendix 05). The identifiers used for the initial assessment were coded as: present, partially-present, or absent.

An additional rubric developed for this research project using Brookhart’s approach for formative assessment (Brookhart, 2013) was used to better understand the proposed learning targets in design studio curriculum. This student learning outcomes rubric, comprised of the attributes: specific, realistic, observable, and outcome-based was used to identify learning targets for the studio syllabi. The identifiers used for the initial student learning assessment were coded as: present, partially-present, or absent.

The coding and data analysis revealed student learning objectives, studio and project type, design thinking skills, graphic and written communication skills. The coding of the syllabi adhered to the method outlined by Grauerholz & Gibson for specific university required course materials and identified faculty dependent

pedagogical strategies: readings, field trips, guest speakers, and reflection (interpreted for this research as desk crits and design reviews/juries) (Grauerholz & Gibson, 2006).

Atlas.ti was used to explore, analyze, and code documents by connecting various source files together using a hermeneutic editor. The hermeneutic interface was used to organize the core content using principles of interpretation that included: quotations, codes, memos, and networks using color-coding in the Code Manager to visualize relationships. This coded data was analyzed in the Network Editor.

Five distinct categories of learning objectives emerged from the content analysis of architecture design studio syllabi:

Design:

1. The student raises the appropriate design questions in response to a given design challenge—site, environmental condition, and material parameters and other design constraints included in a project brief

2. The student effectively references and evaluates design precedents that relate to a given project

3. The student establishes and compares evaluative performance criteria and then synthesizes them iteratively, to result in an architectural design solution.

Communication:

4. The student communicates effectively with appropriate written materials that describe the project and design solution.

5. The student conveys information accurately using the appropriate graphic and representational media.