Empowering Student Learning in the Geologic Sciences with Three-Dimensional Interactive Animation and Low-Cost Virtual Reality (NSF 05-36739)
Progress Report: May 1, 2006 – December 31, 2006 Dr. Laura M. Leventhal and Dr. Dale Klopfer
Bowling Green State University
Project Duration: May 1, 2006 - December 31, 2007 Introduction
The primary goal of this project is to develop, explore, and evaluate the usefulness of interactive three dimensional models (3DIA) in college-level geology education. Specifically, the models are used to teach about topographic maps and how to use them to visualize topographic profiles, a task that is thought to use spatial skills. Individuals vary greatly in how well their spatial skills are developed; of particular interest here is the impact of 3DIA on students with poor spatial skills (as measured by psychometrics tests of spatial ability).
This project consists of two studies. In the original proposal and the approved modifications, these were referred to as Study 2 and Study 3. For clarity we will continue with that nomenclature. In Study 2 we are developing and testing (in the laboratory) a 3DIA prototype that supports instruction in the interpretation of topographic maps and profile analysis. In Study 3 we will develop, implement, and evaluate an instructional unit that uses 3DIA to teach students about topographic maps and profiles within the context of an introductory geology course taught at BGSU. The outcome of Study 3 will be evaluated using the learning goals for the course.
Process and Timeline
Our project began on May 1, 2006. Our team consists of the two PIs, our Geology consultant, Dr. Onasch, our computer graphics consultant, Dr. Zimmerman, a Computer Science graduate assistant and a psychology graduate assistant. The computer science graduate student was assigned to develop the software for both Study 2 and Study 3 as well as to oversee some early usability testing of the software. The psychology graduate student is overseeing final pilot work and actual data collection for Study 2. We also have two undergraduate psychology majors who are helping with data collection and who have helped with stimulus preparation.
Status of Study 2
One goal of Study 2 is to determine whether allowing students to manipulate (virtual) 3D models of terrain (and portions of the terrain) enhances learning of how to visualize topographic profiles, thereby improving performance on such tasks. A second goal is to determine whether improvement on the profile task that is occasioned by the use of 3DIA is inversely related to levels of spatial ability such that persons with low spatial ability improve more than those with higher spatial ability.
Work on Study 2 has progressed in three phases thus far.
Phase 1: Development of materials. (May 1 - September, 2006) During this phase, we developed the 3DIA software, the user interface (including data collection procedures), procedures for generating stimulus materials, and an initial set of 200 stimuli. In addition, we refined the experimental design and developed training materials for the study, and developed background questionnaire to administer to study participants. In late July and August, we performed three usability tests of the software using Computer Science graduate students who were naïve as to the purpose of our study.
Phase 2: Pilot testing and refinement of materials. (September 1 - November 20, 2006) During this phase, we ran a number of pilot studies of the software and experimental materials. The most informative pilot testing we did was with Geology graduate students, most of whom serve as Teaching Assistants for GEOL 104, the course in which Study 3 will be situated. For the most part, these pilot tests followed our experimental protocol, but some included a talk-aloud element as well in which the students told us what they were thinking as they interacted with the software. After testing the software and protocol with students who were highly familiar with topographic maps, we also tested two undergraduate psychology students who had never taken GEOL 104 and had little to no experience with topographic maps.
Phase 3: Data Collection. (November 20, 2006 – ongoing). Our data collection began officially on November 20th and is on-going. We are soliciting participants through the Psychology Department’s web-based research participation tool, experimetrix. As of the date of this report, we have collected data from 11 participants. Data collection was halted during the break between semesters and resumed during the first week of January.
Status of Study 3
Concurrently with the software development work on Study 2, our graduate assistant, under the direction of Dr. Zimmerman has been developing the software to use in Study 3. The software unique to Study 3 shows how contour lines are “painted” onto landforms, and it allows the user both to select which contour lines are to be painted and to rotate the painted landform so that it can be seen from above. The 3D interactive nature of this animation, we believe, will provide a deeper understanding of how topographic maps are made than can be gleaned from static or non-interactive presentations. The instructional unit that will be tested in Study 3 involves combining the “painting” software with the presentation of topographic profile problems to solve, as in Study 2. The prototype software for Study 3 has been completed and was presented in a Computer Science Department Colloquium on December 1st. Attached as Appendix A is the report on this part of the project, written by the graduate assistant, Mr. Jeries. In addition, as part of Study 2's stimulus materials, we have begun to pilot test activities for the Study 3 assessment, the Topo Test.
Details of Status
In subsequent sections of the report we will report on our progress and finding from each of these activities. Experimental Design for Study 2
In order to evaluate the usefulness of the 3DIA tool in a profile analysis task, our original experimental design had independent variable of
o 3DIA vs. control (no 3DIA) vs. 3DIA with reflection
In addition, because the performance on profile analysis is thought to vary with stimulus complexity and subject spatial ability, we also proposed independent variables of
o Subject spatial ability (low vs. high) o Complexity of the stimulus materials
Finally, we initially thought to vary the memory demands that the visualization presents by varying the number of slices in the 3DIA
In our current experimental design, we have retained the following independent variables o 3DIA vs. control (no 3DIA) vs. 3DIA with reflection
o Subject spatial ability (low vs. high) o Complexity of the stimulus materials
To increase external validity and to help us better understand subject strategies, we have added several naturalistic stimuli. These materials will be part of our assessment for Study 3 (Topo Test) ultimately as well, so we are also pilot testing them for that role at this time. In addition, we have elected to collect some background demographic and experience information from our participants. Based on the complexity of our current design, we have opted to not investigate the role of number of slices at this time.
Stimulus Materials for Study 2: Profiles and 3D Models Varying in Complexity
Two types of stimuli were generated for the study: artificial landforms and naturalistic ones. The steps involved in generating the artificial landforms and profiles are described next, followed by a description of how the naturalistic stimuli were made.
Profile Complexity. Through our many discussions with Dr. Onasch, a structural geologist who serves as our consultant, we concluded that there are several variables that determine the complexity of a topographic profile. The primary variable is the number of inflection points in the profile, or the number of hills and valleys. A second factor is local symmetry in the profile: it is easier to visualize a profile of a symmetrical hill (or valley) than one where the ascending slope is much steeper than the descending slope. A third factor in defining stimulus complexity is the location of the prominent topographic feature: when the prominent feature is located at the center of the profile, the profile is easier to visualize than when the prominent feature is located at either end. This factor reflects the global symmetry or redundancy of the profile. Finally, it was felt that hills (convexities) are easier to visualize from topographic maps than valleys (concavities). Incidentally, the first three criteria (number of inflection points, local symmetry, and global symmetry) are similar to those used by perceptual psychologists to characterize the complexity of form and shape. The fourth may be related to the finding that mental images in depth are created from “near” to “far” in the mind’s eye. As described below, these criteria were used to choose the 24 artificial stimuli used in the study.
Profile Generation. We started with five levels of the number-of-hills-and-valleys variable: (a) 1 peak, no valley, (b) 1 peak, 1 valley, (c) 2 peaks, 1 valley, (d) 2 peaks, 2 valleys, and (e) 3 peaks, 2 valleys. For each level, we then generated prototype profiles that resembled sine waves that varied from half a period (1peak, no valley)
to two-and-a-half periods (3 peaks, 2 valleys). Next, we created variations of each prototype by randomly shifting (subject to a set of constraints) both the elevations of the beginning and end points of the profiles and the elevations and horizontal positions of each maximum and minimum (where appropriate) point of the prototype. Ten quasi-random variations of each prototype profile were created in this manner, and each of these variations was inverted, reflected, and inverted-and-reflected to yield 40 profiles for each of the five prototypes. A sample profile from each of the 1-peak-no-valley, the 2-peaks-1-valley, and the 3-peaks-2-valleys families is shown below. (The abscissa shows horizontal position, from Point A to Point B; the ordinate shows elevation.)
1 peak, no valley 2 peaks, 1 valley 3 peaks, 2 valleys
Profile Selection. Rather than use all five levels of the number-of-hills-and-valleys variable we selected the lowest, medium, and highest levels (1-peak-no-valley, 2-peaks-1-valley, 3-peaks-2-valleys) to use as stimuli in Study 2. This was done because it allowed for the instructional unit planned for Study 3 (which incorporates aspects of Study 2) to be no more than 45 minutes in length.
With 40 profiles available for each peak-valley level, we asked Dr. Onasch to select a subset of “easy” and “difficult” profiles from each level to use in the study and to explain why he selected those that he did. As described above, he mentioned that symmetry of the hill(s) and valley(s), the location of the prominent feature, and whether the landform was a hill or a valley were the features that he used to make those discriminations. We treated those factors as binary features (i.e.., symmetric/asymmetric, center/off center, and convexity/concavity) and, where possible, selected eight profiles from each level that would allow us to contrast those features empirically, thereby validating their contribution to an overall measure of complexity. Eight profiles were selected from each of the three peak-valley levels for a total of 24 profile stimuli.
Generation of 3D Models. Each profile is a cross-section, representing elevations along a given, say, west-east axis. Each profile was characterized as a concatenation of interpolated cubic polynomials, and, from those polynomials, 25 (x,y) values along entire profile were generated. In order to create a 3D landform, the nature of the terrain running north-south needs to be specified. Accordingly, a profile of a hill was created, and, from that polynomial describing the profile, 25 (z,y) values were generated. The 25 points in the west-east dimension were multiplied by the 25 points in the north-south dimension, giving 625 (x,y,z) elevation points. For each profile, the array of 625 (x,y,z) points was input into Surfer 8 software to generate the 3D model (a 10,000 element array of elevations along the west-east and north-south directions) and the topographic map of that 3D model.
Dr. Onasch proposed using natural stimuli in the study, viz., profiles and 3D models created from sections of actual terrain. He selected six regions whose profiles had many more low amplitude changes in elevation, less symmetry, and less clear identification of main features than the stimuli we created. Similar procedures to those described above were used to create the topographic map and the 3D model. As these stimuli are representative of the types of profiles that can be seen in the real world, we view performance on these naturalistic stimuli as a test of how well students learned how to do profile analysis with the artificial landforms. Accordingly, we placed the naturalistic stimuli at the end of the experimental session. We plan to use performance on these items as one means of evaluating the effectiveness of our 3D interactive animated instructional module.
Task for Study 2
The basic task of mapping changes in elevation shown on a topographic map to a 2 dimensional profile of those changes seem to require visualization and speeded rotation, two components of spatial ability. First, it seems that a bird’s-eye view of the surface of the terrain along the profile line is visualized from the spacing of the contour lines and labeled elevations. Next, it seems that the visualized surface has to be rotated 90 degrees, going from a
bird’s eye to a ground-level view. Certainly, the profile task may also be accomplished analytically if enough numeric data are included in the profile and contour map, but it was our intention to create a version of the task that would encourage participants to use a spatial rather than an analytic solution.
The specific task and procedure we developed is very similar to the one described in the original proposal. There are three screen regions, Windows A, B, and C. Window A shows the topographic map, the profile line, and three vertical lines dividing the upper half of the map into four segments. Window B contains the 3D model seen from above, with four distinct segments, capable of being rotated 90 degrees, that correspond to the segments in the topographic map in Window A. Figure 3 shows screen shots of Windows A and B. Finally, Window C shows four profiles, one of which is the correct profile and three that are incorrect. The participant selects the option in Window C that he or she thinks is the correct profile for the path seen in Window A. To discourage participants from going back to Window B and rotating segments after seeing the response options for a given trial, Window B was closed whenever C was open. Figure 4 shows a screen shot of Windows A and C.
The sequence of events in a typical trial is as follows: First, a topographic map (including labeled contours), a profile line, and lines dividing the upper portion of the map into four segments are presented in Window A. The participants’ task is to select a profile in Window C that matches the profile along the line shown in the map. Accordingly, they may opt to go directly to the multiple choice responses in Window C to make a selection or they may choose to view in Window B the four-segment, interactive 3D model of the landform that is represented by the top half of the topographic map shown in Window A. After they interact with the 3D model in Window B, rotating segments of the model to see the ground-level view, the participants will open Window C and make a selection.
In Window B (which contains the interactive 3D model), the participant is able to rotate slices of the 3D model that correspond to the segments shown on the topographic map. The participant can select any slice and rotate it up to 90 degrees upwards by dragging the mouse along the slice. Rotating the slice reveals the actual profile for the corresponding portion of the map. When the participant selects another slice, the previously-rotated slice snaps back to the original position, so the entire profile of the 3D model cannot be viewed at once. The slices may be rotated in any order, and for as many times as desired. The program keeps track of which slices are rotated when, how long the profile segments are viewed, and how far the slices are rotated (up to 90 degrees).
Figure 3 shows how contents of Window A and B are linked. After the map is shown in Window A and if the viewer chooses to view the 3D model, Window B opens, showing a bird’s eye view of a 3D model of the terrain shown in the top half of the map. When the viewer drags the mouse vertically across slice of the model – here, the second one from the left – the slice rotates, revealing the actual profile of that slice. In Figure 3, the slice has been rotated about 45 degrees, revealing the general shape of the profile. Whenever a slice is rotated, the corresponding segment on the map is highlighted with a red border. Dragging the mouse across another slice would return the first slice to its starting position and allow the rotate-and-reveal process to be repeated with the new slice.
Figure 4 - Window A and Window C (showing the four response options)
As Figure 4 illustrates, the response options appear in Window C; the correct profile for this example is on the far right. Although not the case with the sample trial shown in Figure 4, we generated three types of incorrect options for each trial: profiles that are incorrect because they contain either one too many or one too few hills (the second profile from the left), profiles that are left-right reflections or inversions of the correct profiles (not shown here), and profiles that were deemed moderately similar to the correct ones (the profile on the far left). Because each profile can be represented by a series of (x,y) coordinates, the product-moment correlation between any two profiles can be computed and can serve as a measure of similarity. The moderately similar profiles were those that correlated between 0.60 – 0.80 with the correct responses. The order of the responses was randomized across trials.
The choices of colors for the 3D model in Window B were a result of our pilot testing, as was the decision not to include elevations or gridlines in the response options in Window C.
Our current procedure with each research participant in the experimental condition is as follows.
o Participants go through an interactive Powerpoint training presentation designed to teach students about topographic maps and the profile task. Many elements of the training were taken from the training materials used in the unit on topographic maps in the introductory geology lab course. The Powerpoint presentation also introduced the participants to the software and the task that they will be asked to complete.
o Participants are guided through a practice trial to make sure they understand the task and how to use the software.
o Participants complete the 24 trials of artificial profiles (eight at each level of complexity), followed by the 6 trials with the naturalistic stimuli. We log their answers to each of the questions in addition to all mouse events and duration of each mouse event.
o Participants complete a two-page background questionnaire.
o Participants take standard paper-and-pencil psychometric tests of spatial ability so that we can look at how people with different spatial abilities perform the task.
o We ask for permission to look up participants’ GPAs and SAT/ACT scores. We will use these data, where available, to control for general scholastic ability.
Subjects in the control condition go through the same procedure. The only difference is that the rotation feature of the 3D model in Window B has been disabled. So essentially, subjects in this condition can see the bird's eye view of the landform but cannot rotate it. Participants in the (3DIA + reflection) condition will follow the same procedure as those in the experimental condition except that while they are doing the profile analysis task, they will be asked to talk aloud and to reflect on their actions.
At this point, we have run 20 participants through our procedure. Ultimately we hope to run 72 participants or 24 per condition (12 low spatial and 12 high spatial ability subjects in each condition). Recruiting participants has proven to be more difficult than had been anticipated.
We expect that accuracy in the 3DIA condition will be better – and the difference in performance between high and low spatial participants to be smaller -- than in the Control condition. With respect to use of the 3DIA interface, we expect that low-spatial subjects will use it more than high-spatial subjects, that use of the reveal-and-rotate function will increase as problem complexity increases, and that there will be a greater increase in the use of the rotate-and-reveal function with complexity for low- than high-spatial subjects. We expect these results because the task makes more demands on working memory as complexity increases. Displaying a part of the profile on the screen reduces memory demands, which allows effort to be allocated to visualizing other slices. Low-spatial individuals will off-load their working memories sooner and more often than high-spatial individuals. We also hope to explore the interaction of experience and demographic variables of the independent variables of the study.
We anticipate that we will complete our data collection by the end of March 2007 and complete our data analysis by May 2007.
Status: Study 3.
At BGSU, an introductory laboratory course in geology (GEOL 104) devotes part of a lab session teaching students to do profile analysis. GEOL 104 satisfies both the science requirement for the College of Arts and Sciences and the university’s General Education requirement. More than 500 students from a variety of colleges and majors enroll in GEOL 104 each year, fewer than 3% of whom are geology majors. The target student population is very diverse. From a practical standpoint, the most important outcome from Study 2 should be that low-spatial individuals, who we expect to do poorly on profile construction when the interactive animation tool is not used, would perform very well when the tool is used. In Study 3, we will demonstrate that incorporating this interactive animation tool in a multimedia package designed to teach students about topographic maps will lead to improved learning and comprehension when compared to the traditional pictures-and-text approach that is currently used.
In Study 3, we will create two 3DIA interfaces. The interactive animations will consist of the rotation tool from Study 2, plus an additional interactive animation in which the landforms can be rotated about the vertical axis. The student picks an elevation and a contour line is “painted” on a landform at that elevation as the landform is rotated. After the landform rotates a full 360o and a contour line has been painted, the student will rotate the model to see the bird’s eye view. After rotating the landform rotates back to the sea level view, the student can select another contour line to paint. After each line is painted, and before the bird’s eye view is made available, the student is asked to anticipate what the bird’s eye view would look like, with special attention drawn to important aspects of the landform (e.g., steepness, ridges and valleys, dual peaks). We believe that drawing attention to those features and how they would look from a bird’s eye view, which is similar to what would be seen in a topographic map.
Thus far our emphasis for Study 3 has been on building the interactive three dimensional paint tool. Our Computer Science graduate assistant, Mr. Jeries has completed the development and testing of this tool and his report is attached as Appendix A.
For Study 3 our plans for next year include:
o Completion of the software for Study 3.
o Development of the Topo Test assessment instrument to use for Study 3
o Development of a prototype web-based deployment tools for the software for Study 3, which includes the tool from Study 3, for dissemination to the Geology education community. o Collection and analysis of data from Study 3.
Explanation of Budget Expenditures to Date
Nearly all of the expenditures through December of 2006 have been associated with personnel costs. Summer salary (1 month for Leventhal, .5 month for Klopfer) and consultant fees ($2000 for Zimmerman, $1400 for Onasch) for faculty totaled $16,758. Costs for Graduate Research Assistants in Computer Science and Psychology totaled $10,782, and $547 was paid to an Undergraduate Research Assistant. Equipment/Supplies purchases (magnetic media for data storage) amount to $81. Fringe benefits ($3,126) and Indirect Costs ($11,922) were also charged to the grant. Altogether, $43,216 was spent from May 15, 2006 through December 31, 2006.
Anticipated Budget Expenditures in 2007.
As indicated in the original budget, expenses anticipated through November, 2007 include faculty salaries and consultation fees, costs of graduate and undergraduate research assistants, a desktop computer, software costs, and research participation incentives.
Future Studies and Modifications
During the design of the software and the stimuli used in Study 2, we discovered that there were some aspects of the rendering of the 3D models that might effect performance of the profile task. In particular, the position of the light source in the 3D model affects the type and amount of shading in the model, and shading has long been known to be used by the visual system to compute undulations in surfaces. The 3D models used in Study 2 have minimal shading, making it more difficult to visualize the surface. Accordingly, some participants (presumably those with lower spatial ability) will choose not to try to visualize the surface based on information from the contour lines, opting instead for a more piecemeal, analytic solution strategy. Increasing the amount of shading on the models may encourage these individuals to use visualization to do the profile task, providing something like training wheels for their visualization skills. Consequently, we plan to investigate the role of shading of the 3D model in the future.
In addition, some previous work at BGSU suggests that subjects who interact with 3D images in tasks with a high spatial element, may benefit from a touch based interaction as opposed to a mouse based interaction. This finding is consistent with a distinction that can be made between (visual) imagery that derives primarily from visual processes and (spatial) imagery that derives from perception-action couplings. To the degree that solving a particular spatial problem is based on the latter form of imagery, requiring that appropriate actions be directed to the object of interest should enhance performance on that task.
We also plan to investigate the role of the number of slices (and memory load) on performance of the profile task, as stated in the original proposal. To that end, the stimuli were created so that the slice boundaries would not coincide with local maxima and minima (i.e., the highest elevation of any hill or the lowest point of any valley). As studies in perception have indicated, local maxima and minima carry the most information about the shape of a contour. Consequently, care was taken to ensure that if they were present in the slices of a 4-slice 3D model, they should also be present in the slices of a 6-slice model.