Group Malfunction Videos Usability Report
By: Anna Chiaro November 6, 2019 Revised: November 11, 2019
Introduction
The Pedagogy Kiosk is a collaborative project consisting of a series of short videos detailing problems such as group malfunctions, classroom management issues, and behavioral issues, along with solutions to these problems. These three categories each contain six thirty second videos where members of a Written Professional Communication class at the University of Pittsburgh detail specific examples of a type of malfunction or issue. The videos teach professors and/or anyone involved in a team setting how to create a productive and
functional group environment. Each clip utilizes a 360° camera angle, providing the viewer with an interactive experience where they follow the sound of the actors’ voices, and drag the screen accordingly, giving them control over their own personal viewing experience. A caption detailing a preliminary synopsis accompanies each scene, giving the viewer a further idea about the content the scene depicts. After watching each series, viewers will understand what makes a group unsuccessful, and exactly which adjustments effectively fix arising
problems.
Process
This Pedagogy Kiosk itself results from effective team work on multiple levels.
Four different groups were assigned a different task-- three groups wrote scripts and acted out the videos seen in each category, and one group designed the website housing the videos. We brainstormed scene ideas for one week before pitching them to our professor and obtaining his filming approval. Using stasis theory, we created a powerpoint detailing what exactly “group
malfunction” means, and how we would go about portraying six common group malfunctions along with their solutions. We described group malfunctions as being emergent issues that hinder the group members’ productivity, and pinpointed six specific group malfunctions--the problem student,
communication errors, unsafe space, lack of accountability, lack of cohesion, and lack of motivation. After the successful idea pitch, each of the six team members took initiative in writing out a scene which portrayed one of the six group malfunctions, and then opened up their work to comments and
suggestions from the rest of the team. This process took another week, and really helped create the cleanest, most understandable product we could
produce. By working in a shared Google doc, we left comments and suggestions on each other’s scenes, and made the necessary adjustments before filming during the following week.
Upon arrival at our film time, our professor explained the 360° camera to us, and left the room so that we could film for one hour. The filming process was easy using the 360° camera, as it eliminated flaws attributed to standard
cameras such as starting and stopping the footage, adjusting the camera angle to get everyone in the shot-- no part of the room was left unfilmed--, or recording audio (the microphone was continuously recording on the camera stand). With everyone on our team being on the pre-health track, and with little to no background in film, the simple equipment used in this project helped us focus on perfecting the design and content of the videos, rather than worry about how to capture the moments on camera.
Once we were left with our scripts and the camera, we began filming our scenes one by one. Each director of their scene positioned the actors accordingly and gave a quick rundown of how the scene should go. After the first attempt at filming each scene, everyone gave their input on how they thought the scene went; essentially, we all gave suggestions on areas that could use improvement, such as amount of dialogue, speech fluidity, scene length, arrangement and movement patterns, and conveying of emotions. Getting the first scene just right took about seven or eight tries, but by the time we reached the sixth scene, we all had our trial and error method perfected, and performed the scene in two takes.
This project really acted as a team bonding experience. At the beginning of filming, we were all a little nervous and confused regarding the filming process,
but after spending time in front of the camera together, and helping each other through our mistakes, we started easing into the situation and having a good time with it. By the end of filming, we all teared up at least once after repeatedly delivering humorous lines that we wrote into the scripts with all seriousness.
The hour we spent filming was one of the most productive, and cohesive
moments that our team has spent together, and it all worked because we knew how to work together.
After filming, our professor helped us title our thumbnail images, and cut the scenes into short thirty second clips. The website design group then took the eighteen total videos, and uploaded them to the website that they created for this project. We then added video captions, and the project was complete.
Usability Study
We assessed just how effectively the video series conveyed our ideas with individually created follow up surveys (customized in accordance with each of our scenes) which ask questions about the viewers’ experiences with the video.
My usability survey for the “Lacking Accountability” video in the “Group Malfunctions” category consisted of the following eight questions : 1
1. The video title is clear and easy to read
1 Questions 1,2,4,5,6, and 7 were assessed on a Strongly Agree to Strongly Disagree scale, while questions 3 and 8 were assessed by open response.
2. The video caption provides a helpful description as to what the video will portray
3. Based on your interpretation, please indicate what the video portrays as the problem and the solution
4. The interactive 360 degree camera angle positively enhanced your viewing experience
5. The audio quality is good
6. This video taught you how to fix a group malfunction
7. Would you use/recommend this video as a teaching tool?
8. If you would not use/recommend this video as a teaching tool, what would you change to make this video more usable?
By asking participants these specific questions, I collected responses detailing both technical and design critiques and preferential personal experiences.
I gathered eight total responses from first time viewers,-- all sophomores in college not taking a Written Professional Communication course--asked that each participant answer the survey truthfully and thoroughly, and obtained useful, relevant feedback that I plan on using as a reference during future video editing stages.
Results and Recommendations
The results show mixed expected feedback along with some critiques. The charts below detail percentages of the multiple choice question results:
Question # Strongly
Agree/Agree Neutral Strongly Disagree/Disagree
1 87.5% 0% 12.5%
2 87.5% 0% 12.5%
4 25% 25% 50%
5 75% 0% 25%
6 75% 0% 25%
7 100% 0% 0%
The results are mostly positive with the exception of the question asking about the effectiveness of the 360° camera. One participant left answered question #8 saying, “There really is no reason for the 360° angle, and it would be easier to watch if everyone was just in the same shot.” This feedback shows that 2
although the 360° camera makes the filming process easy for the actors, it may not prove usable for the viewer.
Written feedback from question #3 shows that, on average, participants understood the problem and solution depicted in the video, as proven by responses like, “The problem: One student doesn't want to help; the solution:
another student reminds the first student that its a lot of work and if they help, the project can get done more efficiently,” and, “One member of the group does not participate in working on the project, but holding that member accountable managed to encourage her to help get the project done more efficiently,”-- two responses that mirror the same idea as 75% of the total responses to this
question. A negative response warranting editing consideration on our part is,
“idk i couldn’t hear”. This response provides that we may consider editing the sound quality in possible future recording sessions.
2 Other participants’ responses included, “i like 360 but the sound is trash and the quality is pixel,”
meaning we might consider using higher quality audio and video in future video recording or editing.
Appendix
Training Manual: https://www.pitt.edu/~anc214
Usability Survey:
https://docs.google.com/forms/d/e/1FAIpQLSfvcowXQwCEWFe0R-4f1etmhXP4 Fx8zmtcmyfLpNZ1RwUyw9A/viewform?usp=sf_link
