Trustworthiness in the qualitative analysis

Trustworthiness was addressed through prolonged engagement with the data, data triangulation, returning to the raters for clarification, documenting decisions, peer debriefing, and use of rich descriptions in the presentation of the findings. In qualitative analysis, the data are filtered through the researcher’s own perspective, based on the researcher’s personal experience, preconceptions, and qualifications; therefore establishing researcher credibility requires openness around the researcher’s background and reflexivity around the ways the researcher may have influenced the findings (Polit & Beck, 2008).

To this end, I (Manders, the researcher for this secondary analysis of the videos) am presenting a short overview of my background influences here. I have an MA in DMT and counseling and am a board-certified dance/movement therapist and licensed professional counselor. I worked clinically with individuals with ASD of all ages and functioning levels for more than ten years before starting this research. This included work as a dance/movement therapist, behavior specialist, in-home therapeutic staff, and mental health professional using a

variety of therapeutic modalities. I worked with professionals from a range of different treatment modalities and observed clinical successes and failures of various interventions in achieving different types of goals and engagements with specific clients. Throughout this work, I maintained my perspective that DMT and movement-based attunement can promote social engagement in some individuals with ASD. While this secondary analysis was entirely based on the video from the parent study, I became familiar with the context of the video by visiting some of the sessions of the parent study and reflecting on these sessions with the therapists and RAs. I observed, but did not partner with any of the participants in the dyadic mirroring dances. For this secondary analysis, I attempted to be self-reflexive and identify my own biases through some reflective journaling: first about my experiences and presumptions, and later around difficulties in data analysis.

The dependability and credibility of this study was strengthened through several forms of data triangulation including the use of multiple participants, multiple video clips for each

participant, and multiple observers for each clip (Polit & Beck, 2008). The qualitative data included: (a) four to five raters’ descriptions of each of the videos, (b) the researcher’s notes from the selection of the purposive videos (listing the interactive behaviors throughout the full videos), and (c) occasional additional input from therapists and research assistants (RAs). Triangulation between these observers decreased the likelihood that a single individual’s perspective dominated without considering other possibilities. Similar descriptions by multiple observers make it likely that the action occurred and indicate which things a third party observer may find meaningful. Although the format allowed the raters the freedom to describe the movement and interactions in the videos in their own words, the fact that they first completed the quantitative scales may have swayed their observations to the terms, constructs, and inherent interpretations in the set of scales they scored. While this limitation was partially mitigated by not training all the raters on the same scales, this influence must still be acknowledged when interpreting the qualitative findings.

The raters viewed all the video clips of a participant (16 - 34 clips per participant) before moving on to the next participant, giving the rater a form of prolonged engagement with each participant’s movement and interaction patterns. The resulting familiarity with each participant’s movement and engagement patterns allowed them to make comparisons between clips, note differences from the participant’s usual patterns, or go deeper into the specifics of the pattern. As the clips were presented in a random order, the raters did not know which came first and could therefore not accurately describe differences as progress, although a few raters did suggest (incorrectly) that they observed specific improvements over time.

4.4.1 Data analysis

Data analysis for this study involved iterative stages of coding, interpretation, and revisions to the coding and themes. These codes and themes emerged out of my prolonged engagement with the descriptions of the movement and interactions of each participant. I first analyzed the descriptions of the video for each participant individually and then cycled back to the data to look for confirming or disconfirming evidence as new patterns emerged in subsequent cases or in the cross-case analysis. This was a messy process with uncertainty, ambiguity, and readjustments to the analysis in iterative stages of analysis. This is consistent with emergent processes of qualitative analysis in general when the methods are left open enough to be responsive to the data itself (Higgins, Barker, & Begley, 2008; Miles, Huberman, & Saldana, 2014).

The large scope of the third research question and the limited consistency and agreement in the rater’s reporting on these movement features, made the qualitative analysis challenging. When I was uncertain how to proceed, I sometimes arrived at new insights into the data and analysis process itself through reflection in the form of memoing, dancing, artmaking, or discussion with experts. For example, reflection on some analytic challenges led me to the realization that I tended to become too micro-analytic and detail oriented while the qualitative

analysis required zooming out and looking at the bigger picture for a holistic understanding of the meaning of the data. This was true in both the selection of interactive sequences for the

purposive sample and the coding process. In coding, this realization led me to reduce the number of codes for specific details of the movement and add more global assessments of the entire set of descriptions for the clip.

The final coding scheme included two levels of coding for each of the segments (see Appendix J for the final coding scheme). One set of codes was applied to the specific phrases in the descriptions that were relevant to that code. These were used in developing the themes, searching for confirming and disconfirming evidence, and, in some cases, for quantifying the data and examining patterns in the frequencies of the codes. The other set was applied to the entire segment, using all the descriptions to assign a holistic assessment of the interaction. These global codes were inserted into tables for each participant and visually inspected for patterns in

comparison to the quantitative results (see Appendices E to I for each participant’s tables). During coding, I expanded the movement specific descriptions for each level of the Flow of the Interaction scale so that this could be coded when the partners were strictly leading and

following.

4.4.2 Peer reviewer and second coder

A peer reviewer and second coder examined a subset of the data to support the confirmability and credibility of the coding. Relatively early in the coding process, a second coder and I (Manders) independently assigned Affective Engagement and Flow of the Interaction codes to the descriptions of one participant’s videos. We then discussed each coding until we reached agreement. I used peer debriefing later in data analysis to provide a partial audit of my analysis and coding decisions. This peer was selected for her relevant expertise with DMT for individuals with ASD, movement observation, and mirroring. The peer reviewer met with me twice for 2.5 hours each and was not otherwise involved in the study. The peer reviewer

examined all my coding decisions for two video segments. We discussed each coding (or lack of coding) until we reached agreement. The peer agreed with almost all of the codes assigned to the descriptions of the video and suggested coding a few more short phrases with additional codes. After discussion, the peer reviewer agreed with my decision to code less and stay more focused on the research questions. This peer then reviewed two more documents for specific codes that were more challenging to code. The peer examined the codebook for completeness and relevance given the research questions and the raters’ descriptions in the documents she had reviewed. While she recommended coding for a couple additional movement features given her knowledge of the population and movement observation, she concurred with the decision to pass on these in this study given the low inter-rater agreement on many movement features and the scope of the research questions. She also examined my expanded movement descriptions for the Flow of the Interaction and determined that given her experience with movement and mirroring, these appeared consistent with the original scale.

4.4.3 Presentation

Qualitative research studies use thick description to give the reader a clear picture of the research context, the participants, their movement, and the patterns in their interactions (Polit & Beck, 2008). In a qualitative descriptive analysis, the rich narrative should stay close to the data with limited layers of theoretical analysis (Neergaard et al., 2009; Sandelowski, 2000, 2010). For this study, the rich narrative includes narrative descriptions of each participant’s usual patterns of engagement and quotes from a variety of observers. This should allow the reader to envision the movement and interactions, get a sense of the range of the descriptions, and perceive the link between the descriptions of specific actions and the overall findings. This study aimed to provide sufficient descriptive detail through these quotes and narratives for the reader to personally determine if the interactions and participants described in this small study are sufficiently similar to another setting or individual to be applicable to that situation (Polit & Beck, 2008). The quotes

from the rater’s descriptions of the videos were corrected for spelling and participant pseudonyms were added, but he quotes were not otherwise altered. Since five of the six raters were not native English speakers, they sometimes made grammatical and word choice errors and one rater sometimes wrote a word or sentence in German. The researcher asked the raters for clarification on some of these descriptions and interpreted other descriptions based on the rater’s usual comments and observations.