Critical reflection on the methodology and methods employed 88

This  study  claims  to  adhere  to  the  constructivist  inquiry  paradigm.  The  paradigm   presents  three  basic  requirements  (Lincoln  &  Guba,  1985),  which  are  considered  to   be  fulfilled  in  this  study.  First,  this  study  has  been  purposefully  planned,  for  which  a   research  proposal  was  created  and  accepted.  Second,  a  digital  research  audit  trail   was  created  for  the  study  as  a  means  of  maintaining  transparent  documentation   throughout  the  different  stages  of  the  research.  Third,  the  research  study  was   carried  out  consistently  with  the  ontological,  epistemological  and  the  five  axiological   assumptions,  as  suggested  by  Lincoln  and  Guba  (1985),  and  which  are  exemplified   next.  The  course  was  studied  in  its  natural  context,  and  examined  from  a  holistic   perspective.  On  the  one  hand,  the  researcher  was  familiar  with  the  way  in  which  the   courses  were  designed  and  run.  This  insider  insight  allowed  the  researcher  to  focus   on  the  observation  of  the  phenomena  of  interest  of  the  course  and  immerse  herself   in  the  development  of  the  course.  On  the  other  hand,  as  a  course  participant,  the   researcher  was  in  the  same  conditions  as  all  other  participants.  This  implied  the  need   of  being  involved  –even  as  an  observer–  in  the  topics  treated  in  the  course  and  in   different  spaces  in  order  to  be  able  to  contribute  to  the  course  in  many  forms  (e.g.   providing  feedback),  if  desired.  The  role  performed  enabled  the  researcher  to   engage  in  the  culture  of  the  analysed  course  and  get  insights  that  would  not  have   been  possible  otherwise.  

Various  forms  of  data  were  collected  to  complement  and  corroborate  observations,   while  at  the  same  time  capturing  different  perspectives.  Findings  are  represented  by   participants’  contributions  during  the  course  that  reflect  the  social  reality  that  was  

created  and  experienced  there.  The  study  of  an  online  learning  event  as  it  developed   in  real  practice  was  chosen  to  contribute  to  the  understanding  and  analysis  of  a  real   situation  where  peer-­‐feedback  is  used  as  a  method  of  assessment  despite  the  unique   characteristics  of  such  an  online  course.  Besides  contributing  from  a  theoretical  and   methodological  point  of  view,  conclusions  emergent  from  the  case  study  may  prove   useful  for  the  enhancement  of  future  MOOCs,  where  approaches  of  assessment  of   this  kind  are  adopted  or  planned  to  be  adopted.  

This  study  continues  the  exploration  of  a  previous  study  undertaken  by  the   researcher  in  which  the  types  of  peer-­‐feedback  in  a  similar  setting  were  studied.   Therein,  working  hypotheses  were  created  that  were  used  as  a  basis  for  this   research.  

The  third  and  fourth  phases  of  analysis  of  the  study  used  the  quantitative  output  of   the  analysed  data  to  report  on  possible  differences  in  the  use  of  types  of  peer-­‐ feedback  in  relation  to  reviewer’s  role  and  to  reviewer’s  gender  within  their  role.   However,  adhering  to  the  understanding  that  ‘causal  linkages’  cannot  be  identified   (Axiom  4  -­‐  Lincoln  &  Guba,  1985),  the  results  are  used  to  describe  possible  

differences  as  occurred  in  the  specific  studied  event,  without  intending  to  draw   conclusions  about  the  causes  and  effects  of  the  variables  analysed.  

Finally,  it  can  be  said  that  the  research  is  value-­‐resonant,  as  the  problem  of   investigation,  the  context  in  which  the  problem  was  studied,  the  theoretical  

perspective  (sociocultural  theory)  under  which  the  problem  was  analysed,  together   with  the  research  paradigm,  are  all  connected.  

The  content  analysis  method  was  used  as  the  principal  method  for  the  analysis  of   data.  Besides  its  clear  suitability  for  the  purpose  of  the  study,  it  is  a  naturalistic   method  that  supports  interpretive  approaches.  The  content  of  text  data  is  

interpreted  subjectively;  however,  it  follows  a  systematic  classification  process  of   coding  (Hsieh  &  Shannon,  2005).  Qualitative  content  analysis  was  employed  for  the   main  research  question,  where  the  purpose  was  to  identify  the  nature  of  feedback  in   the  course.  To  achieve  this,  emergent  feedback  types  along  with  examples  that   would  reflect  the  social  world  as  constructed  in  the  course,  are  presented.  The   quantitative  output  of  the  qualitative  approach  was  employed  for  the  second,  third   and  fourth  phases  of  analysis,  in  which  the  focus  was  placed  on  the  description  of   the  frequencies  of  use  of  the  different  types  of  feedback  in  relation  to  the  variables   of  interest  for  the  study.  

Although  content  analysis  has  been  described  as  a  difficult,  frustrating  and  time-­‐ consuming  method  (e.g.  Rourke,  Anderson,  Garrison,  &  Archer,  2001),  it  is  also  a   gratifying  method  in  which  informed  decisions  are  made  based  on  the  results  of  long   and  rigorous  processes.  The  amount  of  data  was  rich  and  abundant  and  the  time  for   its  analysis  was  long  and  pauses  of  up  to  two  weeks  were  made.  However,  in  order   to  maintain,  in  the  best  possible  way,  a  constant  quality  of  analysis,  the  following   procedure  was  undertaken:  after  the  time  of  pause  and  before  taking  up  new  data  to   analyse,  a  randomly  selected  segment  was  re-­‐coded.  Both  codified  versions  were   compared  and  when  inconsistencies  were  found,  then  the  descriptions  within  the   created  guideline  would  be  revised  until  reaching  the  same  understanding  as  when   they  were  coded  in  the  first  version.  

The  quality  of  the  study  is  discussed  according  to  criteria  suggested  for  the  

constructivist  paradigm,  which  can  be  summarised  in  three  items:  correspondence   between  research  problem  and  research  design,  demonstration  of  trustworthiness   (rigour),  and  the  contribution  of  the  study’s  results  to  the  community  (Lincoln  &   Guba,  1985).  

As  stated  in  the  first  section  of  this  chapter,  the  focus  of  study,  the  context  in  which   the  study  was  carried  out  together  with  the  constructivist  inquiry  paradigm,  are   congruent.  

Rigour  (trustworthiness  and  authenticity)  can  be  achieved  when  observing  the   quality  criteria  that  have  been  defined  for  the  qualitative  approach.  Creswell  (1998)   proposed  the  implementation  of  at  least  two  of  eight  techniques.  These  techniques   go  in  line  with  the  four  criteria  presented  by  Lincoln  and  Guba  (1985),  namely:   credibility,  transferability,  dependability  and  conformability.  There  were  six  

techniques  implemented  in  this  study:  prolonged  engagement,  triangulation,  peer-­‐ debriefing,  thick  description,  the  compilation  and  maintenance  of  a  research  audit   trail,  and  a  reflexive  journal.  

Finally,  different  groups  of  the  community  can  benefit  from  the  results  of  this  study.   The  contributions  that  are  claimed  to  be  made  with  this  study  have  been  specified  in   Chapter  1,  sub-­‐section  1.4.