Limitations, practical implications and suggestions for future research 159

types  of  feedback  used  by  those  in  the  population  who  had  earned  at  least  a  Popular   Evaluator  badge,  a  badge  that  required  great  efforts  and  acknowledgment  from  their   peers  to  be  earned.  This  said,  the  sample  does  not  represent  the  population  of  all   those  who  provided  feedback.  However,  on  the  other  hand,  the  analysis  does  take   into  consideration  the  feedback  provided  by  those  participants  who  seem  to  have   fulfilled  the  purpose  of  formative  feedback  with  their  assessments,  and  which  their   peers  considered  meaningful.  One  of  the  main  efforts  that  is  required  to  earn  this   acknowledgement  is  that  of  writing  clearly  and  precisely  (Ferguson,  2011;  Lizzio  &   Wilson,  2008;  Moon,  2005;  Prins  et  al.,  2005),  which  evidently  requires  not  only  a   good  level  of  proficiency  in  English,  but  also  skilfulness  in  its  use.  This  said,  this   analysis  did  not  take  into  account  the  feedback  that  was  provided  by  participants   who  may  have  been  less  able,  less  resourced,  with  less  academic  experience,  or  less   proficient  in  English  (Meek  et  al.,  2017).  This  is  problematic,  as  it  supports  the  claim   made  by  Liyanagunawardena  (2015)  and  Liyanagunawardena  et  al.  (2013)  that   MOOCs  are  indeed  not  accessible  to  all  because  of  the  requirements  they  impose  on   their  participants.  

In  this  case,  it  would  have  been  important  to  take  into  account  the  whole  population   (N=824)  and  other  variables  that  may  reflect  an  association  with  the  peer-­‐feedback  

had  English  as  a  national  language,  participants’  self-­‐assessed  level  of  English,  or   participants’  career  or  educational  level.  A  study  carried  out  by  Meek  et  al.  (2017)   that  took  into  account  the  language  variable  in  the  analysis  of  peer-­‐review  in  a   MOOC,  reported  a  slight  difference  in  the  quality  of  peer-­‐review  provided  by   participants  who  resided  in  countries  where  English  was  the  national  language,   compared  to  those  who  did  not.  That  study,  however,  presented  limitations  and   those  participants  taking  part  in  that  part  of  the  analysis  had  to  meet  two   requirements:  having  submitted  a  summary  assignment  and  undertaken  peer   review.  

Currently,  little  qualitative  research  has  been  carried  out  regarding  participants’   experiences  in  MOOCs,  and  the  few  studies  that  have  done  so  present  two  common   limitations:  the  sample  size,  and  the  sample  selection.  However,  many  of  these   studies  highlight  the  important  role  played  by  language  and  culture  in  diverse  online   learning  communities  (Popov  et  al.,  2012,  2014),  also  in  MOOCs  (Aharony  &  Bar-­‐Ilan,   2016;  Cho  &  Byun,  2017;  Colas,  Sloep,  &  Garreta-­‐Domingo,  2016;  Engle,  &  Carbrey,   2015;  Liu  et  al.,  2014;  Liyanagunawardena,  et  al.,  2013;  Nkuyubwatsi,  2014;  Reilly  et   al.,  2016;  Sanchez  Gordon  &  Luján  Mora,  2014).  Thus,  further  research  should  take   into  account  different  samples,  including  participants  from  countries  that  do  not   represent  a  majority.  This  could  give  course  designers  a  better  overview  of  the  reach   of  the  course,  the  diversity  of  participants  and  enable  them  to  identify  possible   needs  or  limitations  they  may  encounter.  It  is,  however,  clear  that  this  approach  may   not  be  feasible  in  courses  with  hundreds  of  thousands  of  participants,  but  perhaps  it  

can  be  done  in  those  of  smaller  range  like  the  one  reported  here,  or  other  emerging   types  as  those  reported  by  Sanchez  Gordon  and  Luján  Mora  (2014).  

Although  the  purpose  of  this  analysis  was  fulfilled,  a  further  step  would  be  analysing   the  structure  of  feedback  provided.  This  way,  possible  patterns  could  be  identified.  

For  instance,  it  would  be  intriguing  to  understand  the  usage  pattern  of  Analysis  and  

Revision  (within  a  message  and  throughout  the  phases).  If  these  were  used  together  

in  a  message  (provided  they  referred  to  the  same  idea)  it  could  mean  that  

participants  did  not  want  to  misspend  any  time  waiting  for  a  clarification,  instead   they  took  the  following  step,  and  they  provided  a  suggestion  –using  Revision–  based   on  their  own  understanding.  This  interpretation  would  support  what  Saunders   (1989)  and  van  der  Pol  et  al.  (2008)  assert  about  peer  assessment  as  a  more  limited   form  of  collaborative  learning,  in  which  a  lower  degree  of  interactivity  is  offered.   This  results  in  the  limited  possibility  for  interactive  construction  of  meaning  and   collaborative  knowledge  construction  (Saunders,  1989).  If  these  two  types  of   feedback  were  used  in  the  same  message  (and  referring  to  the  same  idea),  it  could   reflect  the  need  for  a  quicker  way  of  correspondence,  even  if  later  at  some  point   their  understanding  could  have  been  confirmed  to  be  incorrect.  

Further  research  could  add  another  layer  to  the  analysis,  which  regards  a  

differentiation  for  the  use  of  types  of  feedback  between  both  roles  for  each  of  the   learning  phases.  Although  this  information  was  available  for  this  study,  it  was  not   required  for  answering  the  research  question.  However,  as  the  topic  related  to  the   requirements  associated  to  the  SA  arose  during  discussion,  it  would  have  been  useful  

to  have  this  information  in  order  to  understand  the  use  of  types  of  feedback  that   appear  easy  to  use  and  that  seem  not  to  require  much  time,  such  as  Praise  -­‐  general  

or  Criticism  -­‐  general.  It  was  tempting  to  assume  that  these  two  were  used  the  most  

in  the  last  phase  by  supporters  in  order  to  fulfil  the  requirements;  however,  there  is   no  evidence  for  this.  This  is  to  say,  though,  that  adding  another  variable  to  the   analysis  also  adds  another  layer  of  complexity,  implying  that  different  statistical   models  would  need  to  be  employed.  

5.1.3   Differences  in  the  use  of  feedback  between  female  and  male  participants  in