Expanding Assessment of Analy3cal Skills among Biology Majors: From Introductory labs to Upper Division Elec3ves

Full text

(1)

Expanding  Assessment  of  Analy3cal  

Skills  among  Biology  Majors:  

From  Introductory  labs  to  Upper  

Division  Elec3ves  

Presented  by  Kathleen  McAuley  

PI:  Serena  Moseman-­‐Val3erra,  Ph.D.  

(2)

Who  we  are  

Serena  Moseman-­‐Val3erra  

New  Assistant  Prof.  in  Biological  Sciences   Instruc3ng  BIO  102  in  Spring  2013  

Very  sorry  to   miss  the   conference!  

A  Team  photo  from  the  Na3onal  Academies   Summer  Ins3tute,  June  2010.  

Faculty,  Instructors,  and  Administrators  from   Boston  College,  Yale,  Ohio  State,  U  Wisconsin-­‐ Madison    

Kathleen  McAuley  

Undergraduate  Assistant  

URI  2012,  B.S.  Secondary  Educa3on   and  B.A.  Biology  

(3)

General  Goals  

I.  To  iden3fy  common  learning  objec3ves  for  

BIO  101    (Introductory  Biology,  semester  1)  

II.  To  establish  an  approach  for  assessing  data  

analysis  skills  at  both  introductory  and  upper-­‐

levels.    

– 

 BIO  101  laboratory  sec3ons  

– 

 BIO  262  (Ecology)  

(4)

I.  Overview  of  Ac3vi3es  

Mee3ng(S)  with  four  BIO  101   Instructors  

For  Common  BIO  101  Learning  Objec3ves:  

Introduced  idea  of  a  formal,  common  assessment  

Collected  recent  final  exams  from  each  instructor  (or  midterms  from  new  instructors)  

Fall  2011  

Analyzed  prior  final  exams  to   iden3fy  common  learning  

goals:  

-­‐Categorized  each  exam  ques3on  in  terms  of  a  learning   objec3ve  

-­‐ Compared  each  ques3on  to  an  extensive  list  of  possible  

 objec3ves  from  a  colleague  in  the  Undergraduate  Educa3on  in    Biology  Scien3fic  Teaching  Summer  Ins3tute  

-­‐Tallied  ques3ons  in  each  category  

-­‐Iden3fied  areas  of  overlap  among  the  4  instructors    

Found  6  common  objec3ves!   (in  next  slide)      

(5)

Learning  Objec3ves  in  BIO  101  Assessment  (Fall  

2011)  

1)  Dis3nguish  between  a  living  and  non-­‐living  

organism  

2)  Predict  direc3on  of  water  movement  in  

diffusion  across  a  membrane,  given  solute  

concentra3ons  on  either  side  

3)  Iden3fy  similari3es  and  differences  of  

prokaryo3c  and  eukaryo3c  cells  (either  

classifying  examples  or  iden3fying  in  terms  of  

cell  proper3es)  

4)  Iden3fy  the  macromolecules  that  make  up  

cells  and  the  rela3ve  abundance  of  those  

components  in  a  cell  

5)  Recognize  the  steps  involved  in  an  electron  

transport  chain  

6)  Dis3nguish  between  the  outcomes  of  mitosis  

and  meiosis    

How  were  these  used?    

1.  List  was  shared  with  

instructors,  feedback  solicited    

2.  For  Fall  2011  finals:  

Example  ques3ons  for  each     of  the  6  learning  objec3ves   were  wrieen  and  offered  for   instructors  (most  instructors   wrote  their  own  ques3ons)      

 

**Enabled  a  post-­‐assessment   of  student  performance  in   each  sec3on  of  BIO  101.  (Built   into  the  Final  Exam).  

(6)

85.5   68   72.5   51   81   51   36   48   58   31   77   70.33   90.33   53.5   51.5   66   24   57   48   76   30   70   27   47   53   71   36   75   0   10   20   30   40   50   60   70   80   90   100   1   2   3   4   5   6  

Av

er

ag

e  

Pe

rc

en

t  C

or

re

ct

 

Common  Objec7ve  Number  

Post-­‐Assessment  for  6  Common  Objec7ves  

Higbie   Wetherbee   Van  Sciver   Cary  Test  1   Cary  Test  2   Instructor  1   Instructor  2   Instructor  3   Instructor  4   Test  A   Instructor  4   Test  B  

(7)

0   10   20   30   40   50   60   70   80   90   100  

Student  performance  shown  by  objec3ve  

(Instructor  1)  

Objec3ve  1   Objec3ve  2   Objec3ve  3   Objec3ve  4   Objec3ve  5   Objec3ve  6   %  c or re ct  

(8)

0   10   20   30   40   50   60   70   80   90   100  

Common  Objec7ve  Percents  Wetherbee  

Objec3ve  1   Objec3ve  2   Objec3ve  3   Objec3ve  4   Objec3ve  5   Objec3ve  6  

Student  performance  shown  by  objec3ve    

(Instructor  2)  

(9)

0   10   20   30   40   50   60   70   80   90   100  

Common  Objec7ve  Percents  Van  Sciver  

Objec3ve  1   Objec3ve  2   Objec3ve  3   Objec3ve  4   Objec3ve  5   Objec3ve  6  

Student  performance  shown  by  objec3ve    

(Instructor  3)  

(10)

0   10   20   30   40   50   60   70   80   90   100  

Common  Objec7ve  Percents  Cary  Test  1  

Objec3ve  1   Objec3ve  2   Objec3ve  3   Objec3ve  4   Objec3ve  5   Objec3ve  6  

Student  performance  shown  by  objec3ve    

(Instructor  4,  Test  A)  

(11)

0   10   20   30   40   50   60   70   80   90   100  

Common  Objec7ve  Percents  Cary  Test  2  

Objec3ve  1   Objec3ve  2   Objec3ve  3   Objec3ve  4   Objec3ve  5   Objec3ve  6  

Student  performance  shown  by  objec3ve    

(Instructor  4,  Test  A)  

(12)

Total  number  of  student  responses  

• 

Instructor  1:  73  

• 

Instructor  2:  115  

• 

Instructor  3:  71  

• 

Instructor  4  (Test  1):  54  

• 

Instructor  4  (Test  2):  55  

(13)

What  these  data  suggest:  

• 

Students  generally  weakest  with  

objec3ves  1*,3*,and  5  

 (*Scores  <  50%  except  for  1  of  the  4  

instructors)  

• 

Students  strongest  with  objec3ve  6  

(Scores  close  to,  >  70%)  

• 

High  variability  between  

instructors    

 

1)  Dis3nguish  between  a  living  and   non-­‐living  organism  

2)  Predict  direc3on  of  water  

movement  in  diffusion  across  a   membrane,  given  solute  

concentra3ons  on  either  side   3)  Iden3fy  similari3es  and  differences  

of  prokaryo3c  and  eukaryo3c  cells   (either  classifying  examples  or   iden3fying  in  terms  of  cell   proper3es)  

4)  Iden3fy  the  macromolecules  that   make  up  cells  and  the  rela3ve   abundance  of  those  components   in  a  cell  

5)  Recognize  the  steps  involved  in  an   electron  transport  chain  

6)  Dis3nguish  between  the  outcomes   of  mitosis  and  meiosis    

(14)

Other  observa3ons,  sugges3ons  

1.  Instructors  described  minimal  communica3on  

about  the  course  and  scheduling  a  common  

mee3ng  3me  (for  4  instructors)  was  impossible.  

 -­‐  Is  a  mee3ng  possible  before  school  starts?  

   

2.  All  instructors  independently  suggested  that  the  

amount  of  content  in  BIO  101  was  too  large  

(some  specified  this  in  comparison  to  BIO  102)  

– 

Priori3es  are  needed  for  content,  coordinate  with  102  

instructors.  

(Work  in  progress)  

(15)

Bloom's  Levels  of  Understanding:    

Exams  target  levels  1  and  2  

 

6.    Evalua3on:  think  cri3cally  about  and  defend  a   posi3on  

 Judge,  Jus)fy,  Defend,  Cri)cize,  Evaluate    

 

5.    Synthesis:  transform,  combine  ideas  to  create   something  new  

 Develop,  Create,  Propose,  Design,  Invent  

 

4.    Analysis:  break  down  concepts  into  parts  

Compare,  Contrast,  Dis)nguish  

 

3.    Applica3on:  apply  comprehension  to  unfamiliar   situa3ons  

 Apply,  Use,  Compute,  Solve,  Predict    

 

2.    Comprehension:  demonstrate  understanding  of   ideas,  concepts  

Explain,  Summarize,  Interpret,  Describe,  Diagram  

 

1.    Factual  Knowledge:  remember  and  recall  factual   informa3on  

 Define,  List,  State,  Name,  Cite  

Adapted  from  Allen,  D.  and  Tanner,  K.,  Cell  Biol.  Educ.  1:  63-­‐67  (2002)  

-­‐Ok  for  introductory  course?  (But   are  students  bored?  prepared?   Challenged?)  

Ideas:  

-­‐ Freeman  text  indicates  Blooms   levels  for  each  ques3on.  

 

-­‐ Pre-­‐lecture  quizzes  online  

(Mastering  Biology)  can  address   low  Bloom’s  level  content  well,   enabling  lecture  3me  to  focus  on   applica3ons  and  higher  levels  of   understanding.    

(This  does  take  a  lot  of  planning  ,   thought,  and  3me.)  

(16)

II.  Overview  of  Ac3vi3es  

Discussed  goals  with   Instructors  (and  TA)  for  Bio  

262  

Discussed  goals  with   Course  Coordinator  for  BIO  

101  labs  

“  Students  should  be  able  to  build  a  graph”  

“I  want  students  to  be  able  to  interpret  data  on  a  graph”   Introductory  Level  

Intermediate  Level  

New  Forma3ve   Assessment:  

Students  plot  their  grade   in  class  vs.  3me  

New  Pre-­‐  Assessment:   Students  answer   ques3ons  about  graphs,  

interpret  experimental   results    

Spring  2012  

New  Post  Assessment   in  prepara3on.  

-­‐Used  experiences  from  Fall  semester   to  design  a  pre-­‐assessment  for  Spring  

(17)

Example  of  ploqng  ones  grades  

assignment  

• 

Opportunity  for  self-­‐reflec3on  

• 

Extra  credit  opportunity:    

(forma3ve  assessment=  Instructors  and  TAs  

get  feedback  before  there  is  a  penalty  to  

(18)

Pre-­‐Assessment  262  

(Objec3ve:  Students  can  interpret  data)  

•  Administered  in  recita3on  within  the  second  week  of  class  

Format  

6  ques3ons:  Mul3ple  Choice  and    Short  Response  

 

Five  types  of  data  selected  (based  on   instructor’s  emphasis  in  the  course):   (1)  Popula3on  Growth  

(2)  Keeling  Curve  

(3)  Rarefac3on  Curve  (species   richness)  

(4)  Compe33on/Preda3on  Graphs   (5)  Bar  graphs  from  manipula3ve  

experiment  

Sources  for  ques3ons:  

-­‐Assigned  SimU  text  (online  program)   -­‐Teaching  Issues  and  Experiments  in   Ecology  (TIEE)    

(19)

15   37   18   3   0   0   10   20   30   40   A   B   C   D   E   Anxiety  Levels  

(20)

Pre-­‐assessment  BIO  262  

•  74  students  total  

•  Student  performance  was  stronger   than  expected.    

•  Generally  students  performed  poorly  

on  a  compe33on  graph  (required   Ecology  specific-­‐knowledge).    

•  Post  Assessment  (by  end  of  this   semester)  will  measure  

improvement.     5   8   10   14   20   16   1   0   5   10   15   20   25   40   50   60   70   80   90   100  

Overall  Percent  Correct  

Five  types  of  data  selected  (based   on  instructor’s  emphasis  in  the   course):  

(1)  Popula3on  Growth   (2)  Keeling  Curve  

(3)  Rarefac3on  Curve  (species   richness)  

(4)  Compe33on/Preda3on  Graphs   (5)  Bar  graphs  from  manipula3ve  

experiment     45   70   51   66   36   0   10   20   30   40   50   60   70   80   1   2   3   4   5   Co rr ec t  Re sp on se s   Ques7on  Number  

(21)

Next  Steps…  

1.  Revise  Common  learning  objec3ves  

(priori3es)  based  on  all  faculty  and  

instructor  input.    

 -­‐  An  all-­‐faculty  survey  is  being  used  to  

iden3fy  priori3es  for  higher  level  courses.  

2.  Build  a  common  on-­‐line  pre-­‐assessment  for  

Fall  2012  

3.  Compile  a  set  of  similar  final  exam  ques3ons  

across  sec3ons  for  post  assessment.    

(22)

Next  Steps…  

-­‐  Con3nue  graphing  assessment,  revise  based  on   effec3veness.  

 

-­‐  Examine  results  of  post  assessment,  compare  to  pre-­‐ assessment  

-­‐  Con3nue  pre-­‐and  post-­‐  tests  in  future  semesters  (with   instructor  modifica3on).  

-­‐  Present  format  to  faculty  as  a  possible  model  for  other   upper  level  elec3ve  courses.  

 

Assemble  a  team  of  3  faculty  in  Summer  2014  to  apply  for   the  Undergraduate  Educa3on  in  Biology  Conference  on   Scien3fic  Teaching.   BIO  101  lab       BIO  262             Sustaining   Assessment     Ac3vi3es  

Figure

Updating...

References

Updating...

Related subjects :