Data Analysis Summary 38

This  section  provides  a  meta-­‐analysis  of  the  three  projects  contained  in  the  thesis  by   summarizing  the  research  questions  that  guided  each  project  and  the  high  level   findings  for  each  of  the  questions.    This  information  is  presented  in  Table  1-­‐3:  Thesis   Meta-­‐Analysis.  

Table  1-­‐3:    Thesis  Meta-­‐Analysis    

Project  1  

Q1:    What  challenges  do   managers  face  measuring  and   managing  strategic  

performance  in  turbulent   environments?  

The  rate  of  change  and  the  magnitude  of  change  are   increasing,  making  environments  more  complex  to   navigate.    Changes  in  technology  and  competitors  are   causing  much  of  the  turbulence.    Planning  and   coordination  activities  increase,  but  are  more  informal   and  decentralized.    Managers  need  more  information  to   make  higher  risk  decisions  but  they  are  bounded  in   their  ability  to  process  this  information.  

Q2:    How  do  firms  in   turbulent  environments   measure  and  mange  strategic   performance  currently?      

Literature  says  little  about  strategic  performance   measurement  in  turbulent  settings.    Performance   measurement  and  management  is  less  formal  and  more   decentralized  than  in  stable  settings.    Managers’  focus   on  controlling  their  strategic  priorities.      

Q3:    For  those  firms  within   turbulent  environments  that   employ  strategic  performance   measurement  and  

management  systems,  what   elements  are  contained   therein?  

Firms  in  turbulent  environments  have  strategies  that   may  or  may  not  be  formalized;  they  have  goals  and/or   objectives,  performance  measures,  programs,  and  links   to  compensation.    Schreyogg  and  Steinmann’s  (1987)   model  of  strategic  control  and  Simons’s  Levers  of   Control  (1995)  are  theories  that  may  explain   functioning  of  strategic  performance  management   systems  however  it  is  unclear  the  extent  to  which  they   apply  in  turbulent  settings.  

Q4:    What  factors  affect  the   design  of  strategic  

performance  measurement   systems  in  turbulent   environments?  

Four  factors  affect  the  design  of  strategic  performance   measurement  systems  in  a  turbulent  environment:     environmental  factors,  organizational  factors,   technology  factors,  and  management  perceptions.   Q5:    How  can  firms  improve  

strategic  performance   measurement  in  turbulent   environments?  

Three  actions  managers  can  take  to  improve  their   strategic  performance  measurement  and  management   activities  are:    first,  understand  the  variables  that  are   causing  turbulence  and  assess  their  level  of  volatility;  

Edward  A.  Barrows,  Jr.  –  Cranfield  University  –  School  of  Management  –  DBA  Thesis   How  Firms  in  Turbulent  Environments  Measure  Strategic  Performance  

39  

second,  draw  from  existing  control  theory  such  as   Schreyogg  and  Steinmann  (1987)  or  Simons  (1995)  to   aid  in  designing  strategic  measurement  systems;  third,   establish  a  means  to  evaluation  environmental   variables  in  an  ongoing  manner.  

Project  2  

Q1:    What  strategic   performance  measures  are   used  by  firms  operating  in   turbulent  environments?  

Strategic  performance  measures  were  linked  to  critical   objectives  such  as  revenues,  customer  behavior,  and   expenses.    Measures  were  not  balanced;  they  were   clustered  in  depth  in  critical  objective  areas.    They   presented  in  significant  number  in  areas  of  critical   objectives  areas.    

Q2:    What  features,  roles,  and   processes  comprise  the   strategic  performance   measurement  system  of  firms   operating  in  turbulent   environments?  

Each  system  contained  a  set  of  performance  measures   and  a  reporting  infrastructure.    Additionally,  functional   plans  and  a  set  of  strategic  objectives  were  found  in   every  firm.    Every  firm  utilized  a  customer  interaction   component—a  means  of  maintaining  through   measurement  or  action—a  mechanism  to  monitor   customer  behavior.    Six  other  roles  were  found:     manage  strategy,  measure  performance,  manage   products,  communicate  performance,  influence   behavior,  adapt  the  organization.      

Q3:    What  contextual  factors   affect  the  design  of  a  firm’s   strategic  performance   measurement  systems?  

Strategic  performance  measurement  systems  were   affected  by  three  internal  factors  and  one  external   factor.    Top  management  aims,  board  of  director  aims,   culture  comprised  the  internal  factors;  customer   requirements  was  the  external  factor.    The  most   significant  was  management’s  aims.    

Q4:    How  does  the  strategic   performance  measurement   system  inform  strategic   decisions?  

Firms  that  used  the  strategic  performance  

measurement  system  earlier  and  more  often  in  their   decision-­‐making  enjoyed  higher  satisfaction  levels  than   those  that  did  not.  

Project  3  

Q1:    When  strategic  decisions   are  satisfying,  is  the  strategic   performance  measurement   system  effective?  

Partially.    When  strategic  decisions  were  satisfying,  the   strategic  performance  measurement  system  was   ‘somewhat  effective.’    There  was  no  direct  link  found   between  decision  satisfaction  and  the  effectiveness  of   the  strategic  performance  measurement  system.    Top   managers  sought  decision-­‐specific  information  when  it   was  needed.  

Q2:    Given  the  strategic   performance  measurement   system’s  effectiveness,  are   strategic  performance   measures  clustering  on  critical   environmental  variables?  

When  the  strategic  performance  measurement  system   was  seen  as  ‘somewhat  effective’,  there  was  clustering   of  measures  on  critical  variables.    In  the  case,  72%  and   70%  of  strategic  performance  measures  clustered  on   five  critical  variables  identified  by  the  top  team  as  the   ones  driving  industry  change.      

Edward  A.  Barrows,  Jr.  –  Cranfield  University  –  School  of  Management  –  DBA  Thesis   How  Firms  in  Turbulent  Environments  Measure  Strategic  Performance  

40  

performance  management   system’s  effectiveness,  are   strategic  performance   measures  present  in  

significant  numbers  in  critical   environmental  variable  areas?  

was  seen  as  ‘somewhat  effective’,  there  was  clustering   of  measures  on  critical  variables.    For  the  top  two   variables  driving  change,  49%  of  strategic  performance   measures  presented  on  the  variables.    Further,   operational  dashboards  were  developed  which   comprised  in  depth  operational  measures  in  both  of   these  areas.  

Q4:    Given  the  strategic   performance  measurement   system’s  effectiveness,  is   strategic  performance   measurement  system  use   high?  

When  the  strategic  performance  measurement  system   was  seen  as  ‘somewhat  effective’,  usage  was  found  to   be  low.    During  the  period  examined,  the  strategic   performance  measurement  system  was  only  reviewed   twice  by  the  top  team  out  of  16  formal  meetings.     However,  during  the  16  formal  meetings,  33  times   topics  pertaining  to  key  measures  and  objectives  on  the   strategic  performance  measurement  system  were   discussed.  

Q5:    Given  the  strategic   performance  management   system’s  effectiveness,  is   management  attention  to   critical  environmental   variables  high?  

When  the  strategic  performance  measurement  system   was  seen  as  ‘somewhat  effective’,  management’s   attention  to  critical  variables  was  high.    Interviews  and   questionnaire  data  confirmed  that  management  was   focused  on  critical  variables  and  a  review  of  CEO   communications  found  that  on  17  of  26  occasions  direct   communication  to  the  entire  workforce  via  the  

organization’s  newsletter  included  to  topics  related  to   the  strategic  performance  measurement  system.  

Project  1,  the  systematic  review  of  literature,  finds  little  literature  that  informs  the   research  question  directly.    What  is  known  is  the  environment  in  which  most  firms   operate  is  becoming  increasingly  difficult  to  navigate  due  to  technology  and   competitive  changes.    To  combat  these  changes,  managers  are  planning  and  

coordinating  more  frequently  albeit  in  more  informal  and  decentralized  ways.    They   seek  more  information  to  inform  their  decisions  but  remain  limited  in  their  ability  to   process  this  information  effectively.    Turbulent  environments  draw  from  existing   control  theory—Simons’s  (1995)  Levers  of  Control  framework  in  particular,  but  it  is   not  clear  how  effective  it  is  in  this  setting.    Factors  that  affect  strategic  performance   measurement  frameworks  include  environmental,  organizational,  and  technology   factors  as  well  as  management  perceptions.    To  cope  with  increased  turbulence   managers  focus  their  attention  on  critical  variables  versus  all  potential  variables   affecting  their  firms.  

In  Project  2,  the  analysis  of  seven  security  software  firms  found  that  measures  are   linked  to  critical  performance  objectives  such  as  revenue  growth,  customer  behavior  

Edward  A.  Barrows,  Jr.  –  Cranfield  University  –  School  of  Management  –  DBA  Thesis   How  Firms  in  Turbulent  Environments  Measure  Strategic  Performance  

41  

and  expense  management.    Firms’  objectives  and  measures  were  not  balanced  in   nature;  they  were  clustered  in  depth  on  critical  variables.    Further,  there  were  more   measures  in  these  areas  than  anywhere  else.    Each  firm  had  strategic  performance   measures  coupled  with  a  reporting  infrastructure,  a  set  of  strategic  objectives  and  a   customer  interaction  component—measures  relating  to  customers  or  a  mechanism   to  stay  in  contact  with  customers  that  provided  insights  regarding  customer  wants   and  behaviors.    They  also  had  a  set  of  functional  plans—such  as  a  sales  forecast  or   product  roadmap—however  the  content  varied  depending  upon  the  individual  firm’s   focus  areas.    The  strategic  performance  measurement  system  filled  six  roles:    

manage  strategy,  measure  performance,  manage  products,  communicate   performance,  influence  behavior,  and  adapt  the  organization.    Further,  strategic   performance  measurement  systems  were  affected  by  three  internal  and  one   external  factor:    top  management  aims,  board  of  director  aims,  and  culture  

comprised  the  internal  factors;  customer  requirements  was  the  external  factor.    The   most  significant  was  top  management’s  aims.    Through  the  analysis  of  a  set  of  each   firms’  strategic  decisions,  it  was  found  that  firms  using  the  strategic  performance   measurement  system  earlier  and  more  frequently  experienced  higher  decision   satisfaction  levels  than  firms  that  did  not  rely  as  much  on  the  strategic  performance   measurement  system.  

In  Project  3,  the  in  depth  study  of  a  five  unit  health  care  system  in  the  midst  of  a   transformation,  it  was  found  that  the  model  from  Project  2  held  in  the  new  setting   but  needed  modification.    When  a  set  of  satisfying  strategic  decisions  was  reviewed,   it  was  found  that  the  strategic  performance  measurement  system  was  only  deemed   ‘somewhat  effective’  by  top  managers.      Further,  there  was  no  direct  link  between   the  strategic  performance  measurement  system  and  decision-­‐making  or  decision-­‐ satisfaction—the  link  was  indirect.    Strategic  performance  measures—in  this  case   manifest  within  the  Balanced  Scorecard—were  clustered  on  critical  environmental   variables.    Further,  a  greater  number  of  strategic  performance  measures  were  found   in  critical  environmental  areas  than  in  other  areas  of  performance.      Usage  of  the   system  was  not  in  keeping  with  regular  patterns  of  use  such  as  month  or  quarterly   reviews;  formal  reviews  were  infrequent  however  the  top  management  team   regularly  engaged  in  discussion  regarding  actions  or  activities  linked  directly  to   strategic  performance  measures  and  strategic  objectives.    Further,  the  CEO  was   focused  on  critical  performance  variables  and  his  communications  to  the   organization  included  a  high  frequency  of  topics  linked  to  critical  variables.          

Edward  A.  Barrows,  Jr.  –  Cranfield  University  –  School  of  Management  –  DBA  Thesis   How  Firms  in  Turbulent  Environments  Measure  Strategic  Performance  

42  

Regarding  the  overall  research  question  regarding  how  firms  in  turbulent  

environments  measure  strategic  performance,  the  following  is  found.    First,  strategic   performance  measurement  systems  may  or  may  not  be  balanced  in  nature;  more   commonly  they  reflect  measurement  orientation  on  a  narrow  set  of  critical  variables   which  themselves  are  aligned  to  uncertainty  areas  within  the  environment.    A   greater  number  of  performance  measures  are  found  clustering  in  critical  variable   areas  as  well.    Strategic  performance  measurement  systems  fill  traditional  roles  such   as  managing  strategy,  measuring  performance,  managing  products,  communicating   performance,  influencing  behavior,  and  adapting  the  organization  however,  systems   are  focused  on  key  drivers  of  change  like  customer  behavior  so  it  plays  a  sensing  and   probing  role  as  well.    The  overall  design  of  the  strategic  performance  measurement   system  is  driven  largely  by  management’s  aims  and  usage  of  the  system  is  

determined  by  management’s  intent.    What  the  systems  do  is  help  top  leaders  orient   attention  on  critical  variables—performance  based  and  uncertainty  oriented—that   must  be  managed  in  order  to  successfully  meet  overall  firm  goals.