Improving colleges : why courses and programmes improve or decline over time

(1)

report

research

Improving colleges

Why courses and programmes

improve or decline over time

(2)

Improving colleges

Why courses and programmes

improve or decline over time

(3)

Acknowledgements

This report builds on the qualitative findings discussed in Pride or prejudice: college teachers’ views on course performance. The focus here is on the detailed statistical analysis of the survey results.

Our thanks to the hundreds of college teachers who took part in the survey and interviews and to the college coordinators – without whom there would be no project:

Paul Harris, Barnet College Gill Collinge, Barnfield College

John Clarke, Bolton College Jane Davidson, Bournemouth and Poole College

Tim Chapman, City College, Norwich Sally Freeborn, Cornwall College

Sue Temple, Exeter College Jo-Ann Delany, Kensington and Chelsea College

Jan Leivers, Loughborough College Steve Harrison, Richard Huish College

John Airey, Sandwell College John McMinn, Sir George Monoux College

Sandra Hamilton-Fox, Solihull Sixth Form College Rob Chadwick, South Birmingham College

Keith Appleby, Truro College Trixi Blaire, West Herts College.

Thanks also to three distinguished Professors of Education, who took part in a ‘virtual’ steering group:

John Gray, Bridget Somekh and Pam Sammons, and the three college coordinators who also took part in the steering group: Jan Leivers, Rob Chadwick and John Clarke. Final thanks to Graham Knight, LSDA, Survey Researcher, who analysed the questionnaire, Geri Hendrick who re-typed successive drafts and to Vikki Smith who edited the document.

Published by the Learning and Skills Development Agency

www.LSDA.org.uk

Feedback should be sent to Information Services, Learning and Skills Development Agency,

Regent Arcade House, 19–25 Argyll Street; London W1F 7LS Tel 020 7297 9000 Fax 020 7297 9001

[email protected]

Registered with the Charity Commissioners

Printed in the UK

ISBN 1 85338 795 9

1299/10/02/2000

You are welcome to copy this publication for internal use within your organisation. Otherwise, no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, electrical, chemical, optical, photocopying, recording or otherwise, without prior written permission of the copyright owner.

This publication was supported by the Learning and Skills Council as part of a grant to the LSDA for a programme of research and development.

(4)

References

and

bibliography

77

Appendices

79

Appendix 1: Colleges that participated in the survey 79

Appendix 2: Research methods 80

Appendix 3: Letter to survey respondents and guidance 84

Appendix 4: Survey questionnaire 88

Appendix 5: Guidance notes for college coordinators 97

Appendix 6: Data sheet for each course/programme 101

(5)

7DEOHVDQGILJXUHV

7DEOH 2YHUDOODVVHVVPHQWRIWKHLPSDFWRIFKDQJHIDFWRUVE\FDWHJRU\RIFRXUVH

)LJXUH +RZFROOHJHVLPSURYH

7DEOH 0HDQUDWLQJIRUFXUULFXOXPIDFWRUVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUFRQWH[WRIFRXUVHSURJUDPPHIDFWRUVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUDVVHVVPHQWDQGIHHGEDFNIDFWRUVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUVWDIILQJDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUTXDOLW\V\VWHPVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUUHVRXUFHVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUVWXGHQWFRKRUWDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUVXSSRUWIRUVWXGHQWVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUVXSSRUWIRUWHDFKHUVDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUWHDFKLQJDFFRUGLQJWRFRXUVH

7DEOH 0HDQUDWLQJIRUSHUVRQDOWXWRULQJRQWKLVFRXUVHSURJUDPPH DFFRUGLQJ WRFRXUVH

7DEOH 0HDQ UDWLQJ IRU FRXUVHSURJUDPPH RUJDQLVDWLRQ DQG DGPLQLVWUDWLRQ DFFRUGLQJWRFRXUVH

(6)

6XPPDU\RINH\PHVVDJHV

N 7KHUH DUH VWURQJ HOHPHQWV RI VLPLODULW\ DQG FRQWLQXLW\ LQ WKH YLHZV RI FROOHJH WHDFKHUV GUDZQ IURP D ZLGH YDULHW\ RI UHODWLYHO\ VXFFHVVIXO DQG XQVXFFHVVIXO FRXUVHVDQGSURJUDPPHV

N 7HDFKHUV HPSKDVLVH WKH LPSRUWDQFH RI JRRG WHDFKLQJ FRPPLWWHG DQG NQRZOHGJHDEOH WHDFKHUV DQG HIIHFWLYH DQG DSSURSULDWH SURFHVVHV IRU UHFUXLWPHQW LQGXFWLRQ DVVHVVPHQW WXWRULQJ TXDOLW\ DVVXUDQFH DQG VWXGHQW VXSSRUW

N 7HDFKHUV DOVR DJUHH WKH IDFWRUV ZLWK D QHJDWLYH LPSDFW RQ VWXGHQW RXWFRPHV 7KHVH DUH IDFWRUV ZKLFK DUH W\SLFDOO\ RXWVLGH WHDFKHUV¶ LQIOXHQFH RU FRQWURO UHVRXUFHVFKDQJHVWRWKHVWXGHQWFRKRUWWLPHWDEOLQJDQGFRXUVHDGPLQLVWUDWLRQ

N 7KHUH DUH DOVR VRPH HYLGHQW GLVFRQWLQXLWLHV 7HDFKHUV IURP FRXUVHV ZLWK WKH KLJKHVW DQG LPSURYLQJ VWXGHQW VXFFHVV UDWHV ZHUH PRVW RSWLPLVWLF DERXW WKH LPSDFWRIFKDQJHV

N %\FRQWUDVWWHDFKHUVIURPFRXUVHVZKHUHRQFHKLJKVWXGHQWVXFFHVVUDWHVZHUH LQGHFOLQHZHUHWKHPRVWSHVVLPLVWLFDERXWFKDQJH

N ,Q WKH VXUYH\ DQG FRQWUDU\ WR H[SHFWDWLRQV WKH YLHZV RI WHDFKHUV IURPORZ EDVHOLQHGHFOLQLQJFRXUVHVWHQGHGWRUHVHPEOHWKHYLHZVH[SUHVVHGE\WHDFKHUV IURPKLJKEDVHOLQHLPSURYLQJFRXUVHV

N &RQWUDU\ WR H[SHFWDWLRQV WHDFKHUV IURPORZ EDVHOLQH LPSURYLQJ FRXUVHV ZHUH QRWSDUWLFXODUO\RSWLPLVWLFDERXWWKHLPSDFWRIFKDQJHV

N 7KHIROORZXSLQWHUYLHZVKHOSHGWRH[SODLQWKHVHDSSDUHQWO\SDUDGR[LFDOILQGLQJV

N 7HDFKHUVRQFRXUVHVZLWKKLJKVWXGHQWUHWHQWLRQDQGDFKLHYHPHQWUDWHVVHHPWR EDVH WKHLU WHDFKLQJ RQ LQWHUDFWLRQ DQG SDUWQHUVKLS ZLWK WKHLU VWXGHQWV ,Q WKHLU YLHZHIIHFWLYHWHDFKHUVDGDSWDQGGHYHORS WKHLUWHDFKLQJWRPHHWWKHQHHGVRI WKHLUOHDUQHUV

N 7HDFKHUVIURPFRXUVHVZLWKORZVWXGHQWVXFFHVVUDWHVZKLFKDUHQRWLPSURYLQJ RUZKLFKDUHGHFOLQLQJVHHPWREODPHHLWKHUWKHLUPDQDJHUVWKHLUVWXGHQWVRU ERWK 7KH\ VXJJHVWHG LW LV QRW WKHLU WHDFKLQJ WKDW QHHGV WR FKDQJH EXW UDWKHU WKHLUVWXGHQWV

N 7KHLQWHUYLHZVDOVRSRLQWWRWKHLPSRUWDQFHRIXQZULWWHQRULPSOLHGFRQWUDFWVLQ FROOHJHV 7HDFKHUV IURP WKH PRVW VXFFHVVIXO FRXUVHV HPSKDVLVH LQ SDUWLFXODU WKHFRQWUDFWVEHWZHHQPHPEHUVRIWHDFKLQJWHDPVDQGEHWZHHQWHDFKHUVDQG WKHLUVWXGHQWV

N 7HDFKHUV IURP WKH OHDVW VXFFHVVIXO FRXUVHV WHQG WR VWUHVV WKH LPSRUWDQFH RI LPSOLFLWFRQWUDFWVZLWKFROOHJHPDQDJHUVZKLFKKDYHEURNHQGRZQ

N 7HDFKLQJ WHDPV DSSHDU WR KDYH DQ DPELYDOHQW UROH (IIHFWLYH FRQILGHQW DQG DXWRQRPRXV WHDPV DSSHDU WR VXVWDLQ DQG HQWKXVH WHDFKHUV IURP WKH PRVW VXFFHVVIXO FRXUVHV ,Q XQVXFFHVVIXO FRXUVHV WHDPV VHHP WR VXSSRUW DQG FRQILUPWKHYLHZWKDWWKHUHLVOLWWOHRUQRVFRSHIRULPSURYHPHQW

N (IIRUWVWRLPSURYHFROOHJHVWKDWGRQRWHQJDJHWHDFKHUV¶YDOXHV\VWHPVDQGWKH ZD\VWKDWWKH\FRQFHLYHRIWKHWHDFKLQJUROHDUHQRWOLNHO\WREHVXFFHVVIXO

(7)

(8)

,QWURGXFWLRQWRWKHUHVHDUFK

7KH LVVXH RI KRZ WR LPSURYH WKH TXDOLW\ RI FROOHJHV FXUUHQWO\ KDV KLJK SROLWLFDO SURIHVVLRQDO DQG PDQDJHULDO SULRULW\ $OWKRXJK FROOHJHV UHPDLQ XQGHUUHVHDUFKHG FRPSDUHGZLWKVFKRROVWKHUHLVDODUJHDQGJURZLQJOLWHUDWXUHLQWKHSXEOLFGRPDLQDQG LQ WKH JUH\ DUHD RI XQSXEOLVKHG UHVHDUFK GLVVHUWDWLRQV DQG UHVHDUFK SURGXFHG IRU LQWHUQDO FRQVXPSWLRQ ZLWK FROOHJHV 0DUWLQH] 7KH UHVHDUFK UHSRUWHG KHUH ZDV GHVLJQHGWRVKHGVRPHQHZOLJKWRQFROOHJHLPSURYHPHQWLVVXHVE\IRFXVLQJRQZKDW FRXOG EH VHHQ DV WKH HQJLQH URRP RI LPSURYHPHQW LQGLYLGXDO FRXUVHV DQG SURJUDPPHV

7KHUHVHDUFKTXHVWLRQFDQEHSXWTXLWHVLPSO\

N :KDWPDNHVWKHPRVWGLIIHUHQFHEHWZHHQFRXUVHVRUSURJUDPPHVZKHUHOHDUQHU SHUIRUPDQFH LPSURYHV DQG FRXUVHV DQG SURJUDPPHV ZKHUH OHDUQHU SHUIRUPDQFHGHFOLQHV"

6XEVLGLDU\TXHVWLRQVLQFOXGHGWKHIROORZLQJ

N $UH WKHUH GLIIHUHQFHV EHWZHHQ FRXUVHV ZKHUH OHDUQHU SHUIRUPDQFH LPSURYHV IURPDKLJKEDVHOLQHSRVLWLRQFRPSDUHGWR FRXUVHVZKLFKLPSURYHIURPDORZ EDVHOLQHSRVLWLRQ"

N $UHWKHUHGLIIHUHQFHVEHWZHHQFRXUVHVZKHUHOHDUQHUSHUIRUPDQFHGHFOLQHVIURP D KLJK EDVHOLQH SRVLWLRQ FRPSDUHG ZLWK FRXUVHV ZKHUH OHDUQHU SHUIRUPDQFH GHFOLQHVIURPDORZEDVHOLQHSRVLWLRQ"

N :KHUH OHDUQHU SHUIRUPDQFH LPSURYHV GR WHDFKHUV WHQG WR DWWULEXWH WKH LPSURYHPHQWWRWKHLURZQDJHQF\DQGGLVFRXQWWKHLPSRUWDQFHRIIDFWRUVWKDWOLH RXWVLGHWKHLUFRQWURO"

N :KHUH OHDUQHU SHUIRUPDQFH LV GHFOLQLQJ GR WHDFKHUV WHQG WR DWWULEXWH WKH GHWHULRUDWLRQWRIDFWRUVRXWVLGHWKHLUFRQWURODQGGLVFRXQWWKHLPSRUWDQFHRIWKHLU RZQDJHQF\"

5HVHDUFKGHVLJQ

7KH UHVHDUFK ZDV XQGHUWDNHQ LQ WZR VWDJHV TXDQWLWDWLYH DQG TXDOLWDWLYH ,Q WKH TXDQWLWDWLYHVWDJHLQYLWDWLRQVZHUHVHQWWRDURXQGFROOHJHVZKLFKKDGSDUWLFLSDWHG LQDWOHDVWWZRRIWKHILUVWWKUHHURXQGVRIDFWLRQUHVHDUFKSURMHFWVWRLPSURYHUHWHQWLRQ DQG DFKLHYHPHQW 7KHVH SURMHFWV ZHUH VXSSRUWHG E\ WKH 5DLVLQJ 4XDOLW\ DQG $FKLHYHPHQW3URJUDPPHIRXQGHGE\'I(6DQGOHGE\/6'$IRUGLVFXVVLRQVRIWKLV SURJUDPPHVHH0DUWLQH]DDQG&RXVLQ

(9)

ORFDOFRQWH[WVIURPLQQHUFLW\WRVPDOOWRZQDQGUXUDOORFDWLRQV7KHFROOHJHVWKDWWRRN SDUWLQWKHUHVHDUFKDUHOLVWHGLQ$SSHQGL[

&ROOHJHFRRUGLQDWRUVDWWHQGHGWZRVHPLQDUVWRGHWHUPLQHWKH

N RYHUDOOUHVHDUFKGHVLJQ

N GHVLJQRIWKHTXHVWLRQQDLUH

N FULWHULDIRUWKHVHOHFWLRQRIWKHVDPSOH

N SURFHVVRIDGPLQLVWHULQJWKHTXHVWLRQQDLUH

N $ PRUH GHWDLOHG QRWH RI WKH UHVHDUFK GHVLJQ DQG SURFHVV LV DWWDFKHG DV $SSHQGL[7KHVXUYH\LQVWUXPHQWVDUHDOVRLQFOXGHGLQDSSHQGLFHVDVIROORZV

N TXHVWLRQQDLUH$SSHQGL[

N FRYHULQJOHWWHUWRDFFRPSDQ\WKHTXHVWLRQQDLUH$SSHQGL[

N JXLGDQFHQRWHVIRUFROOHJHFRRUGLQDWRUV$SSHQGL[

N H[DPSOHRIGDWDVKHHWIRUHDFKFRXUVHSURJUDPPH$SSHQGL[

&ROOHJH FRRUGLQDWRUV VHOHFWHG WHDFKHUV ZKR ZRXOG EH LQYLWHG WR FRPSOHWH WKH TXHVWLRQQDLUH7HDFKHUVZHUHVHOHFWHGIURPDFURVVVHFWLRQRIFRXUVHVZLWKKLJKRUORZ EDVHOLQHSRVLWLRQV,QWKLVFRQWH[WEDVHOLQHSRVLWLRQPHDQVKLJKRUORZVXFFHVVUDWHVLQ WKHDFDGHPLF\HDUZHUHUHODWLYHWRWKHQRUPIRUHDFKFROOHJH6XFFHVVUDWHV ZHUH FKRVHQ DV WKH EHVW VLQJOH LQGLFDWRU RI WKH UHODWLYH VXFFHVV RI D FRXUVH 7KH VXFFHVVUDWHLVWKHSURSRUWLRQRIVWXGHQWVZKRVWDUWIURPWKHILUVWFHQVXVGDWHRIWKH ILUVW\HDURIDFRXUVHDQGJRRQWRDFKLHYHWKHLUTXDOLILFDWLRQ,QRWKHUZRUGVVXFFHVV UDWHV FDQ EH FDOFXODWHG E\ PXOWLSO\LQJUHWHQWLRQ UDWHV E\DFKLHYHPHQW UDWHV 7ZR IXUWKHU VHOHFWLRQ FULWHULD ZHUH WKDW WHDFKHUV KDG WDXJKW RQ WKH FRXUVH IRU DW OHDVW \HDUV DQG WKDW WKH FRXUVH RU SURJUDPPH VKRXOG KDYH D PLQLPXP JURXS VL]H RI VWDUWHUVWRPLQLPLVHWKHHIIHFWRIUDQGRPYDULDWLRQV

(10)

DUUDQJHPHQWV DQG QRW WR KDYH IHOW FRQVWUDLQHG E\ WKH LQWHUYLHZ SURFHVV 7KHVH LQWHUYLHZVFRPSULVHGWKHTXDOLWDWLYHSKDVHRIWKHUHVHDUFK

6WUXFWXUHRIWKHUHSRUW

(11)

(12)

3$5748$17,7$7,9(5(6($5&+

,QWURGXFWLRQ

7KHUHDUHDOUHDG\PDQ\FDVHVWXGLHVRIFROOHJHLPSURYHPHQW2YHULPSURYHPHQW SURMHFWV JHQHUDWHG E\ WKH 5DLVLQJ 4XDOLW\ DQG $FKLHYHPHQW 54$ 3URJUDPPH DUH DYDLODEOH DW ZZZUTDRUJXN 7KHUH DUH DOVR V\QWKHVHV RI FROOHJH LPSURYHPHQW VWUDWHJLHVEDVHGRQFDVHVWXGLHVJHQHUDWHGE\WKH54$3URJUDPPH0DUWLQH]E DQG&RXVLQ7KHUHDUHDOVRVRPHV\QWKHVHVRILPSURYHPHQWDFWLYLWLHVWKDW KDYHRFFXUUHGRXWVLGHWKH54$3URJUDPPH.HQZULJKW0DUWLQH]DQG 1DWLRQDO$XGLW2IILFH

7KH FDVH VWXGLHV KDYH WKH ZHDNQHVVHV DQG VWUHQJWKV DVVRFLDWHG ZLWK DQ DFWLRQ UHVHDUFKDSSURDFK7KH\DUHHPSLULFDOO\EDVHGFROODERUDWLYHUHDOZRUOGSURMHFWV7KH\ KDYHDVWURQJIRFXVRQLPSURYHPHQWDQGWKHWUDQVIHURIVXFFHVVIXOSUDFWLFHWRJHWKHU ZLWK D OHYHO RI GHWDLO VXIILFLHQW WR IDFLOLWDWH UHSOLFDWLRQ DQG WUDQVIHU DQG WR DOORZ SUDFWLWLRQHUVWRPDNHWKHLURZQMXGJHPHQWVDERXWUHOHYDQFH2QWKHRWKHUKDQGWKH\ DUH DOVR YDULDEOH LQ WKHLU PHWKRG WKH ULJRXU RI WKHLU DQDO\VLV DQG VRPHWLPHV WKH UREXVWQHVVRIWKHLUGDWD,WLV GLIILFXOWPRUHRYHUWRGHULYHJHQHUDOLVDWLRQVFRQFHUQLQJ µJRRG¶ VWLOO OHVV µEHVW¶ SUDFWLFH WUDQVIHUDELOLW\ WR GLIIHUHQW FRQWH[W DQG VRPHWLPHV FDXVH DQG HIIHFW UHODWLRQVKLSV 7KH FDVH VWXGLHV GR QRW GLVWLQJXLVK HDVLO\ WKRVH VWUDWHJLHVWKDWKDYHKDGDSDUWLFXODUO\ODUJHLPSDFWRQUHWHQWLRQDFKLHYHPHQWDQGWKRVH ZLWK D UHODWLYHO\ VPDOO LPSDFW )LQDOO\ EHFDXVH RI DQ XQGHUVWDQGDEOH UHOXFWDQFH WR UHSRUW LPSURYHPHQW HIIRUWV WKDW KDYH IDLOHG WKH LPSURYHPHQW FDVH VWXGLHV ODFN D FRQWUROJURXSRIFROOHJHVZKHUHLPSURYHPHQWHIIRUWVKDYHQRWEHHQVXFFHVVIXO

7KH DLP WKHUHIRUH RI WKLV SDUW RI WKH UHSRUW LV WR H[WHQG WKH ILHOG RI FROOHJH LPSURYHPHQW UHVHDUFK E\ DSSO\LQJ PRUH TXDQWLWDWLYH UHVHDUFK WHFKQLTXHV WR D VWUXFWXUHGVDPSOHRIFRXUVHVZLWKLQFROOHJHV

7KHUHVHDUFKGHVLJQ

7KH UHVHDUFK GHVLJQ ZDV LQVSLUHG E\ VRPH UHVHDUFK XQGHUWDNHQ E\ -RKQ UD\ DQG RWKHUVZKLFKLVSXEOLVKHGDV,PSURYLQJVFKRROVUD\HWDO7KHPHWKRGRORJLFDO LQQRYDWLRQ GHYHORSHG E\ *UD\ FRPELQHG HOHPHQWV RI PHWKRGRORJ\ IURP ERWK VFKRRO HIIHFWLYHQHVVDQGVFKRROLPSURYHPHQWWUDGLWLRQV$VDPSOHRIVFKRROVZDVVHOHFWHGWR LQFOXGHVFKRROVZLWKGLIIHUHQWEDVHOLQHSRVLWLRQVDQGGLIIHUHQWUDWHVRILPSURYHPHQW$ FURVVVHFWLRQRIVWDIILQHDFKRIWKHVDPSOHVFKRROVZDVVXUYH\HGWRJDWKHULQIRUPDWLRQ DERXWGLIIHUHQWFDXVDWLYHIDFWRUVDQGSURFHVVYDULDEOHV

(13)

«µUDSLG¶LPSURYHPHQWZDVOLQNHGWRFKDQJHVLQWKHVFKRROV¶XVH RI YDULRXV WDFWLFV IRU PD[LPLVLQJ H[DP JUDGHV WKH GHYHORSPHQW RI SROLFLHV IRU VXSSRUWLQJ WHDFKLQJ DQG OHDUQLQJ WKH GHJUHH WR ZKLFK WKH\ KDYH EHJXQ WR WDFNOH WKH SURFHVV RI WHDFKLQJ DQG OHDUQLQJ DW FODVVURRP OHYHO DQG WKH H[WHQW WR ZKLFK UHVSRQVLELOLWLHVKDYHEHHQJLYHQWRSXSLOV

VFKRROVZKLFKKDGLPSURYHGPRUHµUDSLGO\¶KDGKLJKHUSURSRUWLRQV RIWHDFKHUVUHSRUWLQJWKDWWKHUHKDGEHHQµVXEVWDQWLDO¶FKDQJHVLQ WKH TXDOLW\ RI WHDFKLQJ DQG OHDUQLQJ RYHU WKH ODVW \HDUV ,Q µVORZO\¶LPSURYLQJVFKRROVRQO\DYHU\VPDOOSURSRUWLRQRIWKHVWDII W\SLFDOO\DURXQGLQRUIHZHUUHSRUWHGVLPLODUFKDQJHV

WKHUH DUH VRPH VXJJHVWLRQV WKDW ZKLOH VFKRROV VHHP WR KDYH ODXQFKHG FKDQJH LQLWLDWLYHV RQ PDQ\ IURQWV WKHLU DSSURDFKHV WR LPSURYHPHQWFRXOGEHEXQGOHGWRJHWKHU6FKRROVZKLFKGLGVRPH WKLQJVDOVRGLGRWKHUV7KHPRVWUDSLGO\LPSURYLQJVFKRROVVHHP KRZHYHU WR KDYH IRXQG ZD\V RI VWUDGGOLQJ GLVWLQFWLYHO\ GLIIHUHQW DSSURDFKHVDWWKHVDPHWLPH

*UD\HWDOS

5HVHDUFKK\SRWKHVHV

(DUOLHUZRUNRQWHDFKHUV¶DWWLWXGHVLQIXUWKHUHGXFDWLRQ0DUWLQH]IRXQGWKDW

N IDFWRUVWRGRZLWKUHVRXUFHVWKHQDWXUHRIWKHVWXGHQWFRKRUWDQGFROOHJH PDQDJHPHQW DUH JHQHUDOO\ SHUFHLYHG WR KDYH D QHJDWLYH LPSDFW RQ VWXGHQWDFKLHYHPHQW

N IDFWRUVWRGRZLWKWHDFKHUVDQGWHDFKLQJDUHJHQHUDOO\SHUFHLYHGWRKDYH DSRVLWLYHLPSDFWRQVWXGHQWDFKLHYHPHQW

N WHDFKHUV ZKR PDGH PRUH SHVVLPLVWLF REVHUYDWLRQV DERXW WKHLU ZRUN WHQGHG WR HPSKDVLVH WKH LQIOXHQFH RI QHJDWLYH IDFWRUV RQ VWXGHQW DFKLHYHPHQWWHDFKHUVZKRWHQGHGWRPDNHPRUHRSWLPLVWLFMXGJHPHQWV WHQGHGWRHPSKDVLVHWKHLPSRUWDQFHRIWHDFKHUVDQGWHDFKLQJRQVWXGHQW DFKLHYHPHQW

N WKHLVVXHLVE\QRPHDQVFOHDUFXWWHDFKHUDWWLWXGHVDUHKLJKO\LQGLYLGXDO DQG FROOHDJXHV LQ WKH VDPH WHDFKLQJ WHDP PD\ KDYH GLDPHWULFDOO\ RSSRVHGYLHZV

(14)

,QWKHOLJKWRIWKHVHILQGLQJVDQG*UD\¶VZRUNWKHIROORZLQJK\SRWKHVHVZHUHSURSRVHG

N WKHUHZRXOGEHGLIIHUHQWSDWWHUQVRIUHVSRQVHEHWZHHQWHDFKHUVZRUNLQJ RQGLIIHUHQWFDWHJRULHVRIFRXUVH

N WKHPRVWPDUNHGGLIIHUHQFHVZRXOGRFFXUEHWZHHQFRXUVHVZKHUHUHVXOWV ZHUHLPSURYLQJDQGWKRVHZKHUHUHVXOWVZHUHGHFOLQLQJ

N WHDFKHUV RQ LPSURYLQJ FRXUVHV ZRXOG WHQG WR DWWULEXWH LPSURYHPHQWV WR WKHLURZQDJHQF\DQGWRIDFWRUVRYHUZKLFKWKH\KDYHPRVWFRQWURO

N WHDFKHUVRQFRXUVHVZKHUHUHVXOWVZHUHGHFOLQLQJZRXOGWHQGWRDWWULEXWH WKHGHFOLQHWRIDFWRUVRYHUZKLFKWKH\KDGOLWWOHRUQRFRQWURO

N WKHEDVHOLQHSRVLWLRQRIFRXUVHVZRXOGDIIHFWWKHSDWWHUQVRIUHVSRQVHEXW WKHUHZHUHQRVSHFLILFH[SHFWDWLRQVFRQFHUQLQJVXFKSDWWHUQV

$QDO\VLVRIWKHUHVSRQVH

:H UHFHLYHG FRPSOHWHG TXHVWLRQQDLUHV IRU GLIIHUHQW FRXUVHVSURJUDPPHV ,Q WHUPV RI OHYHO RI SURJUDPPH WKHVH VSOLW LQWR WKH IROORZLQJ SURSRUWLRQV URXQGHG WR ZKROHILJXUHV

N /HYHO

)RXU W\SHV RI TXDOLILFDWLRQ DFFRXQWHG IRU WKUHHTXDUWHUV RI WKH FRPSOHWHG TXHVWLRQQDLUHV

N *194

N 194

N $6$OHYHO

N 1DWLRQDO'LSORPD

7KH TXHVWLRQQDLUHV ZHUH PDLQO\ FRPSOHWHG E\ WHDFKHUV ZKR ZHUH RUJDQLVHG LQ UHODWLYHO\VPDOOWHDPVDQGWHDFKLQJRQIXOOWLPHFRXUVHVXQGHUWDNHQE\±\HDUROG VWXGHQWV

N RIWKHFRXUVHVZHUHIXOOWLPHSDUWWLPH

N WKHUDQJHRIKRXUVRIWHDFKLQJSHUZHHNSHUFRXUVHZDVIURPWR KRXUVZDVWKHPRGHDQGKRXUVZDVWKHPHGLDQ

N URXJKO\KDOIWKHFRXUVHVODVWHGIRU\HDUDQGKDOIIRU\HDUV

N RI FRXUVHV ZHUH SULPDULO\ IRU ± \HDU ROGV DQG IRU VWXGHQWVDJHGSOXV

(15)

0DQ\ RI WKH UHVSRQGHQWV KDG WZR RU PRUH UROHV 7KH\ GHVFULEHG WKHLU UROHV LQ WKH IROORZLQJZD\VWHDFKHUWXWRUFRXUVHOHDGHURWKHU7KHUROHV DGGXSWRPRUHWKDQVLQFHUHVSRQGHQWVFRXOGVHOHFWDQ\DQGDOORIWKHUROHV

$ IXUWKHU DQDO\VLV ZDV XQGHUWDNHQ WR LQYHVWLJDWH ZKHWKHU UHVSRQGHQWV ZKR ZHUH FRXUVH WHDP OHDGHUV H[SUHVVHG GLIIHUHQW YLHZV IURP WHDFKHUV WXWRUV DQG RWKHU UHVSRQGHQWV,QJHQHUDOWHUPVWHDFKHUVDQGWXWRUVWHQGHGWRJLYHVOLJKWO\PRUHSRVLWLYH

DVVHVVPHQWVRIWKHLPSDFWRIFKDQJHWKDQFRXUVHWHDPOHDGHUVDOWKRXJKWKLVZDVQRW XVXDOO\VWDWLVWLFDOO\VLJQLILFDQWHYHQDWWKHOHYHO

7KH H[FHSWLRQ WR WKLV REVHUYDWLRQ RFFXUUHG LQ UHVSHFW RI FKDQJHV WR WKH VWXGHQW FRKRUWV+HUHFRXUVHWHDPOHDGHUVZHUHVLJQLILFDQWO\OHVVQHJDWLYHLQWKHLUDVVHVVPHQW RIWKHLPSDFWRIFKDQJHVWRWKHVWXGHQWFRKRUWLQWHUPVRIVWXGHQWV¶PRWLYDWLRQDELOLW\ OHDUQLQJVNLOOVFODULW\RIFDUHHUJRDOVWKHLUSHUVRQDOSUREOHPVDQGKRZQHDUWKH\OLYHG WR FROOHJH :LWK WKLV QRWDEOH H[FHSWLRQ LW VHHPHG VDIH WR FRQFOXGH WKDW FRXUVH WHDP OHDGHUVKDYHEURDGO\VLPLODUYLHZVWRWKRVHRIWKHWHDFKHUVWXWRUVDQGRWKHUPHPEHUV RIVWDIILQFOXGHGLQWKHTXDQWLWDWLYHUHVHDUFK

2YHUDOOILQGLQJV

<HDURQ\HDUVWDELOLW\

2QH PDMRU ILQGLQJ LV WKDW VXFFHVV UDWHV RQ WKH GLIIHUHQW FRXUVHV DQG SURJUDPPHV YDULHGFRQVLGHUDEO\RYHUWKH\HDUVRIWKHVXUYH\DQGPRUHWKDQFRXOGEHDFFRXQWHG IRUE\FKDQJHVWRWHDFKLQJWHDFKHUVRUVWXGHQWFRKRUWRURWKHUIDFWRUV7KLVLPSOLHVD VXEVWDQWLDO DPRXQW RI PRUHRUOHVV UDQGRP YDULDWLRQ DW FRXUVH RU SURJUDPPH OHYHO 7KHUHDUHWZRIXUWKHULPSOLFDWLRQV)LUVWWKHGHJUHHRI\HDURQ\HDUYDULDWLRQWHQGVWR µPXGG\¶WKHGDWDDQGLPSRVHVDUHTXLUHPHQWIRUDFDXWLRXVDQGFDUHIXOLQWHUSUHWDWLRQ

6HFRQG DQG LQ WKH FRQWH[W RI PDQDJHPHQW FRQFHUQV WR LPSURYH SHUIRUPDQFH WRRPXFKVLJQLILFDQFHVKRXOGQRWEHDWWDFKHGWRDVLQJOHDQGSHUKDSVXQW\SLFDO \HDU¶VUHWHQWLRQDQGDFKLHYHPHQWUDWHVIRUDQLQGLYLGXDOFRXUVH

6XFFHVVUDWHVYDULHGFRQVLGHUDEO\IURP\HDUWR\HDU2QWKHDVVXPSWLRQWKDWµVWDELOLW\¶ LPSOLHVDQDQQXDOXSZDUGRUGRZQZDUGPRYHPHQWRIQRPRUHWKDQSHUIRUPDQFH DWFRXUVHOHYHOVHHPVWREHTXLWHXQVWDEOH6SHFLILFDOO\

N RQO\DTXDUWHURIFRXUVHVKDGVWDEOHVXFFHVVUDWHVRYHUDOOWKUHH \HDUV

N MXVWRYHUKDOIKDGVXFFHVVUDWHVZKLFKZHUHVWDEOHRYHUWZR \HDUV

N MXVWRYHUDILIWKRIFRXUVHVKDGVXFFHVVUDWHVZKLFKYDULHG E\PRUHWKDQLQWZRVXFFHVVLYH\HDUV

(16)

N ZKHUH VXFFHVV UDWHV ZHUH VWDEOH IRU WZR \HDUV WKH µURJXH¶ \HDU FRXOGEHLQDQ\RQHRIWKHWKUHH\HDUVDQGWKHµURJXH¶VXFFHVVUDWH FRXOGEHDERYHRUEHORZWKRVHRIWKHRWKHUWZR\HDUV

N ZKHUHVXFFHVVUDWHVZHUHXQVWDEOHWKLVFRXOGEHZLWKDQXSZDUG RUGRZQZDUGWUHQGRUZLWKQRGLVFHUQLEOHWUHQGHLWKHUZD\

7KHH[WHQWRI\HDURQ\HDUYDULDWLRQSURPSWVWKHTXHVWLRQGRHVWKHSUHVHQFHRUODFNRI VWDELOLW\FRORXUWHDFKHUV¶YLHZV"7RLQYHVWLJDWHWKLVTXHVWLRQWZRIXUWKHUDQDO\VHVZHUH XQGHUWDNHQ ZLWK D VSHFLILF IRFXV RQ WKRVH KLJK DQG ORZ EDVHOLQH FRXUVHV WKDW KDG

PDLQWDLQHG WKHLU SHUIRUPDQFH RYHU WKH \HDUV 7KH K\SRWKHVLV WR EH WHVWHG ZDV ZKHWKHUJUHDWHUVWDELOLW\ZDVDVVRFLDWHGZLWKDPRUHSRVLWLYHDVVHVVPHQWRIWKHLPSDFW RIYDULRXVFKDQJHIDFWRUV

7KLVSURYHGQRWWREHWKHFDVH7KHUHZHUHUHODWLYHO\IHZRFFDVLRQVZKHUHWKHYLHZVRI WHDFKHUV IURP WKH PRUH VWDEOH FRXUVHV SURYHG WR EH GLIIHUHQW IURP WKH RWKHUV LQ D VWDWLVWLFDOO\VLJQLILFDQWZD\HYHQDWWKHOHYHO(YHQZKHUHGLIIHUHQFHVFRXOGEH GHWHFWHGWKHUHZDVQRSURQRXQFHGSDWWHUQWHDFKHUVRQWKHPRUHVWDEOHFRXUVHVZHUH PRUHRSWLPLVWLFDERXWVRPHDQGOHVVRSWLPLVWLFDERXWRWKHUFKDQJHIDFWRUVFRPSDUHG ZLWKWHDFKHUVRQFRXUVHVZLWKOHVVFRQVLVWHQWRXWFRPHV

5DWHVRIFKDQJHRYHUWKH\HDUSHULRG

%HIRUHDVNLQJWHDFKHUVWRDVVHVVWKHSRVLWLYHRUQHJDWLYHLPSDFWRI FKDQJHVWKH\ZHUH DVNHG WR VFRUH HDFK IDFWRU IRU WKH UDWH RI FKDQJH 7KH FKRLFH WKDW ZDV RIIHUHG ZDV EHWZHHQ OLWWOH RU QR FKDQJH DQG VLJQLILFDQW FKDQJH 6RPHZKDW WR RXU VXUSULVH PRVW UHVSRQGHQWVVFRUHGPRVWIDFWRUVDVKDYLQJFKDQJHGRQO\DOLWWOHRUQRWDWDOO

)DFWRUVWKDWDWKLUGRUPRUHRIUHVSRQGHQWVLGHQWLILHGDVFKDQJLQJVLJQLILFDQWO\DUHOLVWHG EHORZ7KHILJXUHVLQEUDFNHWVUHIHUWRWKHSHUFHQWDJHRIUHVSRQGHQWVZKRDJUHHGWKDW WKH IDFWRU KDG FKDQJHG VLJQLILFDQWO\ $V ZH VKDOO VHH PRVW RI WKH FKDQJHV VHHQ DV VLJQLILFDQW DUH DOVR LGHQWLILHG ZLWK D QHJDWLYH LPSDFW RQ UHWHQWLRQ DQG DFKLHYHPHQW 2QO\ WKUHH FKDQJHV DUH JHQHUDOO\ VHHQ DV KDYLQJ D SRVLWLYH HPSKDVLV RQ LPSURYLQJ WHDFKLQJDQGOHDUQLQJREVHUYDWLRQRIWHDFKLQJDQGOHDUQLQJDQGDPRXQWRIVXSSRUWIRU DWULVNVWXGHQWV7KHFKDQJHVWKDWPRVWUHVSRQGHQWVDJUHHGKDGFKDQJHGVLJQLILFDQWO\ LQFOXGHGDPRXQWRIFRXUVHDGPLQLVWUDWLRQWLPHIRUSUHSDUDWLRQPRUDOHDQGQXPEHURI WHDFKLQJKRXUV$OORIWKHVHDUHVHHQDVKDYLQJDQDGYHUVHHIIHFWRQVWXGHQWUHWHQWLRQ DQGDFKLHYHPHQW

N DPRXQWRIFRXUVHDGPLQLVWUDWLRQ

N WLPH\RXKDYHDYDLODEOHIRUSUHSDUDWLRQ

N PRUDOHZLWKLQ\RXUWHDP

N PRUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

N PHPEHUVKLSRI\RXUFRXUVHSURJUDPPHWHDP

N QXPEHURIWHDFKLQJKRXUV\RXKDYH

N HPSKDVLVSODFHGRQLPSURYLQJWHDFKLQJDQGOHDUQLQJ

(17)

N DELOLW\RIVWXGHQWV

N DPRXQWRISDLGSDUWWLPHZRUNGRQHE\VWXGHQWV

N DUUDQJHPHQWVIRUNH\VNLOOV

N \RXUGHSDUWPHQWDOIDFXOW\KHDGVKLS

N DGPLQLVWUDWLYHFOHULFDOVXSSRUW

N WLPHDYDLODEOHIRUWHDPPHHWLQJV

,QUHVSRQVHWRWKHILQDOVXPPDWLYHTXHVWLRQUHVSRQGHQWVZHUHDVNHGWRLGHQWLI\WKH FKDQJHVKDYLQJWKHODUJHVWQHJDWLYHRUSRVLWLYHLPSDFW$JDLQILJXUHVLQEUDFNHWVUHIHU WR WKH SHUFHQWDJH RI UHVSRQGHQWV ZKR LGHQWLILHG WKH FKDQJH DV KDYLQJ D SDUWLFXODUO\ QHJDWLYHLPSDFW

N UHVRXUFHV

N VXSSRUWIRUWHDFKHUV

N VWXGHQWFRKRUW

N WLPHWDEOLQJ

N FRXUVHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

7KHFKDQJHVZLWKWKHPRVWSRVLWLYHLPSDFWZHUHLGHQWLILHGDVIROORZV

N VXSSRUWIRUVWXGHQWV

N WHDFKLQJ

N SHUVRQDOWXWRULQJ

N VWDIILQJ

N DVVHVVPHQWDQGIHHGEDFN

(18)

5HVSRQVHVE\FDWHJRU\RIFRXUVHSURJUDPPH

7KH QH[W VWDJH RI WKH DQDO\VLV H[DPLQHG VXUYH\ UHVSRQVHV E\ WKH VL[ FDWHJRULHV RI FRXUVH IURP KLJK EDVHOLQH LPSURYLQJ WR ORZ EDVHOLQH GHFOLQLQJ $V H[SHFWHG WKHUH ZHUH GLIIHUHQW UHVSRQVHV IURP WKH GLIIHUHQW FDWHJRULHV &RQWUDU\ WR H[SHFWDWLRQV KRZHYHU WKH GLIIHUHQFHV ZHUH QRW YHU\ ODUJH ,QGHHG IHZ RI WKH GLIIHUHQFHV ZHUH VLJQLILFDQWDWWKHOHYHO7KHPRVWXQH[SHFWHGDQGFRXQWHULQWXLWLYHILQGLQJLVWKDWWKH UHVSRQVHV WKDW UHVHPEOH HDFK RWKHU PRVW FORVHO\ DUH IURP KLJK EDVHOLQH LPSURYLQJ FRXUVHVDQGORZEDVHOLQHGHFOLQLQJFRXUVHV7KHYLHZVRIUHVSRQGHQWVIURPWKHVHWZR FDWHJRULHVRIFRXUVHFDQEHGHVFULEHGDVWKHPRVWµRSWLPLVWLF¶RUµSRVLWLYH¶$WWKHRWKHU HQGRIWKHVSHFWUXPWKHYLHZVRIVWDIIIURPFRXUVHVLQWKHORZEDVHOLQHLPSURYLQJDQG KLJKEDVHOLQHGHFOLQLQJFDWHJRULHVDOVRUHVHPEOHHDFKRWKHU

7DEOH EHORZ JLYHV WKH PHDQ VFRUH IRU HDFK RI WKH W\SHV RI FKDQJH JLYHQ E\ UHVSRQGHQWV LQ GLIIHUHQW FDWHJRULHV RI FRXUVH 7KH FKDQJHV WKDW KDYH WKH PRVW SRVLWLYHLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQWDUHDVIROORZV

N FRQWH[WRI\RXUFRXUVH N DVVHVVPHQWDQGIHHGEDFN N TXDOLW\V\VWHPV

N VXSSRUWIRUVWXGHQWV N WHDFKLQJ

N SHUVRQDOWXWRULQJ

7DEOH2YHUDOODVVHVVPHQWRIWKHLPSDFWRI FKDQJHIDFWRUVE\FDWHJRU\RIFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

&XUULFXOXP

&RQWH[WRIWKHFRXUVHSURJUDPPH

$VVHVVPHQWDQGIHHGEDFN

6WDIILQJ

4XDOLW\V\VWHPV

5HVRXUFHV

6WXGHQWFRKRUW

6XSSRUWIRUVWXGHQWV

6XSSRUWIRUWHDFKHUV

7HDFKLQJ

3HUVRQDOWXWRULQJ

&RXUVHSURJUDPPHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

7LPHWDEOLQJ

.(<

+, +LJKEDVHOLQHLPSURYLQJ /, /RZEDVHOLQHLPSURYLQJ

+0 +LJKEDVHOLQHPDLQWDLQLQJ /0 /RZEDVHOLQHPDLQWDLQLQJ

(19)

7KLVLVEURDGO\FRQVLVWHQWZLWKWKHPRUHLPSUHVVLRQLVWLFVXPPDWLYHMXGJHPHQWVPDGH E\ UHVSRQGHQWV LQ WKH SUHYLRXV VHFWLRQ RI µUDWHV RI FKDQJH RYHU WKH \HDU SHULRG¶ 'LIIHUHQWSDWWHUQVRIUHVSRQVHIURPGLIIHUHQWFDWHJRULHVRIFRXUVHDUHDOVRDSSDUHQW,Q JHQHUDOWKHKLJKHUVRPRUHRSWLPLVWLFRUSRVLWLYHUDWLQJVRIFKDQJHDUHIURPWKHKLJK EDVHOLQH LPSURYLQJ DQG WKH ORZ EDVHOLQH GHFOLQLQJ FRXUVHV 7KH ORZHVW UDWLQJV DUH IURPWKHKLJKEDVHOLQHGHFOLQLQJDQGWKHORZEDVHOLQHLPSURYLQJFRXUVHV

6LPLODULWLHVDQGGLIIHUHQFHVLQWKHUHVSRQVHVIURPGLIIHUHQWFDWHJRULHVRIFRXUVHFDQEH VHHQHYHQPRUHFOHDUO\LQ)LJXUH*LYHQWKDWWKHUHVSRQVHVZKLFKGLIIHUHGPRVWZHUH IURPKLJKEDVHOLQHLPSURYLQJDQGKLJKEDVHOLQHGHFOLQLQJFRXUVHVWKHLWHPVZKHUH WKHGLIIHUHQFHZDVJUHDWHVWZHUHSORWWHGILUVW7KHFRUUHVSRQGLQJUHVSRQVHVIURPRWKHU FDWHJRULHVRIFRXUVHZHUHDOVRSORWWHGIRUIXUWKHUSRLQWVRIFRPSDULVRQ

)LJXUH+RZFROOHJHVLPSURYH

.(<

% &RQWH[WRIFRXUVHSURJUDPPH' 6WDIILQJ ) 5HVRXUFHV

* 6WXGHQWFRKRUW / &RXUVHSURJUDPPHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

%:KHWKHUH[SHULPHQWVLQ\RXUWHDFKLQJDUHHQFRXUDJHG ' 7KHPHPEHUVKLSRI\RXUFRXUVHSURJUDPPHWHDP * µ2WKHU¶VWXGHQWFRKRUWIDFWRU

/ µ2WKHU¶FRXUVHSURJUDPPHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQIDFWRU )µ2WKHU¶UHVRXUFHIDFWRU

% µ2WKHU¶FRQWH[WIDFWRU

) $YDLODELOLW\RIHVVHQWLDOHTXLSPHQWIRU\RXUFRXUVHSURJUDPPH % 0RUDOHZLWKLQ\RXUWHDP

/ $PRXQWRIFRXUVHDGPLQLVWUDWLRQ * )LQDQFLDOSUREOHPVRIVWXGHQWV

% 0RUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

(20)

,I WKH RULJLQDO K\SRWKHVHV KDG EHHQ FRUUHFW WKH ORZ EDVHOLQH LPSURYLQJ UHVSRQVHV ZRXOGKDYHEHHQFORVHUWRWKHKLJKEDVHOLQHLPSURYLQJDQGWKHORZEDVHOLQHGHFOLQLQJ UHVSRQVHV ZRXOG KDYH EHHQ PRVW VLPLODU WR WKRVH IURP KLJK EDVHOLQH GHFOLQLQJ FRXUVHV$VWKHILJXUHPDNHVFOHDUWKLVLVQRWWKHFDVH:LWKWKHQRWDEOHH[FHSWLRQRI YLHZVRIFKDQJHVWRWKHDPRXQWRIFRXUVHDGPLQLVWUDWLRQUHVSRQVHVIURPORZEDVHOLQH GHFOLQLQJ FRXUVHV DUH TXLWH VLPLODU WR WKRVH IURP KLJK EDVHOLQH LPSURYLQJ FRXUVHV &RQYHUVHO\ DFURVV PRVW RI WKH LWHPV WKH UHVSRQVHV IURPORZ EDVHOLQH LPSURYLQJ FRXUVHVDUHTXLWHVLPLODUWRWKRVHIURPKLJKEDVHOLQHGHFOLQLQJFRXUVHV7KHVHILQGLQJV GHPDQGIXUWKHUDQDO\VLVDQGSURYLGHWKHIRFXVIRUWKHTXDOLWDWLYHSKDVHRIWKHUHVHDUFK ZKLFKLVGLVFXVVHGLQ3DUW

7KHFKDQJHVZLWKWKHPRVWQHJDWLYHLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQWDUH

N UHVRXUFHV N VWXGHQWFRKRUW N VXSSRUWIRUWHDFKHUV

N FRXUVHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ N WLPHWDEOLQJ

7KH IRXU FKDQJHV DVVRFLDWHG ZLWK WKH OHDVW SRVLWLYHPRVW QHJDWLYH LPSDFW EURDGO\ PDWFK WKH FKDQJHV LGHQWLILHG PRUH LPSUHVVLRQLVWLFDOO\ LQ WKH UHVSRQVH WR WKH ILQDO VXPPDWLYH TXHVWLRQ LQ WKH VXUYH\ 7DEOH DOVR VKRZV WKH IDLUO\ VPDOO GLIIHUHQFHV VHSDUDWLQJ WKH GLIIHUHQW FDWHJRULHV RI FRXUVH )RXU W\SHV RI FKDQJH DUH UDWHG SDUWLFXODUO\QHJDWLYHO\E\UHVSRQGHQWVIURPKLJKEDVHOLQHGHFOLQLQJFRXUVHV

N FRXUVHFRQWH[W

N VWXGHQWFRKRUW

N FRXUVHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

&KDQJHVWRUHVRXUFHVDUHUDWHGPRVWQHJDWLYHO\E\UHVSRQGHQWVIURPDOOFDWHJRULHVRI FRXUVH5HVSRQGHQWVIURPKLJKEDVHOLQHLPSURYLQJFRXUVHVKRZHYHUPDGHWKHOHDVW QHJDWLYH DVVHVVPHQW RI WKH LPSDFW RI FKDQJHV WR UHVRXUFHV 7KH PRVW SRVLWLYH

MXGJHPHQW FRQFHUQLQJ DQ\ FKDQJHV PDGH E\ UHVSRQGHQWV LQ WKH ORZ EDVHOLQH

(21)

)LQGLQJVIRUVSHFLILFFKDQJHIDFWRUV

&XUULFXOXP

&XUULFXOXPFKDQJHVWKDWZHUHUDWHGPRVWSRVLWLYHO\ZHUH

N VFKHPHRIZRUN

N WKH ZD\ LQ ZKLFK WKH VFKHPHV RI ZRUN DUH LQWURGXFHG WR VWXGHQWV

N WXWRULQJDUUDQJHPHQWV

&XUULFXOXPFKDQJHVWKDWZHUHUDWHGOHDVWSRVLWLYHO\ZHUH

N GHJUHHRIGLIILFXOW\RIWKHFRXUVH N DUUDQJHPHQWVIRUNH\VNLOOV

+LJK EDVHOLQH LPSURYLQJ FRXUVHV ZHUH SDUWLFXODUO\ SRVLWLYH DERXW FKDQJHV WR WKH VFKHPHRIZRUNDQGWKHZD\LQZKLFKVFKHPHVRIZRUNZHUHLQWURGXFHGWRVWXGHQWV 7KH VRUWV RI FKDQJH LQ VFKHPHV RI ZRUN ZKLFK ZHUH LGHQWLILHG E\ KLJK EDVHOLQH LPSURYLQJFRXUVHVDVKDYLQJDSDUWLFXODUO\SRVLWLYHLPSDFWLQFOXGH

N WKHLQFOXVLRQRIDSHUVRQDOOHDUQLQJMRXUQDO

N DODUJHQXPEHURIHQULFKPHQWRSSRUWXQLWLHV

N ZLGHUFKRLFHLQRSWLRQPRGXOHV

N WKHLQWURGXFWLRQRIH[DPVDWWKHHQGRIWKHILUVW\HDU

N UHSODFHPHQWRIZHHNPRGXOHVE\VKRUWHUZHHNPRGXOHV

7DEOHEHORZVKRZVWKHDVVHVVPHQWRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFXUULFXOXP FKDQJHE\GLIIHUHQWFDWHJRULHVRIFRXUVH

7DEOH0HDQUDWLQJVIRUFXUULFXOXPIDFWRUV DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

&RXUVHV\OODEXVFRQWHQW

&RXUVHSURJUDPPHVWUXFWXUH

'HJUHHRIGLIILFXOW\RIFRXUVHSURJUDPPHIRUVWXGHQWV

$ZDUGLQJERG\

6FKHPHRIZRUN

:D\LQZKLFKVFKHPHLQWURGXFHGWRVWXGHQWV

$PRXQWRIHQULFKPHQWOLQNHGWRFRXUVHSURJUDPPH

$PRXQWRIHQULFKPHQWQRWOLQNHGWRFRXUVHSURJUDPPH

7XWRULQJDUUDQJHPHQWV

$UUDQJHPHQWVIRUNH\VNLOOV

(22)

.(<

+0 +LJKEDVHOLQHPDLQWDLQLQJ /0 /RZEDVHOLQHPDLQWDLQLQJ

+' +LJKEDVHOLQHGHFOLQLQJ /' /RZEDVHOLQHGHFOLQLQJ

/RZEDVHOLQHLPSURYLQJFRXUVHVZHUHOHDVWSRVLWLYHDERXWFKDQJHVWRFRXUVHFRQWHQW DQGµRWKHUFXUULFXOXPIDFWRUV¶µ2WKHUFXUULFXOXPIDFWRUV¶LQWKLVFRQWH[WLQFOXGH

N WKHFHQWUDOLVDWLRQRIWLPHWDEOLQJ

N H[FHVVLYHVWXGHQWZRUNORDG

N UHGXFWLRQLQFRXUVHKRXUV

N WKH WHQVLRQ EHWZHHQ \HDU UHJLVWUDWLRQV ZLWK DQ DZDUGLQJ ERG\ DQGWKHQHHGIRUVWXGHQWVWRFRPSOHWHSRUWIROLRVZLWKLQWKH\HDUV RIWKHLUFRXUVH

/RZEDVHOLQHGHFOLQLQJFRXUVHVUDWHGFXUULFXOXPFKDQJHVOHVVSRVLWLYHO\WKDQWKH\GLG RWKHU VRUWV RI FKDQJHV DQG ZHUH OHDVW SRVLWLYH DERXW WKH LPSDFW RI FKDQJHV WR WKH DPRXQW RI HQULFKPHQW OLQNHG WR FRXUVH SURJUDPPH DQG DUUDQJHPHQWV IRU NH\ VNLOOV 7\SLFDOFRPPHQWVLQFOXGHG

6WXGHQWV ZHUH HQWHUHG IRU NH\ VNLOOV IRU WKH ODVW \HDUV ZKLFK KDVDGGHGH[WUDKRXUVWRWKHLUWLPHWDEOHDQGUHTXLUHGDQGH[WUD ZRUNIRUWKHPWRFRPSOHWH

7KHUHKDVEHHQODFNRIVXSSRUW«DVWKHYRFDWLRQDOVWDIIDUHQRZ KDYLQJWRGHOLYHUDVVHVVDQGUHFRUGDOOWKHNH\VNLOODUHDVQRWH NH\VNLOOVDUHDJRRGLGHDEXWVKRXOGEHFRXUVHVSHFLILF

&RQWH[WRIFRXUVHVSURJUDPPHV

&KDQJHVWRFRXUVHFRQWH[WZKLFKZHUHUDWHGPRVWSRVLWLYHO\ZHUH

N WKHHPSKDVLVSODFHGRQLPSURYLQJWHDFKLQJDQGOHDUQLQJ N KRZVXFFHVVHVDUHGHDOWZLWKE\WKHWHDPOHDGHU

N KRZSUREOHPVDUHGHDOWZLWKE\WKHWHDPOHDGHU N ZKHWKHULQQRYDWLRQLQWHDFKLQJLVHQFRXUDJHG

N TXDOLW\ RI LQWHUQDO FRPPXQLFDWLRQV ZLWKLQ WKH WHDFKLQJ WHDP

&RXUVHFRQWH[WFKDQJHVWKDWZHUHUDWHGOHDVWSRVLWLYHO\ZHUH

N PRUDOHZLWKLQWKHWHDP

(23)

7DEOHEHORZVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJHVWR FRXUVHFRQWH[WE\GLIIHUHQWFDWHJRULHVRIFRXUVH

7DEOH0HDQUDWLQJVIRUFRQWH[WRI

FRXUVHSURJUDPPHIDFWRUVDFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

4XDOLW\RILQWHUQDOFRPPXQLFDWLRQVZLWKLQ\RXUWHDP

&ODULW\RIZKDWLVH[SHFWHGRI\RXE\\RXUOLQHPDQDJHU

+RZVXFFHVVHVDUHGHDOWZLWKE\FRXUVHOHDGHU

+RZSUREOHPVDUHGHDOWZLWKE\FRXUVHOHDGHU

0RUDOHZLWKLQ\RXUWHDP

0RUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

:KHWKHULQQRYDWLRQLQ\RXUWHDFKLQJLVHQFRXUDJHG

(PSKDVLVSODFHGRQLPSURYLQJWHDFKLQJDQGOHDUQLQJ

$Q\RWKHUFRQWH[WIDFWRU

.(<

7KH OHDVW SRVLWLYH DVVHVVPHQW RI WKH LPSDFW RI FKDQJHV WR WKH FRXUVH FRQWH[W ZDV PDGH E\ WKH KLJK EDVHOLQH GHFOLQLQJ FRXUVHV ZKLFK DOVR PDGH WKH PRVW QHJDWLYH DVVHVVPHQW RI WKH LPSDFW RI FKDQJHV WR PRUDOH DQG µRWKHU FRQWH[W IDFWRUV¶ µ2WKHU FRXUVHFRQWH[WIDFWRUV¶LGHQWLILHGE\KLJKEDVHOLQHGHFOLQLQJFRXUVHVLQFOXGH

N ODFNRIVWDIIGHYHORSPHQW

N SRRUDYDLODELOLW\RIHTXLSPHQWDQGDFFRPPRGDWLRQ

N VWDIILQJOHYHOV

N GHWHULRUDWLRQLQUHVRXUFLQJ

N IHDURIFORVXUHUHGXQGDQF\

7KHIODYRXURIWKHSHVVLPLVPFRQFHUQLQJFKDQJHVWRFRXUVHFRQWH[WDPRQJWHDFKHUV RQKLJKEDVHOLQHGHFOLQLQJFRXUVHVFDQEHFRQYH\HGE\WKH IROORZLQJFRPPHQWV

&ROOHJHPRUDOHKDVEHHQVHYHUHO\GDPDJHGGXHWRUHVWUXFWXULQJ WKLV KDV FRPPXQLFDWHG LWVHOI WR VWXGHQWV ZKR KDYH EHHQ PDGH XQFHUWDLQRIWKHIXWXUH

0RUDOH ZLWKLQ WKH GHSDUWPHQW LV YHU\ ORZ 1R FRPPXQLFDWLRQ RU YHU\OLWWOHLVHQFRXUDJHG

(24)

1HZ RU ZHDNHU PHPEHUV RI WHDPV >DUH@ VFRUQHG RU PHHWLQJV KHOGZKHQWKH\FDQQRWHDVLO\DWWHQGRUWKH\DUHQRWLQIRUPHG

7KHPRVWSRVLWLYHDVVHVVPHQWVRIFKDQJHVWRFRXUVHFRQWH[WZHUHPDGHE\WKHKLJK EDVHOLQH LPSURYLQJ DQG ORZ EDVHOLQH GHFOLQLQJ FRXUVHV 7KH VRUWV RI FRXUVH FRQWH[W FKDQJHVZLWKWKHPRVWSRVLWLYHLPSDFWZHUHLGHQWLILHGDV

N UHFRJQLWLRQDQGDSSUHFLDWLRQRIVXFFHVV

N HQFRXUDJHPHQWE\WKHKHDGRIFHQWUH

N FRQVLVWHQF\ZLWKLQWKHWHDP

N FKDQJHRIFRXUVHWHDPOHDGHU

N QHZPRQLWRULQJSURFHGXUHV

N PRUHUHJXODUWHDPPHHWLQJV

2SHQHQGHGFRPPHQWVIURPKLJKEDVHOLQHLPSURYLQJFRXUVHWHDPPHPEHUVLQFOXGH

6XFFHVVLVPXFKPRUHUHFRJQLVHGDQGDSSUHFLDWHG

,QQRYDWLRQ LQ WHDFKLQJ LV DOZD\V HQFRXUDJHG E\ WKH KHDG RI FHQWUH

0RUDOHKDVGURSSHGEHFDXVHRIFKDQJHVLQFROOHJHVWUXFWXUHEXW VWXGHQWVKDYHQRWVXIIHUHG¶

7KH FRXUVH FRQWH[W FKDQJH ZKLFK ZDV LGHQWLILHG E\ UHVSRQGHQWV IURP KLJK EDVHOLQH LPSURYLQJFRXUVHVDVKDYLQJWKHPRVWEHQHILFLDOLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQW ZDVWKHHPSKDVLVSODFHGRQLPSURYLQJWHDFKLQJDQGOHDUQLQJ$PRQJWKHFKDQJHVWR FRXUVHFRQWH[WHPSKDVLVZDVSODFHGRQLPSURYLQJWHDFKLQJDQGUHVSRQGHQWVIURPDOO

FDWHJRULHVYLHZHGOHDUQLQJPRVWRSWLPLVWLFDOO\

$VVHVVPHQWDQGIHHGEDFN

&KDQJHV WR DVVHVVPHQW DQG IHHGEDFN IDFWRUV WKDW ZHUH UDWHGPRVWSRVLWLYHO\ ZHUH

N KHOSIXOQHVVRIIHHGEDFNWRVWXGHQWV N VWXGHQWV¶XQGHUVWDQGLQJRIDVVHVVPHQW N PHWKRGVRIDVVHVVPHQW

N RWKHUDVVHVVPHQWIHHGEDFNIDFWRUV

7KHVRUWVRIRWKHUFKDQJHVWRDVVHVVPHQWIHHGEDFNZKLFKZHUHLGHQWLILHGDVKDYLQJD EHQHILFLDOHIIHFWRQUHWHQWLRQDQGDFKLHYHPHQWLQFOXGH

(25)

N VWDQGDUGLVDWLRQRIDVVLJQPHQWEULHIV

N JUHDWHUVWDIIIDPLOLDULW\ZLWKDVVHVVPHQWWDVNV

N LQWURGXFWLRQRIDSLORWIHHGEDFNIRUPIRUXVHE\WHDFKHUV

6RPH RI WKH HYLGHQFH LV UDWKHU DPELJXRXV 0RVW RI WKH µRWKHU¶ DVVHVVPHQW IDFWRUV LGHQWLILHGWHQGWRKDYHDQHJDWLYHLPSDFW7KHVHLQFOXGH

N ODFNRIFOHDUSROLFLHVIRUFRXUVHZRUNGHDGOLQHV

N H[FHVVLYHZRUNORDGRQWHDFKHUV

N IHZDVVHVVRUV

N GHOD\VLQUHWXUQLQJZRUNWRVWXGHQWV

N LQFUHDVHVLQVWDIIZRUNORDG

7KHUH ZHUH QR FKDQJHV WR DVVHVVPHQW DQG IHHGEDFN IDFWRUV WKDW ZHUH UDWHG SDUWLFXODUO\QHJDWLYHO\7KHPRVWSRVLWLYHHYDOXDWLRQVRYHUDOORIWKHLPSDFWRIFKDQJHV WRDVVHVVPHQWDQGIHHGEDFNZHUHPDGHE\KLJKEDVHOLQHLPSURYLQJDQGORZEDVHOLQH GHFOLQLQJFRXUVHV([DPSOHVRISDUWLFXODUO\SRVLWLYHFKDQJHVLQFOXGHG

N LPSURYHPHQWVWRDVVHVVPHQWPHWKRGVDIWHUDOHFWXUHUEHFDPHDQ H[DPLQHUIRURQHRIWKHPRGXOHV

N QRWLFHERDUGGLVSOD\VRIZRUNGXHE\WKHHQGRIHDFKKDOIWHUP

N SURYLVLRQRIIHHGEDFNRQWHVWSDSHUVWRVWXGHQWV

N XVHRIVWDQGDUGIHHGEDFNVKHHWVIRUDOOZULWWHQZRUNE\VWXGHQWV

7DEOHEHORZVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJHVWR DVVHVVPHQWDQGIHHGEDFNE\GLIIHUHQWFDWHJRULHVRIFRXUVH

7DEOH0HDQUDWLQJVIRUDVVHVVPHQWDQG IHHGEDFNIDFWRUVDFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

6WXGHQWVXQGHUVWDQGLQJRIDVVHVVPHQW

$PRXQWRIDVVHVVPHQW

0HWKRGVRIDVVHVVPHQW

6FKHGXOLQJRIDVVHVVPHQW

3ROLF\RQGHDGOLQHVIRUKDQGLQJZRUNLQ

$PRXQWRISULYDWHVWXG\KRPHZRUN

+HOSIXOQHVVRIIHHGEDFNWRVWXGHQWV

$Q\RWKHUDVVHVVPHQWIHHGEDFNIDFWRU

.(<

(26)

6WDIILQJ

7KHFKDQJHVWRVWDIILQJZKLFKZHUHUDWHGPRVWSRVLWLYHO\ZHUH

N PL[RIVNLOOVLQWKHWHDP N FRXUVHWHDPOHDGHUVKLS N PHPEHUVKLSRIWKHWHDP

6WDIILQJFKDQJHVZKLFKZHUHUDWHGPRVWQHJDWLYHO\ZHUH

N DQ\RWKHUVWDIILQJIDFWRU N DPRXQWRIVWDIIDEVHQFH N VHQLRUFXUULFXOXPOHDGHUVKLS

µ2WKHUVWDIILQJIDFWRUV¶LQWKLVFRQWH[WLQFOXGH

N DODUJHLQIOX[RISDUWWLPHVWDII

N LQFUHDVHVLQVWDIIZRUNORDG

N VWDII WXUQRYHU OHDGLQJ WR LQFRQVLVWHQW SUDFWLFHV DQG D ODFN RI VWXGHQWFRQILGHQFHLQVRPHWXWRUV

N ODUJHDQGXQZLHOG\FRXUVHWHDPV

N GLIILFXOWLHV LQ UHFUXLWLQJ VWDII ZLWK WKH QHFHVVDU\ H[SHULHQFH DQG DELOLWLHV

7KHZRUGVRIRQHUHVSRQGHQWDUHTXLWHUHSUHVHQWDWLYHRIWKHVRUWVRIYLHZVH[SUHVVHG

6WDWLVWLFV VKRZ WKDW ZKHUH VWDII DUH LOO IRU ORQJ SHULRGV ZLWKRXW DGHTXDWHFRYHUPRUDOHDQGZRUNVXIIHUDPRQJVWVWXGHQWV$KLJK SHUFHQWDJH RI SDUWWLPH VWDII ILQG LW GLIILFXOW WR IXOO\ LQWHJUDWH LQWR ZRUNLQJVRIGHSDUWPHQW

7KH JHQHUDO SLFWXUH WKDW HPHUJHV IURP YLHZV RI FKDQJHV WR VWDIILQJ LV RI D UDWKHU EHOHDJXHUHG SURIHVVLRQ ZKLFK LV FRQILGHQW DERXW LWV FUXFLDO FRQWULEXWLRQ WR UHWHQWLRQ DQGDFKLHYHPHQWEXWEHVHWE\GLIILFXOWLHV7KHVSHFWUXPRIYLHZVIURPWKHPRVWWRWKH OHDVWRSWLPLVWLFFDQEHLOOXVWUDWHGE\WKHIROORZLQJFRPPHQWV

9HU\ VNLOOHG ZHOOTXDOLILHG WHDFKHUV DUH DEOH WR VWUHWFK KLJKHU DELOLW\VWXGHQWV>OHDGLQJWR@EHWWHUIXUWKHUPDWKVUHVXOWV

(27)

2QHPHPEHURIVWDIIZDVUHSODFHGE\VRPHRQHZLWKPRUHHQHUJ\ DQG HQWKXVLDVP DQG >«@ WKLV PDGH DQ LPSDFW RQ WKH LQWHUHVW OHYHORIWKHVWXGHQWV

&RQVLGHUDEOHSUREOHPVGXHWRXQVXLWDEOHVWDIIORQJWHUPVLFNQHVV RIRQHPHPEHUDQGUHGXQGDQF\RIDQRWKHU

$OOWKHVWDIIDUHQRZSDUWWLPH7KHUHLVQRGD\ZKHQZHDUHDOOLQ &RPPXQLFDWLRQLVQRWVWUDLJKWIRUZDUG6HFRQG\HDUV>«@KDYHQR RQHWRSXOOWKHLUJUDGHVWRJHWKHU

7DEOHVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIVWDIILQJFKDQJH E\GLIIHUHQWFDWHJRULHVRIFRXUVH

7DEOH0HDQUDWLQJVIRUVWDIILQJ DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

0HPEHUVKLSRI\RXUFRXUVHSURJUDPPHWHDP

0L[RIVNLOOVLQ\RXUFRXUVHSURJUDPPHWHDP

<RXUFRXUVHSURJUDPPHWHDPOHDGHUVKLS

5DWLRRIIXOOWRSDUWWLPHWHDFKHUVLQ\RXUFRXUVHWHDP

$PRXQWRIVWDIIDEVHQFHLQ\RXUWHDP

<RXUGHSDUWPHQWIDFXOW\KHDGVKLS

6HQLRUFXUULFXOXPOHDGHUVKLSLQ\RXUFROOHJH

&RPSRVLWLRQRIWKHJURXSRISHUVRQDOWXWRUV

ZKRWXWRU\RXUVWXGHQWV

$Q\RWKHUVWDIILQJIDFWRU

.(<

4XDOLW\V\VWHPV

7KHRQO\FKDQJHWRTXDOLW\V\VWHPVZKLFKZDVUDWHGQHJDWLYHO\ZDV

N DYDLODELOLW\ RI LQIRUPDWLRQ IURP PDQDJHPHQW LQIRUPDWLRQ V\VWHPV0,6

7KHFKDQJHWRTXDOLW\V\VWHPVWKDWZDVUDWHGPRVWSRVLWLYHO\ZDV

(28)

/RZEDVHOLQHGHFOLQLQJFRXUVHVJDYHDSDUWLFXODUO\QHJDWLYHHYDOXDWLRQRIWKHLPSDFWRI WKHDYDLODELOLW\RILQIRUPDWLRQIURP0,67KHUHZDVDVWDWLVWLFDOO\VLJQLILFDQWGLIIHUHQFH DWWKHOHYHOEHWZHHQWKHUHVSRQVHRIKLJKEDVHOLQHPDLQWHQDQFHFRXUVHVDQGORZ EDVHOLQH LPSURYLQJ FRXUVHV LQ UHVSHFW RI FKDQJHV WR REVHUYDWLRQ RI WHDFKLQJ DQG OHDUQLQJ3RVLWLYHFRPPHQWVPDGHE\KLJKEDVHOLQHPDLQWHQDQFHFRXUVHVLQFOXGHG

«PXFKPRUHREVHUYDWLRQRIWHDFKLQJ«VLJQLILFDQWSRVLWLYHLPSDFW RQWHDFKLQJ±VRPH SDUWWLPH VWDII LGHQWLILHG DV UHTXLULQJ IXUWKHU WUDLQLQJDQGDGYLFH

7HDFKLQJ REVHUYDWLRQV DUH FRPSOHWHG DQQXDOO\ DQG , SHUVRQDOO\ IHHOWKLVKDVDSRVLWLYHLPSDFWRQWKHWHDFKLQJPHWKRGVZLWKLQWKH FROOHJHDQGWKHGHSDUWPHQW

,Q FRQWUDVW RQH UHVSRQGHQW IURP D ORZ EDVHOLQH LPSURYLQJ FRXUVH QRWHG WKH µLQWURGXFWLRQ RI D IDLOHG V\VWHP IRU FODVVURRP REVHUYDWLRQV¶ $QRWKHU UHPDUNHG WKDW µLQFUHDVHG³LQVSHFWLRQ´IRUVHQLRUOHFWXUHUVKDVDGHPRWLYDWLQJHIIHFW±QRWKLQJSRVLWLYH LVDFKLHYHGE\LW¶

7DEOHVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJHWRTXDOLW\ V\VWHPVE\GLIIHUHQWFDWHJRULHVRIFRXUVH

7DEOH0HDQUDWLQJVIRUTXDOLW\V\VWHPV DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

6HOIDVVHVVPHQWSURFHVVIRU\RXUFRXUVHSURJUDPPH

$YDLODELOLW\RILQIRUPDWLRQIURP0,6

2EVHUYDWLRQRIWHDFKLQJDQGOHDUQLQJ

2EVHUYDWLRQRIWXWRULQJ

3URFHVVHVIRUJHQHUDWLQJIHHGEDFNIURPVWXGHQWV

8VH\RXPDNHRIIHHGEDFNIURPVWXGHQWV

:D\VWKDWPDQDJHUVPRQLWRUSHUIRUPDQFH

RQ\RXUFRXUVHSURJUDPPH

$Q\RWKHUTXDOLW\V\VWHPVIDFWRU

.(<

(29)

5HVRXUFHV

$OODVSHFWVRIFKDQJHVWRUHVRXUFHVZHUHHYDOXDWHGQHJDWLYHO\7KHPRVWQHJDWLYH HYDOXDWLRQFRQFHUQ

N WLPH\RXKDYHDYDLODEOHIRUSUHSDUDWLRQ N QXPEHURIWHDFKLQJKRXUV\RXKDYH N DYDLODELOLW\RIHVVHQWLDOHTXLSPHQW

N DYDLODELOLW\RIWHFKQLFDOVXSSRUWDQGEDFNXS

3DUWLFXODUO\ QHJDWLYH HYDOXDWLRQV ZHUH PDGH FRQFHUQLQJ WKH WLPH DYDLODEOH IRU SUHSDUDWLRQDQGWKHQXPEHURIWHDFKLQJKRXUV7\SLFDOFRPPHQWVLQFOXGH

OLPLWHG WLPH IRU JHWWLQJ OHVVRQ UHDG\ VRPHWLPHV PHDQV IHZHU QRWHVPDWHULDOVSUHSDUHG

«WLPH IRU OHVVRQ SUHSDUDWLRQ ZRUN LV UHGXFHG GXH WR WHDFKLQJ KRXUVRWKHUSDSHUZRUNDGPLQLVWUDWLRQ7KLVUHIOHFWVRQWKHQXPEHU RIGLIIHUHQWFRXUVHVWDXJKWE\LQGLYLGXDOWHDFKHUV

,ZRUNKRXUVDZHHNDQGVWLOOQHYHUVHHPWRNHHSDKHDG

+LJK EDVHOLQH GHFOLQLQJ FRXUVHV PDGH QRWDEO\ PRUH QHJDWLYH HYDOXDWLRQV WKDQ RWKHU FDWHJRULHVRIFRXUVHLQUHVSHFWRIWLPHDYDLODEOHIRUSUHSDUDWLRQDQGWKHDYDLODELOLW\RI HVVHQWLDOHTXLSPHQW$VHOHFWLRQRIFRPPHQWVIURPVXFKFRXUVHVJLYHVWKHLPSUHVVLRQ RIVRPHWKLQJDSSURDFKLQJGHVSDLU

'RQRWIHHO,KDYHHQRXJKWLPHWRSUHSDUHOHVVRQVWRWKHEHVWRI P\DELOLW\GXHWRPRXQWDLQVRIUHSHWLWLYHSDSHUZRUN

,¶P ZRUNLQJ KDUGHU WKDQ HYHU EXW DP OHVV SUHSDUHG IRU DOO P\ OHVVRQV±VWXGHQWVQHHGJRRGOHVVRQV

(30)

7DEOH VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI GLIIHUHQW DVSHFWV RI FKDQJH WR UHVRXUFHVE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUUHVRXUFHV DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

7LPH\RXKDYHDYDLODEOHIRUSUHSDUDWLRQ

1XPEHURIWHDFKLQJKRXUV\RXKDYH

1XPEHURIGLIIHUHQWFRXUVHV\RXWHDFKRQ

$YDLODELOLW\RIHVVHQWLDOHTXLSPHQWIRU\RXU

FRXUVHSURJUDPPH

$YDLODELOLW\RIWHFKQLFDOVXSSRUWDQGEDFNXS

$FFRPPRGDWLRQIRU\RXUFRXUVHSURJUDPPH

$Q\RWKHUUHVRXUFHIDFWRU

.(<

6WXGHQWFRKRUW

$OODVSHFWVRIFKDQJHWRWKHVWXGHQWFRKRUWZHUHDOVRHYDOXDWHGQHJDWLYHO\ZLWK WKHH[FHSWLRQRIµRWKHUIDFWRUV¶7KHOHDVWQHJDWLYHHYDOXDWLRQVFRQFHUQFKDQJHVWR WKHFODULW\RISHUVRQDOFDUHHUJRDOVRIVWXGHQWVDQGWKHSURSRUWLRQRIVWXGHQWVOHDYLQJ IRUDMRE7KHPRVWQHJDWLYHHYDOXDWLRQVZHUHPDGHLQUHVSHFWRIFKDQJHVLQWKH

N DPRXQWRISDUWWLPHZRUNGRQHE\VWXGHQWV N ILQDQFLDOSUREOHPVRIVWXGHQWV

N SHUVRQDOHPRWLRQDOKHDOWKSUREOHPVRIVWXGHQWV N OHDUQLQJVNLOOVRIVWXGHQWV

N DELOLW\RIVWXGHQWV

+LJK EDVHOLQH GHFOLQLQJ FRXUVHV PDGH VRPH RI WKHPRVW QHJDWLYH HYDOXDWLRQV RI FKDQJHVWRWKHVWXGHQWFRKRUWSDUWLFXODUO\LQUHVSHFWRI

N ILQDQFLDOSUREOHPV

N DPRXQWRISDUWWLPHZRUN

N SHUVRQDOHPRWLRQDOKHDOWKSUREOHPV

(31)

6WXGHQWVFRPHLOOSUHSDUHGIRUD FRXUVHZKLFKGHPDQGVDKLJK GHJUHH RI VHOIPRWLYDWLRQ ODWHUDO WKLQNLQJ FRPPLWPHQW DQG GHWHUPLQDWLRQ $VLJQLILFDQWVKLIWIURP VWXGHQWV ZLWK $OHYHOVWR *194TXDOLILFDWLRQV6WXGHQWVVHHPWRKDYHIDUPRUHSUREOHPV ILQDQFLDO HPRWLRQDO KHDOWK WKDQ D IHZ \HDUV DJR WKHVH DGYHUVHO\ DIIHFW DWWHQGDQFH PRWLYDWLRQ DQG WKH TXDOLW\ RI PDWHULDOVWKDWVWXGHQWVDUHDEOHWRSXUFKDVHDQGFRQVHTXHQWO\ WKHTXDOLW\RIZRUNWKH\SURGXFH

2YHUDOO VWXGHQWV DUH OHVV PRWLYDWHG WR FRPSOHWH SUHVFULSWLYH FRXUVHZRUN 7KH\ GR QRW ILQG VWXG\ HDV\ DQG DOWKRXJK WKH\ KDYHWKHVDPHQXPEHURI>*&6(@SRLQWVWKH\GRQRWVHHPVR DFDGHPLFDOO\DEOH0DQ\PRUHQHHGOHDUQLQJVXSSRUWDQGVSHQG D ORW RI IUHH WLPH DW SDUWWLPH ZRUN 0DQ\ PRUH KDYH SHUVRQDO SUREOHPVWKDQSUHYLRXVO\

7DEOH VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI GLIIHUHQW DVSHFWV RI FKDQJH WR WKH VWXGHQWFRKRUWE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUVWXGHQWFRKRUW DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

0RWLYDWLRQRIVWXGHQWVRQ\RXUFRXUVHSURJUDPPH

$ELOLW\RIVWXGHQWV

4XDOLILFDWLRQVRIVWXGHQWVDWFRPPHQFHPHQWRI

FRXUVHSURJUDPPH

/HDUQLQJVNLOOVRIVWXGHQWV

&ODULW\RISHUVRQDOFDUHHUJRDOVRIVWXGHQWV

3URSRUWLRQRIVWXGHQWVOHDYLQJIRUDMRE

$PRXQWRISDLGSDUWWLPHZRUNGRQHE\VWXGHQWV

+RZQHDUWRFROOHJHGRVWXGHQWVOLYH

)LQDQFLDOSUREOHPVRIVWXGHQWV

3HUVRQDOHPRWLRQDOKHDOWKSUREOHPVRIVWXGHQWV

$Q\RWKHUVWXGHQWFRKRUWIDFWRU

.(<

(32)

6XSSRUWIRUVWXGHQWV

1R FKDQJHV DVVRFLDWHG ZLWK VXSSRUW IRU VWXGHQWV ZHUH HYDOXDWHG DV KDYLQJ D QHJDWLYHLPSDFW7KHFKDQJHVWKDWZHUHHYDOXDWHGPRVWSRVLWLYHO\ZHUH

N DPRXQWRIVXSSRUWDYDLODEOHWRµDWULVN¶VWXGHQWV N FRXQVHOOLQJDQGJXLGDQFHVXSSRUWDYDLODEOH

N ZD\VWKDWVWXGHQWVDUHHQFRXUDJHGWRVXSSRUWHDFKRWKHU N LQWHJUDWLRQRIOHDUQLQJVXSSRUW

6RPH RI WKH PRVW SRVLWLYH HYDOXDWLRQV ZHUH PDGH E\ WKH ORZ EDVHOLQH GHFOLQLQJ FRXUVHVLQUHVSHFWRIWKHLQYROYHPHQWRISDUHQWVRI\RXQJHUVWXGHQWVILQDQFLDOVXSSRUW DYDLODEOHDQGFRXQVHOOLQJDQGJXLGDQFHVXSSRUWDYDLODEOH

7KHFRPPHQWVSURYLGHGE\DQXPEHURIUHVSRQGHQWVKHOSHGWRLOOXVWUDWHWKRVHDVSHFWV RIVXSSRUWIRUVWXGHQWVWKDWVHHPHGWREHPRVWHIIHFWLYH

,QWHUYLHZ SURFHVV IRU VWXGHQW LQWDNH KDV LPSURYHG FRQVLGHUDEO\ 7KLV LGHQWLILHV VWXGHQWV ZLWK GLIILFXOWLHVSUREOHPV $OVR REVHUYDWLRQV GXULQJ LQGXFWLRQ KDYH KHOSHG WR KLJKOLJKW VWXGHQW SUREOHPV

>VXSSRUW IRU VWXGHQWV@ KDV EHHQ DIIHFWHG IDYRXUDEO\ E\ WKH LPSOHPHQWDWLRQ RI VHQLRU WXWRU >UROH@ LGHQWLILHG DV EHLQJ UHVSRQVLEOHRIVWXGHQWVXSSRUW

(33)

7DEOHVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJHWRVXSSRUW IRUVWXGHQWVE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUVXSSRUWIRUVWXGHQWV DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

,QYROYHPHQWRISDUHQWVRI\RXQJHUVWXGHQWV

:D\VWKDWVWXGHQWVDUHHQFRXUDJHGWRVXSSRUW

HDFKRWKHU

)LQDQFLDOVXSSRUWDYDLODEOH

&RXQVHOOLQJDQGJXLGDQFHVXSSRUWDYDLODEOH

$YDLODELOLW\RIVWDIIWRSURYLGHLQGLYLGXDOVXSSRUW

ZKHQQHHGHG

7DNHXSRIOHDUQLQJVXSSRUWE\VWXGHQWV

,QWHJUDWLRQRIOHDUQLQJVXSSRUWZLWKLQ\RXU

FRXUVHSURJUDPPH

/HQJWKRIWLPHLWWDNHVWRLGHQWLI\DWULVNVWXGHQWV

$PRXQWRIVXSSRUWDYDLODEOHIRUDWULVNVWXGHQWV

0HWKRGVRIDFNQRZOHGJLQJUHZDUGLQJVWXGHQWSURJUHVV

$Q\RWKHUVXSSRUWIRUVWXGHQWVIDFWRU

.(<

(34)

6XSSRUWIRUWHDFKHUV

1R DVSHFWV RI FKDQJHV WR VXSSRUW IRU WHDFKHUV ZHUH HYDOXDWHG DV KDYLQJ D SDUWLFXODUO\ SRVLWLYH LPSDFW RQ VWXGHQW UHWHQWLRQ DQG DFKLHYHPHQW&KDQJHV WR VXSSRUWIRUWHDFKHUVWKDWZHUHHYDOXDWHGSDUWLFXODUO\QHJDWLYHO\ZHUH

N PHFKDQLVPVWRDFNQRZOHGJHUHZDUGHIIHFWLYHWHDFKLQJ N VXSSRUWDYDLODEOHIRU\RXIURPZLWKLQWKHFROOHJH

N DGPLQLVWUDWLYHFOHULFDOVXSSRUW

6RPHRIWKHPRVWQHJDWLYHHYDOXDWLRQVZHUHPDGHE\UHVSRQGHQWVIURPKLJKEDVHOLQH GHFOLQLQJFRXUVHVDERXW

N VXSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUFROOHJH

N PHFKDQLVPVWRDFNQRZOHGJHUHZDUGHIIHFWLYHWHDFKLQJ

N VXSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUWHDP

(YHQDWWKHLUPRVWSRVLWLYHWHDFKHUVRQKLJKEDVHOLQHGHFOLQLQJFRXUVHVDSSHDUWRIHHO XQVXSSRUWHGDQGXQGHUYDOXHG$WWKHLUPRVWQHJDWLYHWKH\IHHOXQGHUPLQHGDQGHYHQ GLVSDUDJHG7KLVUDQJHRIYLHZVFDQEHLOOXVWUDWHGE\WKHIROORZLQJFRPPHQWVIURPWKH TXHVWLRQQDLUHV

7KH FROOHJH VWDII GR WKHLU EHVW WR RIIHU VXSSRUW XQGHU GLIILFXOW FRQGLWLRQV LH VKRUW VWDIILQJ LQ DUHDV VXFK DV FRXQVHOOLQJ RU DGPLQLVWUDWLRQ

6XSSRUWLVMXVWDERXWQRQH[LVWHQW

,GRQ¶WIHHOVXSSRUWHGE\FHQWUDOPDQDJHPHQW±,IHHOGLVSDUDJHG ,QPDQ\ZD\VWKH\FRQYH\>WKHYLHZ@WKDWZHDUHQRWYDOXHG

, IHHO WKDW DQ\ LQLWLDWLYH RU VRPHWKLQJ VOLJKWO\ RXW RI WKH QRUP LV VHHQDVDSUREOHPDGPLQLVWUDWRUVFDQQRWFRSHZLWK

6XSSRUW LV VXEMHFW WR YDULRXV LQWHUSUHWDWLRQV ,W LV SUHVHQW LI \RX WRZWKHOLQHDQGGRQRWSXWIRUZDUGQHZLGHDV

7KHPRVWSRVLWLYHHYDOXDWLRQRIDQ\DVSHFWRIVXSSRUWIRUWHDFKHUVZDVPDGHE\ORZ EDVHOLQHGHFOLQLQJFRXUVHVLQUHVSHFWRIVXSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUWHDP 7KHPRVWIDYRXUDEOHHYDOXDWLRQVRYHUDOORIFKDQJHVWRVXSSRUWIRUWHDFKHUVZHUHPDGH E\ORZEDVHOLQHGHFOLQLQJFRXUVHVDQGKLJKEDVHOLQHLPSURYLQJFRXUVHV

(35)

VXSSRUWDPRQJWHDFKHUVRQORZEDVHOLQHGHFOLQLQJFRXUVHVPD\SHUKDSVLQGLFDWHWKDW VXFK WHDPV VXSSRUW HDFK RWKHU LQ GLYHUVLW\ 7KH SRVLWLYH EXW QRW VWURQJO\ SRVLWLYH HYDOXDWLRQVRIWHDPDQGGHSDUWPHQWDOVXSSRUWIURPKLJKEDVHOLQHLPSURYLQJFRXUVHV PD\LQGLFDWHWKDWPHPEHUVRIVXFKWHDPVHLWKHUWDNHWKHLUWHDPVXSSRUWIRUJUDQWHGRU DOWHUQDWLYHO\WKDWWKH\DUHOHVVGHSHQGHQWRQLW

7KHVHDVVHVVPHQWVRIWKHLPSDFWRIFKDQJHVWRVXSSRUWIRUWHDFKHUVKDYHVRPHTXLWH ZRUU\LQJLPSOLFDWLRQVIRUFROOHJHOHDGHUVKLS,WLVSHUKDSVQDWXUDODQGXQVXUSULVLQJWKDW WHDFKHUVVKRXOGKDYHWKHPRVWSRVLWLYHYLHZRIVXSSRUWIURPWKHLUWHDFKLQJWHDPV,WLV KRZHYHU UDWKHU DODUPLQJ WKDW WKH\ VKRXOG KDYH QHJDWLYH YLHZV RI WKH LPSDFW RI FKDQJHVWRVXSSRUWDYDLODEOHIURPWKHLUFROOHJHRUIURPPHFKDQLVPVWRDFNQRZOHGJH DQGUHZDUGHIIHFWLYHWHDFKLQJ

7DEOH VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI GLIIHUHQW DVSHFWV RI FKDQJH WR VXSSRUWIRUWHDFKHUVE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUVXSSRUWIRUWHDFKHUV DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

6XSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUWHDP

6XSSRUWDYDLODEOHWR\RXIURPZLWKLQGHSWRUIDFXOW\

6XSSRUWDYDLODEOHWR\RXIURPFROOHJH

0HFKDQLVPVWRDFNQRZOHGJHUHZDUGHIIHFWLYHWHDFKLQJ

2SSRUWXQLWLHVDYDLODEOHIRUVWDIIGHYHORSPHQW

$GPLQLVWUDWLYHFOHULFDOVXSSRUW

$Q\RWKHUVXSSRUWIRUWHDFKHUVIDFWRU

.(<

7HDFKLQJ

&KDQJHV WR DVSHFWV RI WHDFKLQJ ZKLFK ZHUH VDLG WR KDYH WKH PRVW SRVLWLYH LPSDFWRQVWXGHQWRXWFRPHVZHUH

N YDULHW\RIWHDFKLQJVWUDWHJLHV

N ZD\V\RXPRQLWRULQGLYLGXDOVWXGHQWSURJUHVV N LQGXFWLRQSURFHVVHV

(36)

7DEOHEHORZVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJH WRWHDFKLQJE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUWHDFKLQJ DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

<RXUUROHDVDWHDFKHU

5HFUXLWPHQWDQGVHOHFWLRQSURFHVVHV

(QWU\UHTXLUHPHQWVDGPLVVLRQVSROLF\

,QGXFWLRQSURFHVVHV

:D\V\RXPRQLWRULQGLYLGXDOVWXGHQWSURJUHVV

9DULHW\RIWHDFKLQJVWUDWHJLHV\RXXVH

$PRXQWRIWHDFKLQJWLPHLQYROYLQJGLIIHUHQWWDVNVIRUORZHU

RUKLJKHUDWWDLQLQJVWXGHQWV

$PRXQWRIWHDFKLQJWLPHDYDLODEOHWRZRUNZLWK

LQGLYLGXDOVWXGHQWV

3URSRUWLRQRIWHDFKLQJWLPHLQYROYLQJDFWLYHOHDUQLQJ

E\VWXGHQWV

<RXULQYROYHPHQWLQGHYHORSPHQWDFWLYLW\LQVLGH

\RXUFROOHJH

<RXULQYROYHPHQWLQSURIHVVLRQDODFWLYLW\RXWVLGH

WKHFROOHJH

$Q\RWKHUWHDFKLQJIDFWRU

.(<

7KHIUHHWH[WFRPPHQWVIURPWKHTXHVWLRQQDLUHVDGGFRQVLGHUDEO\WRWKLVUDWKHUEDUH DQDO\VLV ,W LV FOHDU IURP D QXPEHU RI VXFK FRPPHQWV WKDW FKDQJHV WR UHFUXLWPHQWVHOHFWLRQ DQG LQGXFWLRQ SURFHVVHV DUH JHQHUDOO\ UHYLHZHG DV KDYLQJ D SRVLWLYHHIIHFW)XUWKHUDQXPEHURIWHDFKHUVFRQYH\DVHQVHRIVDWLVIDFWLRQDQGSULGH LQ WKHLU SURIHVVLRQDO GHYHORSPHQW DQG LQGHHG DGYDQFHPHQW 7KHVH DUH YLHZHG DV KDYLQJKHOSHGWHDFKHUVEHWWHUWRDVVLVWWKHLUVWXGHQWV

7KHFRQWULEXWLRQWRGHYHORSLQJDFWLYLW\ERWKLQVLGHFROOHJHDQGWKH RSSRUWXQLW\WRGHYHORSP\RZQSURIHVVLRQDOOLIHRXWVLGHKDVKDGD SRVLWLYHLPSDFWRQWKHFRXUVH

(37)

>0RUDOHZLWKLQP\WHDP@KDVFKDQJHGGUDPDWLFDOO\\HDUVDJR, ZDV DQ LQVWUXFWRU EXW QRZ , DP D IXOOWLPH OHFWXUHU >DQG SURJUDPPHPDQDJHU@>«@DIWHUFRPSOHWLQJD&HUW(G

«\HDUWHDFKLQJH[SHULHQFHLQWKH86$KDVKHOSHGWRLPSURYH WHDFKLQJVNLOOVDQGWHFKQLTXHV

,QYROYHPHQW ZLWK H[DPDZDUGLQJ ERG\ KDV KDG D VLJQLILFDQW SRVLWLYHLPSDFW

$W WKH RWKHU HQG RI WKH VSHFWUXP KRZHYHU FRPPHQWV FRQFHUQLQJ WKH UHGXFWLRQ RI WHDFKLQJ WLPH DYDLODEOH WR ZRUN ZLWK LQGLYLGXDO VWXGHQWV UHVHPEOH DOPRVW H[DFWO\ WKH FRPPHQWVPDGHRQUHVRXUFHVGLVFXVVHGDERYH

5HPRYDO RI ZRUNVKRS WLPH KDV PDGH LW LQFUHDVLQJO\ GLIILFXOW WR KHOSVWXGHQWVZLWKLQGLYLGXDOSUREOHPV

7HDFKLQJWLPHLQWKHFROOHJHLVFXWIRUWKLVFRXUVHDQQXDOO\$VD UHVXOWVRPHRIWKHZHDNHUVWXGHQWVZLOOVXIIHURUOHDYH

6L]H RI WHDFKLQJ JURXSV KDV LQFUHDVHG UHVXOWLQJ LQ D PRUH XQLIRUP DSSURDFK WR WHDFKLQJ VWUDWHJLHV DQG OHVV WLPH IRU GLIIHUHQWLDWLRQRULQGLYLGXDOVXSSRUW

/HVV WLPH IRU GHDOLQJ ZLWK LQGLYLGXDO QHHGV RI VWXGHQWV DQG VLPLODUO\µJURXSZRUN¶LQYROYHVZKROHFRXUVHDQGDOODUHWUHDWHGLQ DVLPLODUZD\1RWLPHIRUDSSO\LQJWHDFKLQJVW\OHV

6SHQGLQJWRRPXFKWLPHZLWKZHDNVWXGHQWVLVWKURZLQJWKHJURXS RIIEDODQFHDQGQRWHQDEOLQJHQRXJKWLPHIRUPRUHDEOHVWXGHQWV

2YHUDOO WKH KLJK EDVHOLQH LPSURYLQJ FRXUVHV HYDOXDWHG FKDQJHV WR WHDFKLQJ PRVWSRVLWLYHO\$WWKHRWKHUH[WUHPHORZEDVHOLQHLPSURYLQJFRXUVHVZHUHWKH OHDVWRSWLPLVWLFFRQFHUQLQJWKHLPSDFWRIFKDQJHVWRWHDFKLQJ

3HUVRQDOWXWRULQJ

$OODVSHFWVRIFKDQJHVWRSHUVRQDOWXWRULQJZHUHHYDOXDWHGDVKDYLQJDJHQHUDOO\ SRVLWLYH LPSDFW RQ UHWHQWLRQ DQG DFKLHYHPHQW &KDQJHV WR WKH DVSHFWV RI WXWRULQJZKLFKZHUHUDWHGPRVWSRVLWLYHO\ZHUH

N DQ\RWKHUSHUVRQDOWXWRULQJIDFWRU N WKHUROHRIWKHSHUVRQDOWXWRU

N WDUJHWVHWWLQJSURFHVVHVLQSHUVRQDOWXWRULDOV

(38)

2WKHUSHUVRQDOWXWRULQJIDFWRUVLQWKLVFRQWH[WLQFOXGH

N VXSSRUWIRUEDVLFVNLOOVDQGVSHFLDOQHHGV

N LQWHUDFWLRQZLWKSDUHQWV

N VXSSRUWIRUSHUVRQDOVRFLDODQGHPRWLRQDOSUREOHPVRIVWXGHQWV

N LPSURYHGWXWRULDOIUDPHZRUN

7\SLFDOSRVLWLYHFRPPHQWVIURPWKHTXHVWLRQQDLUHVLQFOXGH

0XFK PRUH WLPH DOORFDWHG ZLWK PXFK LPSURYHG UHVSRQVH >«@ EHWWHU UHYLHZLQJ RI DFWLRQ SODQV GXULQJ WXWRULDOV KDV LPSURYHG WXWRULDOVDQGUHWHQWLRQ

3HUVRQDO WXWRU H[WUHPHO\ JRRG LQ KHU UROH ± YHU\ VXSSRUWLYH \HW UHDOLVWLFZLWKVWXGHQWV

7KH UHYLHZLQJ KDV EHHQ H[FHOOHQW DQG UHDOO\ LV XVHIXO IRU WDUJHW VHWWLQJ

7XWRULDO SURYLVLRQ DFWV DV VDIHW\ QHW WR UHWDLQ VWXGHQWV RQ WKH SURJUDPPH±WKH\FDQEHGLUHFWHGWRVXLWDEOHVXSSRUW

7KH SHUVRQDO WXWRU SURFHVV KDV HQDEOHG LVVXHV WR EH UHVROYHG HDUO\±HQKDQFLQJUHWHQWLRQILJXUHV

&KDQJHVWRWXWRULQJZHUHHYDOXDWHGPRVWSRVLWLYHO\E\UHVSRQGHQWVIURPORZEDVHOLQH GHFOLQLQJFRXUVHV$VZHKDYHVHHQLQUHODWLRQWRRWKHUFKDQJHIDFWRUVKLJKEDVHOLQH LPSURYLQJFRXUVHVWHQGHGWRJLYHVRPHRIWKHPRVWSRVLWLYHHYDOXDWLRQVWRRWKHUW\SHV RI FKDQJH ,Q UHODWLRQ WR SHUVRQDO WXWRULQJ KRZHYHU WKH\ MRLQHG WKH ORZ EDVHOLQH LPSURYLQJFRXUVHVLQHYDOXDWLQJFKDQJHVWRSHUVRQDOWXWRULQJLQDIDLUO\QHXWUDOZD\

7KH PDLQ OLPLWLQJ IDFWRU DVVRFLDWHG ZLWK SHUVRQDO WXWRULQJ LV ODFN RI VXIILFLHQW WLPH :KHUH SHUVRQDO WXWRULQJ DWWUDFWV QHJDWLYH DVVHVVPHQWV WKLV LV LQYDULDEO\ DVVRFLDWHG ZLWK WKLV SUREOHP RU PRUH UDUHO\ ZLWK DQ LQDSSURSULDWH IRFXV RQ SDSHUZRUN DQG DGPLQLVWUDWLRQ7KHIROORZLQJFRPPHQWVDUHIDLUO\W\SLFDO

7KHNH\LVVXHKHUHLVWKHDPRXQWRIWLPH>«@2WKHULVVXHVVXFK DVLQFUHDVHGWHDFKLQJORDGDUHVLJQLILFDQWKHUH

7KHUROH>RISHUVRQDOWXWRU@UHPDLQVYHU\LPSRUWDQW7KHFUDSWKDW JRHV ZLWK LW VXFK DV WDUJHW VHWWLQJ DFWLRQ SODQQLQJ HWF SXWV VWXGHQWV RII DQG JHWV LQ WKH ZD\ RI µSURSHU WXWRULQJ¶ /HW XV EH HIIHFWLYHIURQWOLQHWXWRUV :K\LVHYHU\WKLQJVRSDSHUERXQG"

(39)

7DEOH0HDQUDWLQJVIRUSHUVRQDOWXWRULQJ RQWKLVFRXUVHSURJUDPPHDFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

5ROHRIWKHSHUVRQDOWXWRU

$PRXQWRIWLPHVWXGHQWVKDYHIRURQHWRRQHWXWRULDOV

$PRXQWRIWLPHVWXGHQWVKDYHIRUJURXSWXWRULDOV

7DUJHWVHWWLQJSURFHVVHVLQSHUVRQDOWXWRULDOV

5HYLHZDQGDFWLRQSODQQLQJSURFHVVHV

LQSHUVRQDOWXWRULDOV

$Q\RWKHUSHUVRQDOWXWRULQJIDFWRU

.(<

&RXUVHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

2YHUDOO FKDQJHV WR FRXUVH RUJDQLVDWLRQ DQG DGPLQLVWUDWLRQ ZHUH QRW VHHQ DV KDYLQJDSDUWLFXODUO\SRVLWLYHLPSDFW7KHFKDQJHWKDWZDVHYDOXDWHGPRVWSRVLWLYHO\

ZDVWKHZD\V\RXPRQLWRUDQGUHFRUGVWXGHQWSURJUHVV7KHFKDQJHVWKDWZHUHUDWHG PRVWQHJDWLYHO\ZHUHWKHDPRXQWRIFRXUVHDGPLQLVWUDWLRQDQGWLPHDYDLODEOHIRUWHDP PHHWLQJV ,W LV LQWHUHVWLQJ WR QRWH WKDW WKH KLJK EDVHOLQH LPSURYLQJ FRXUVHV JDYH WKH PRVW SRVLWLYH DVVHVVPHQW RI FKDQJHV WR FRXUVH RUJDQLVDWLRQ DQG DGPLQLVWUDWLRQ 4XDOLWDWLYHFRPPHQWVIURPWKHLUTXHVWLRQQDLUHVLQFOXGHGWKHIROORZLQJ

&RXUVHDGPLQLVWUDWLRQKDVJUHDWO\LQFUHDVHGKRZHYHULWKDVEHHQ EHQHILFLDOWRXV

(40)

7DEOH VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI GLIIHUHQW FKDQJHV WR FRXUVH DGPLQLVWUDWLRQDQGRUJDQLVDWLRQE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUFRXUVHSURJUDPPH RUJDQLVDWLRQDQGDGPLQLVWUDWLRQDFFRUGLQJWR FRXUVH

+,+0 +' /,/0 /' 2YHUDOO

:D\V\RXPRQLWRUDQGUHFRUGVWXGHQWSURJUHVV

$PRXQWRIFRXUVHDGPLQLVWUDWLRQ

7LPHDYDLODEOHIRUWHDPPHHWLQJV

(IIHFWLYHQHVVRIWHDPPHHWLQJV

7HDPXVHRIVKDUHGOHDUQLQJPDWHULDOV

7HDPLQYROYHPHQWLQSODQQLQJDOODVSHFWVRIFRXUVH

$PRXQWRIWHDPWHDFKLQJ

$Q\RWKHUFRXUVHSURJUDPPHRUJDQLVDWLRQ

DQGDGPLQLVWUDWLRQIDFWRU

.(<

7LPHWDEOLQJ

:LWK VRPH H[FHSWLRQV FKDQJHV WR WLPHWDEOLQJ ZHUH VHHQ WR KDYH D QHJDWLYH LPSDFW RQ VWXGHQW RXWFRPHV 7KH FKDQJHV WKDW ZHUH HYDOXDWHG DV KDYLQJ WKH PRVWQHJDWLYHHIIHFWZHUH

NKRXUVZLWKLQZKLFKWKHFRXUVHVZHUHWDXJKWHDFKGD\ NDPRXQWRIJDSVLQWKHWLPHWDEOH

NDPRXQWRIWLPHWDEOHGSULYDWHVWXG\E\VWXGHQWV

NDPRXQWRIWLPHRQWKHWLPHWDEOHIRUVWXGHQWVWRµFDWFKXS¶

(41)

7DEOH EHORZ VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI DVSHFWV RI FKDQJH WR WLPHWDEOLQJE\FDWHJRU\RIFRXUVH

7DEOH0HDQUDWLQJVIRUWLPHWDEOLQJ DFFRUGLQJWRFRXUVH

+,+0 +' /,/0 /' 2YHUDOO

+RXUVZLWKLQZKLFKWKHFRXUVHSURJUDPPHLVWDXJKW

HDFKGD\

$PRXQWRIJDSVLQWLPHWDEOH

1XPEHURIZHHNVWKHFRXUVHLVWDXJKWRYHUWKH\HDU

$PRXQWRIWLPHWDEOHGSULYDWHVWXG\WLPHIRUVWXGHQWV

$PRXQWRIWLPHRQWLPHWDEOHIRUVWXGHQWVWRFDWFKXS

$Q\RWKHUWLPHWDEOLQJIDFWRU

.(<

+0 +LJKEDVHOLQHPDLQWDLQLQJ /0 /RZEDVHOLQHPDLQWDLQLQJ

'LVFXVVLRQFRPSDULVRQVEHWZHHQFDWHJRULHVRIFRXUVH

+LJKEDVHOLQHFRXUVHV

5HVSRQGHQWVIURPKLJKEDVHOLQHFRXUVHVVHHPWRPDNHEURDGO\VLPLODUDVVHVVPHQWVRI WKHVRUWRIFKDQJHVWKDWKDYHWKHPRVWSRVLWLYHDQGPRVWQHJDWLYHLPSDFWRQVWXGHQW RXWFRPHV $V RULJLQDOO\ DQWLFLSDWHG WKH DVSHFWV RI WKH VWXGHQW H[SHULHQFH ZKLFK DUH PRVWFORVHO\FRQWUROOHGDQGLQIOXHQFHGE\WHDFKHUVLQQRYDWLRQWHDPVNLOOVHPSKDVLV RQ LPSURYLQJ WHDFKLQJ DQG OHDUQLQJ DUH VHHQ DV KDYLQJ EURDGO\ SRVLWLYH LPSDFWV $VSHFWVWKDWDUHQRWFRQWUROOHGE\WHDFKHUVVWXGHQWFKDUDFWHULVWLFVPRUDOHZLWKLQWKH GHSDUWPHQWWKHGHJUHHRIGLIILFXOW\RIWKHFRXUVHDUHVHHQDVKDYLQJODUJHO\QHJDWLYH LPSDFWVRQUHWHQWLRQDQGDFKLHYHPHQW,QDGGLWLRQFKDQJHVWRIDFWRUVZKLFKKDYHWKH HIIHFW RI FRQVWUDLQLQJ RU UHGXFLQJ WKH DELOLW\ RI WHDFKHUV WR WHDFK HIIHFWLYHO\ WLPH IRU SUHSDUDWLRQDPRXQWRIFRXUVHDGPLQLVWUDWLRQQXPEHURIWHDFKLQJKRXUVDUHVHHQDV UHGXFLQJDFKLHYHPHQWDQGUHWHQWLRQUDWHV

(42)

PRUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\5HVSRQVHVIURPKLJKEDVHOLQH FRXUVHVZKLFK DUHPDLQWDLQLQJWKHLUUHWHQWLRQDQGDFKLHYHPHQWUDWHVIDOOVRPHZKHUHEHWZHHQWKHWZR 7KLVSRLQWFDQEHGHPRQVWUDWHGE\UHIHUHQFHEDFNWRWKHRYHUDOOHYDOXDWLRQVRIGLIIHUHQW FKDQJH IDFWRUV DQG WR WKH IDLUO\ FRQVLVWHQW JDS LQ UHVSRQVHV IURP LPSURYLQJ DQG GHFOLQLQJKLJKEDVHOLQHFRXUVHVVHHSDJHV±

7KHVDPHSRLQWFDQEHGHPRQVWUDWHGLQDGLIIHUHQWZD\7KHILYHPRVWSRVLWLYHDQGWKH ILYH PRVW QHJDWLYH VFRUHV IRU DQ\ DVSHFW RI FKDQJH DUH OLVWHG EHORZ DORQJ ZLWK WKH PHDQVFRUHIRUHDFKLWHP7KHKLJKEDVHOLQHLPSURYLQJFRXUVHVLHWKHPRVWSRVLWLYH JDYHWKHKLJKHVWVFRUHVIRUWKHPRVWSRVLWLYHLWHPVDQGDOVRWKHKLJKHVWLHWKHOHDVW QHJDWLYH VFRUHV IRU WKH LWHPV WKDW ZHUH VHHQ DV KDYLQJ WKH PRVW DGYHUVH HIIHFW RQ UHWHQWLRQDQGDFKLHYHPHQW$WWKHRWKHUHQGRIWKHVSHFWUXPKLJKEDVHOLQHGHFOLQLQJ FRXUVHV JDYH WKH PRVW SHVVLPLVWLF LH ORZHVW DVVHVVPHQWV KLJK EDVHOLQH FRXUVHV ZKLFKZHUHPDLQWDLQLQJWKHLUVXFFHVVUDWHVZHUHVRPHZKHUHLQEHWZHHQ

7KHUH LV DQ LQGLFDWLRQ PRUHRYHU WKDW UHVSRQGHQWV IURP KLJK EDVHOLQH LPSURYLQJ FRXUVHV DUH SDUWLFXODUO\ SRVLWLYH DERXW WKH HPSKDVLV RQ LPSURYLQJ WHDFKLQJ DQG OHDUQLQJWKHHQFRXUDJHPHQWRILQQRYDWLRQDQGVRPHDVSHFWVRIWKHZD\VLQZKLFKWKHLU WHDFKLQJWHDPVIXQFWLRQ

+LJKEDVHOLQHLPSURYLQJFRXUVHV

7KH FKDQJHV ZKLFK UHVSRQGHQWV IURP KLJK EDVHOLQH LPSURYLQJ FRXUVHV YLHZHG PRVW SRVLWLYHO\DUHOLVWHGEHORZ7KHILJXUHVLQEUDFNHWVJLYHWKHPHDQVFRUH

N ZKHWKHULQQRYDWLRQLQWHDFKLQJLVHQFRXUDJHG

N PL[RIVNLOOVLQ\RXUFRXUVHWHDP

N TXDOLW\RILQWHUQDOFRPPXQLFDWLRQVLQ\RXUWHDP

N YDULHW\RIWHDFKLQJVWUDWHJLHV\RXXVH

7KH FKDQJHV WKDW DUH VHHQ WR KDYH WKH PRVW QHJDWLYH LPSDFW RQ UHWHQWLRQ DQG DFKLHYHPHQWDUH

N DELOLW\RIVWXGHQWV

N GHJUHHRIGLIILFXOW\RIWKHFRXUVHIRUVWXGHQWV

(43)

+LJKEDVHOLQHPDLQWDLQLQJFRXUVHV

7KH FKDQJHV WKDW UHVSRQGHQWV IURP KLJK EDVHOLQH PDLQWDLQLQJ FRXUVHV LGHQWLILHG DV KDYLQJWKHPRVWEHQHILFLDOLPSDFWDUH

N KHOSIXOQHVVRIIHHGEDFNWRVWXGHQWV

N RWKHUSHUVRQDOWXWRULQJIDFWRUV

N XVH\RXPDNHRIIHHGEDFNIURPVWXGHQWV

7KHFKDQJHVWKDWWKHVDPHUHVSRQGHQWVVDLGKDGWKHPRVWQHJDWLYHLPSDFWDUH

NWLPH\RXKDYHDYDLODEOHIRUSUHSDUDWLRQ

NDPRXQWRISDLGSDUWWLPHZRUNGRQHE\VWXGHQWV

NDPRXQWRIFRXUVHDGPLQLVWUDWLRQ

NWLPHDYDLODEOHIRUWHDPPHHWLQJV

NILQDQFLDOSUREOHPVRIVWXGHQWV

+LJKEDVHOLQHGHFOLQLQJFRXUVHV

7KHFKDQJHVZLWKWKHPRVWSRVLWLYHLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQW DFFRUGLQJ WR UHVSRQGHQWVIURPKLJKEDVHOLQHGHFOLQLQJFRXUVHVDUH

N WDNHXSRIOHDUQLQJVXSSRUWE\VWXGHQWV

N XVH\RXPDNHRIIHHGEDFNIURPVWXGHQWV

N ZD\V\RXPRQLWRULQGLYLGXDOVWXGHQWSURJUHVV

N UHYLHZDQGDFWLRQSODQQLQJSURFHVVHVLQSHUVRQDOWXWRULDOV

7KH FKDQJHV ZLWK WKH PRVW QHJDWLYH LPSDFW DFFRUGLQJ WR UHVSRQGHQWV IURP KLJK EDVHOLQHGHFOLQLQJFRXUVHVDUH

N PRUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

N ILQDQFLDOSUREOHPVRIVWXGHQWV

N DPRXQWRISDUWWLPHZRUNGRQHE\VWXGHQWV

(44)

/RZEDVHOLQHFRXUVHV

7KHPRVWVXUSULVLQJIHDWXUHRIWKHORZEDVHOLQHFRXUVHVLVWKDWUHVSRQGHQWVIURPORZ EDVHOLQHLPSURYLQJFRXUVHVDUHPXFKOHVVSRVLWLYHWKDQUHVSRQGHQWVIURPFRXUVHVWKDW KDYHRQO\PDLQWDLQHGWKHLUVXFFHVVUDWHVDQGUHVSRQGHQWVIURPORZEDVHOLQHFRXUVHV ZKRVHVXFFHVVUDWHVKDYHDFWXDOO\GHWHULRUDWHG,QWKHVHFWLRQVRIWKHVXUYH\WKH\ JDYHWKHOHDVWRUWKHQH[WOHDVWRSWLPLVWLFDVVHVVPHQWRIWKHLPSDFWRIFKDQJHVLQQLQH FDWHJRULHV7KHLUµSHVVLPLVP¶ZDVH[FHHGHGRQO\E\UHVSRQGHQWVIURPKLJKEDVHOLQH GHFOLQLQJFRXUVHVZKRJDYHWKHOHDVWRUWKHQH[WWROHDVWRSWLPLVWLFHYDOXDWLRQRIWKH LPSDFWRIFDWHJRULHVRIFKDQJH7KLVSRLQWLVZRUWKHPSKDVLVLQJLQDGLIIHUHQWZD\ :KDW GLVWLQJXLVKHV LPSURYLQJ ORZ EDVHOLQH FRXUVHV LV WKHLU UHODWLYH SHVVLPLVP

FRQFHUQLQJWKHLPSDFWRIFKDQJHVWKDWDUHUHJDUGHGDVKDYLQJDEHQHILFLDOLPSDFWRQ UHWHQWLRQDQGDFKLHYHPHQW7KHLUHYDOXDWLRQRIWKHLPSDFWRIFKDQJHVWKDWDUHJHQHUDOO\ UHJDUGHGDVQHJDWLYHLVQRWPDUNHGO\GLIIHUHQWIURPWKDWRIRWKHUFDWHJRULHVRIFRXUVH

7KH VHFRQG VXUSULVLQJ IHDWXUH RI WKLV GDWD LV WKDW UHVSRQGHQWV IURP ORZ EDVHOLQH FRXUVHVZLWKGHFOLQLQJVXFFHVVUDWHVVHHPWRKDYHDYLHZRISRVLWLYHFKDQJHIDFWRUV ZKLFKGRHVQRWUHDGLO\DFFRUGZLWKGDWDFRQFHUQLQJWUHQGVDQGVXFFHVVUDWHVRQWKHLU RZQ FRXUVHV 7KXV UHVSRQGHQWV IURP ORZ EDVHOLQHGHFOLQLQJ FRXUVHV DUH QRWLFHDEO\ PRUHSRVLWLYHLQWKHLUDVVHVVPHQWRIFKDQJHIDFWRUVZLWKDJHQHUDOO\IDYRXUDEOHLPSDFW RQDFKLHYHPHQWDQGUHWHQWLRQWKDQUHVSRQGHQWVIURPORZEDVHOLQHLPSURYLQJFRXUVHV ,QGHHGWKHLUDVVHVVPHQWVTXLWHFORVHO\UHVHPEOHWKHDVVHVVPHQWVPDGHE\WKHKLJK EDVHOLQH LPSURYLQJ FRXUVHV ,Q WKH FDWHJRULHV RI FKDQJH UHVSRQGHQWV IURP ORZ EDVHOLQHGHFOLQLQJFRXUVHVJDYHWKHPRVWRUQH[WPRVWRSWLPLVWLFHYDOXDWLRQRIHLJKW FDWHJRULHV7KLVJHQHUDOREVHUYDWLRQFDQEHLOOXVWUDWHGE\UHIHUHQFHWRWKHVFRUHVJLYHQ E\GLIIHUHQWFDWHJRULHVRIORZEDVHOLQHFRXUVHVWRWKHPRVWSRVLWLYHDQGPRVWQHJDWLYH FKDQJHIDFWRUV7KHVFRUHVIRUWKHPRVWQHJDWLYHIDFWRUVDUHTXLWHVLPLODUEHWZHHQWKH GLIIHUHQWFDWHJRULHVRIORZEDVHOLQHFRXUVHV7KHVFRUHVIRUWKHPRVWSRVLWLYHIDFWRUV KRZHYHU DUH KLJKHU IURP WKH ORZ EDVHOLQH PDLQWDLQLQJ RU GHFOLQLQJ FRXUVHV DQG SDUDGR[LFDOO\ORZHUIURPWKHORZEDVHOLQHLPSURYLQJFRXUVHV

/RZEDVHOLQHLPSURYLQJFRXUVHV

7KH FKDQJHV ZKLFK UHVSRQGHQWV IURP ORZ EDVHOLQH LPSURYLQJ FRXUVHV YLHZHG PRVW SRVLWLYHO\DUHOLVWHGEHORZ

N VXSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUWHDP

N FRPSRVLWLRQRIWKHJURXSRISHUVRQDOWXWRUVZKRWXWRU\RXUVWXGHQWV

N UHFUXLWPHQWDQGVHOHFWLRQSURFHVVHV

(45)

7KHFKDQJHVWKDWDUHDVVHVVHGDVKDYLQJWKHPRVWQHJDWLYHLPSDFWDUHOLVWHGEHORZ

N DYDLODELOLW\RIWHFKQLFDOVXSSRUWDQGEDFNXS

N PRUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

N DYDLODELOLW\RIHVVHQWLDOHTXLSPHQW

/RZEDVHOLQHPDLQWHQDQFHFRXUVHV

7KH FKDQJHV VHHQ PRVW SRVLWLYHO\ E\ UHVSRQGHQWV IURP ORZ EDVHOLQH FRXUVHV ZKRVH VXFFHVVUDWHVZHUHPDLQWDLQHGDUH

N KRZVXFFHVVHVDUHGHDOWZLWKE\\RXUFRXUVHOHDGHU

N WKHUROHRIWKHSHUVRQDOWXWRU

N KRZSUREOHPVDUHGHDOWZLWKE\\RXUFRXUVHOHDGHU

N ZD\V\RXPRQLWRUDQGUHFRUGVWXGHQWSURJUHVV

N ZD\V\RXPRQLWRULQGLYLGXDOVWXGHQWSURJUHVV

7KHFKDQJHVWKDWWKHVDPHUHVSRQGHQWVHYDOXDWHGDVKDYLQJWKHPRVWQHJDWLYHLPSDFW RQUHWHQWLRQDQGDFKLHYHPHQWDUHDOOUHODWHGWRUHVRXUFHV

N DQ\RWKHUVXSSRUWIRUWHDFKHUV

N DQ\RWKHUUHVRXUFHIDFWRU

N DYDLODELOLW\RIHVVHQWLDOHTXLSPHQW

/RZEDVHOLQHGHFOLQLQJFRXUVHV

&RQWUDU\WRH[SHFWDWLRQUHVSRQGHQWVIURPFRXUVHVZLWK DORZEDVHOLQH DQGGHFOLQLQJ VXFFHVVUDWHVHYDOXDWHGDQXPEHURIVWUDWHJLHVDVKDYLQJDVWURQJ SRVLWLYH LPSDFW RQ UHWHQWLRQDQGDFKLHYHPHQW7KHFKDQJHVZKRVHLPSDFWZDVDVVHVVHGPRVWSRVLWLYHO\ DUH

N TXDOLW\RILQWHUQDOFRPPXQLFDWLRQVLQ\RXUWHDP

N ILQDQFLDOVXSSRUWDYDLODEOHWRVWXGHQWV

(46)

7KHFKDQJHVVDLGWRKDYHWKHPRVWQHJDWLYHLPSDFWDUH

N WLPHDYDLODEOHIRUSUHSDUDWLRQ

N DUUDQJHPHQWVIRUNH\VNLOOV

N DPRXQWRISDLGSDUWWLPHZRUNGRQHE\VWXGHQWV

6XPPDU\

7KH ELJJHVW VXUSULVH IURP WKH TXDQWLWDWLYH SKDVH RI WKLV UHVHDUFK ZDV WKH UHODWLYHO\

VPDOOGLIIHUHQFH LQ YLHZV H[SUHVVHG EHWZHHQ WKH GLIIHUHQW FDWHJRULHV RI FRXUVH 1RW RQO\ZHUHWKHGLIIHUHQFHVIDLUO\VPDOOEXWWKH\WHQGHGQRWEHVWDWLVWLFDOO\VLJQLILFDQWDW WKHDQGHYHQDWWKHOHYHO7KLVLPSOLHVDFHUWDLQVKDUHGSURIHVVLRQDORXWORRN DFURVV WKLV UHODWLYHO\ ODUJH GLYHUVH DQG VWUXFWXUHG VDPSOH RI H[SHULHQFHG WHDFKHUV WXWRUVDQGFRXUVHWHDPOHDGHUV

)DFWRUVWKDWDUHVHHQDVKDYLQJDSRVLWLYHHIIHFWRQVWXGHQWUHWHQWLRQDQGDFKLHYHPHQW UDWHVDUHPDLQO\WKRVHIDFWRUVWKDWWHDFKHUVFDQFRQWURORULQIOXHQFH

N VXSSRUWIRUVWXGHQWV

N TXDOLW\V\VWHPV

N SHUVRQDOWXWRULQJ

N DVVHVVPHQWDQGIHHGEDFN

N FRQWH[WRI\RXUFRXUVH

N WHDFKLQJ

)DFWRUV WKDW DUH VHHQ DV KDYLQJ DQ DGYHUVH LPSDFW RQ VWXGHQW UHWHQWLRQ DQG DFKLHYHPHQWDUHDOVRLGHQWLILHGDVWKRVHIDFWRUVZKLFKFKDQJHGPRVWVLJQLILFDQWO\RYHU WKH \HDU SHULRG 7KHVH DUH JHQHUDOO\ IDFWRUV RYHU ZKLFK WHDFKHUV KDYH OHDVW LQIOXHQFHRUFRQWURO

N UHVRXUFHV

N VWXGHQWFRKRUW

N FRXUVHRUJDQLVDWLRQRUDGPLQLVWUDWLRQ

N WLPHWDEOLQJ

(47)

7KH WKLUG ILQGLQJ ZDV WR VD\ WKH OHDVW FRXQWHULQWXLWLYH 5HVSRQGHQWV IURP ORZ EDVHOLQH GHFOLQLQJ FRXUVHV WHQGHG WR H[SUHVV YLHZV WKDW ZHUH TXLWH VLPLODU WR WKRVH H[SUHVVHG E\ WHDFKHUV IURP WKH KLJK EDVHOLQH LPSURYLQJ FRXUVHV &RQYHUVHO\ WKH YLHZVRIWHDFKHUVRQORZEDVHOLQHLPSURYLQJFRXUVHVVHHPHGWREHTXLWHSHVVLPLVWLF SDUWLFXODUO\ ZKHUH WKHLU YLHZV DUH FRPSDUHG ZLWK WKH YLHZV RI UHVSRQGHQWV IURP FRXUVHVZLWKWKHVDPHORZEDVHOLQHEXWZKLFKKDGDFWXDOO\GHFOLQHGRYHUWKH\HDU SHULRG

(48)

3$5748$/,7$7,9(5(6($5&+

,QWURGXFWLRQ

7KHLQWHUYLHZVZHUHFRQGXFWHGEHWZHHQ6HSWHPEHUDQG'HFHPEHU7KHUHZHUH LQWHUYLHZV ZLWK RI WKH RULJLQDO UHVSRQGHQWV )RU WKH PRVW SDUW LQWHUYLHZV ZHUH FRQGXFWHGLQGLYLGXDOO\RQWZRRFFDVLRQVWHDFKHUVIURPWKHVDPHSURJUDPPHFKRVHWR EH LQWHUYLHZHG WRJHWKHU )XUWKHU LQIRUPDWLRQ FRQFHUQLQJ WKH LQWHUYLHZ PHWKRGV LV FRQWDLQHGLQ$SSHQGL[

$VOLJKWO\GLIIHUHQWYHUVLRQRI3DUWRIWKLVUHVHDUFKLVEHLQJSXEOLVKHGDVDVWDQGDORQH SXEOLFDWLRQ0D\QDUGDQG0DUWLQH])RUHDVHRIUHIHUHQFHWKHVWUXFWXUHRI3DUW RIWKHUHSRUWLVVLPLODUWRWKDWLQ3DUW6RPHRYHUDOOFRQFOXVLRQVDUHGUDZQDWWKHHQG RIWKHUHSRUW

2YHUDOOILQGLQJV

7HDFKHUVLQWHUYLHZHGKDGVWURQJYLHZVDERXWZKLFKIDFWRUVPRVWDIIHFWHGUHWHQWLRQDQG DFKLHYHPHQW DOWKRXJK WKHVH ZHUH VRPHWLPHV FRQWUDGLFWRU\ DQG UHTXLUHG IXUWKHU FODULILFDWLRQ )RU H[DPSOH WHDPV ZLWK WKH KLJKHVW UHWHQWLRQ DQG DFKLHYHPHQW RIWHQ FODLPHGWKDWQHJDWLYHIDFWRUVZHUHDIIHFWLQJWKHLUFRXUVHVZKHQVWDWLVWLFDOO\UHWHQWLRQ DQGDFKLHYHPHQWKDGUHPDLQHGFRQVLVWHQWO\KLJKIRUVHYHUDO\HDUV&RQYHUVHO\PDQ\ WHDFKHUV RQ FRXUVHV ZLWK ORZ UHWHQWLRQ DQG DFKLHYHPHQW FODLPHG WKDW VWXGHQWV OHIW EHFDXVHRIHFRQRPLFRUVRFLDOFLUFXPVWDQFHVDOWKRXJKFRXUVHVZLWKKLJKUHWHQWLRQDQG DFKLHYHPHQWH[LVWHGLQWKHLUFROOHJHVDQGUHFUXLWHGDVLPLODUVWXGHQWFRKRUW

2YHUDOOWKHLQWHUYLHZVGHPRQVWUDWHGWKDWWKHRULJLQDOK\SRWKHVHVDUHEURDGO\FRUUHFW WHDFKHUVLQLPSURYLQJFRXUVHVODUJHO\DWWULEXWHLPSURYHPHQWVWRWKHLURZQDJHQF\DQG WHDFKHUVLQFRXUVHVZLWKORZUHWHQWLRQDQGDFKLHYHPHQWODUJHO\DWWULEXWHWKHLUSRVLWLRQ WRIDFWRUVRYHUZKLFKWKH\KDYHOLWWOHRUQRFRQWURO7KDWVDLGLWDOVREHFDPHFOHDUWKDW VWDIIRQFRXUVHVZLWKWKH KLJKHVW UHWHQWLRQ DQG DFKLHYHPHQW EURXJKW WR WKHLU FRXUVHV VRPH VSHFLILF DWWLWXGHV DSWLWXGHV DQG VWUDWHJLHV WKDW FRQWULEXWHG VLJQLILFDQWO\ WR WKHLU VXFFHVV7KHVHVHHPWRFRQWULEXWHWRZKDWZHPLJKWWHUPHIIHFWLYHWHDFKLQJ

&RQYHUVHO\VWDIILQFRXUVHVZLWKWKHORZHVWUHWHQWLRQDQGDFKLHYHPHQWDSSHDUWRKDYH EURXJKWWRWKHLUFRXUVHVVSHFLILFDWWLWXGHVDQGEHKDYLRXUVWKDWFRQWULEXWHGVLJQLILFDQWO\ WRDIDLOXUHWRUHWDLQVWXGHQWVRUWRLPSURYHVXFFHVVUDWHV7KLVGRHVQRWQHFHVVDULO\ PHDQ WKDW VWDII RQ ORZHU EDVHOLQH FRXUVHV EHOLHYHG WKHPVHOYHV WR EH DQ\ WKH OHVV HIIHFWLYH DV WHDFKHUV VLQFH WKH\ UDUHO\ DVVRFLDWHG UHODWLYHO\ SRRU UHWHQWLRQ DQG DFKLHYHPHQWZLWKWKHLUWHDFKLQJ

(49)

VHHQ IURP WKH VXUYH\ WKDW WHDFKHUV RQ VXFK FRXUVHV JDYH WKH OHDVW RSWLPLVWLF

DVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWFKDQJHVRQWKHVXFFHVVRIWKHLUVWXGHQWV

:KHQ FRXUVHV ZLWK ORZ UHWHQWLRQ DQG DFKLHYHPHQW LPSURYHG LW ZDV XVXDOO\ EHFDXVH WKHWHDFKHUVEHJDQWRDGRSWVWUDWHJLHVDQGSURFHVVHVWKH\EHOLHYHGWREHUHVSRQVLEOH IRU FRQWULEXWLQJ WR LPSURYHG UHWHQWLRQ DQG DFKLHYHPHQW RQ RWKHU FRXUVHV LQ WKHLU GHSDUWPHQW RU FROOHJH 7KLV VRPHWLPHV RFFXUUHG EHFDXVH RI D FKDQJH LQ WHDP OHDGHUVKLS RU FROOHJH VHQLRU PDQDJHPHQW EXW PRUH RIWHQ FDPH DERXW EHFDXVH WKH WHDPDFFHSWHGWKDWDQHZVWUDWHJ\RUSURFHVVVHHQWREHHIIHFWLYHHOVHZKHUHFRXOGEH HTXDOO\HIIHFWLYHIRUWKHP

$OWKRXJK HIIHFWLYH WHDFKHUV DGRSWHG D QXPEHU RI GLIIHUHQW VWUDWHJLHV WKDW FOHDUO\ DIIHFWHGUHWHQWLRQDQGDFKLHYHPHQWLWLVLPSRVVLEOHHLWKHUWRSURGXFHDFKHFNOLVWRUWR LVRODWHRQHSURFHVVDVPRUHHIIHFWLYHWKDQDQRWKHU1RULVWKHUHPXFKYDOXHLQWU\LQJWR GHWHUPLQH ZKLFK SRVLWLYH IDFWRUV RXWZHLJKHG ZKLFK QHJDWLYH RQHV 7KHUH LV VRPH HYLGHQFH KRZHYHU VXJJHVWLQJ WKDW HIIHFWLYH WHDFKHUV ERWK DV LQGLYLGXDOV DQG WHDP PHPEHUVDUHVWURQJO\RQWKHVLGHRIVWXGHQWV±UHJDUGOHVVRISURJUDPPHOHYHO±DQG GHGLFDWHGWRVWXGHQWVXFFHVV

7KH TXHVWLRQV LQ WKLV VHFWLRQ DVNHG WHDFKHUV WR UDWH WKH LPSDFW RI FXUULFXOXPUHODWHG LVVXHV OLNH FRXUVH FRQWHQW OHYHO DQG VFKHPH RI ZRUN RQ VWXGHQW UHWHQWLRQ DQG DFKLHYHPHQW 7KH\ ZHUH DOVR DEOH WR VD\ ZKHWKHU WKH\ WKRXJKW WKDW LVVXHV OLNH HQULFKPHQWWXWRULQJDUUDQJHPHQWVDQGNH\VNLOOVZHUHSRVLWLYHRUQHJDWLYHIDFWRUV

7HDFKHUV RQ FRXUVHV ZLWK WKH KLJKHVW UHWHQWLRQ DQG DFKLHYHPHQW UDWHV VDZ WKH FXUULFXOXPDVDQLPSRUWDQWHOHPHQWLQVWXGHQWUHWHQWLRQDQGDFKLHYHPHQWDQGEHOLHYHG LQWKHV\VWHPDWLFSUHSDUDWLRQRIVWXGHQWFHQWUHGVFKHPHVRIZRUNZKLFKDUHLQWHUHVWLQJ DQGVWLPXODWLQJEXWGRQRWLQYROYHDQ\µGXPELQJGRZQ¶7KHLUYLHZZDVWKDWWHDFKHUV VKRXOGQRWRQO\XQGHUVWDQGLQGLYLGXDOVWXGHQWPRWLYDWLRQIRUWDNLQJWKHFRXUVHEXWDOVR WKH QHHG WR PDLQWDLQ WKDW PRWLYDWLRQ WKURXJKRXW WKH SURJUDPPH &RXUVHV ZLWK KLJK UHWHQWLRQDQGDFKLHYHPHQWLQFOXGHGDVWURQJSUDFWLFDOHOHPHQWIRUERWKDFDGHPLFDQG QRQDFDGHPLF FRXUVHV DQG DGGUHVVHG OLWHUDF\QXPHUDF\ VXSSRUW QHHGV DW DQ HDUO\ VWDJH 5HVSRQVHV DERXW NH\ VNLOOV KRZHYHU ZHUH DPELYDOHQW 6RPH WHDFKHUV VDZ WKHPDVDYDOXDEOHLQLWLDWLYHRWKHUVGLGQRW*HQHUDOO\VSHDNLQJWHDPVWKRXJKWLWPRUH LPSRUWDQWWRSODQHQULFKPHQWDFWLYLWLHVWKDWFRPSOHPHQWHGWKHPDLQFRXUVHLQFOXGLQJ RXWVLGH VSHDNHUV UHVLGHQWLDOV HWF DQG WR HQVXUH WKDW DOO VWXGHQWV DUH WUDFNHG DQG VXSSRUWHGWKURXJKRXWWKHLUSURJUDPPHV

7UDGLWLRQDOO\ ZKHQ , FDPH KHUH \HDUV DJR WKH UHWHQWLRQ UDWH IRU &RPSXWLQJZDVSRRUVRRQHRIWKHWKLQJV,ZDVNHHQWRGRZDVWR DFWXDOO\ DGGUHVV WKDW E\ RIIHULQJ ,&7 >LQIRUPDWLRQ DQG FRPPXQLFDWLRQV WHFKQRORJ\@ DV DQ DOWHUQDWLYH EHFDXVH LW KDG D VWURQJXVHUIRFXVDQGZDVPRUHUHOHYDQWWRVWXGHQWQHHGV

(50)

FKDQJHV ZHUH QHFHVVDU\ 6RPH RI WKRVH ZKRVH FRXUVHV ZHUH PRVW LQ GHFOLQH FRQIHVVHG WR KDYLQJ QR VFKHPHV RI ZRUN DQGRU QR HQULFKPHQW 2Q RQH FRXUVH VWXGHQWVZHUHQRWJLYHQKDQGRXWVEHFDXVHWKH\ZHUHH[SHFWHGWREX\WKHLURZQERRNV HYHQ WKRXJK PDQ\ FKRVH QRW WR GR VR 2Q DQRWKHU FRXUVH WKH VWXGHQWV KDG EHHQ ODEHOOHGDVµQRWLQWHUHVWHGLQWKHFXUULFXOXPDQ\ZD\¶

7HDFKHUVRQFRXUVHVWKDWKDGKDGDKLJKEDVHOLQHLQUHWHQWLRQDQGDFKLHYHPHQWEXW ZHUHQRZGHFOLQLQJKDGEHJXQWRDGRSWVRPHRIWKHDWWLWXGHVGLVSOD\HGE\WHDFKHUVRQ ORZEDVHOLQHFRXUVHV7KH\DFFHSWHGWKDWWKHLUFRXUVHVZHUHEHFRPLQJµERJVWDQGDUG¶ RURXWRIWRXFK

Improving colleges : why courses and programmes improve or decline over time

report

Acknowledgements

Contents

References

7DEOHVDQGILJXUHV

7DEOH 2YHUDOODVVHVVPHQWRIWKHLPSDFWRIFKDQJHIDFWRUVE\FDWHJRU\RIFRXUVH

6XPPDU\RINH\PHVVDJHV

N 7KHUH DUH VWURQJ HOHPHQWV RI VLPLODULW\ DQG FRQWLQXLW\ LQ WKH YLHZV RI FROOHJH WHDFKHUV GUDZQ IURP D ZLGH YDULHW\ RI UHODWLYHO\ VXFFHVVIXO DQG XQVXFFHVVIXO FRXUVHVDQGSURJUDPPHV

N ,Q WKH VXUYH\ DQG FRQWUDU\ WR H[SHFWDWLRQV WKH YLHZV RI WHDFKHUV IURPORZ EDVHOLQHGHFOLQLQJFRXUVHVWHQGHGWRUHVHPEOHWKHYLHZVH[SUHVVHGE\WHDFKHUV IURPKLJKEDVHOLQHLPSURYLQJFRXUVHV

N 7KHLQWHUYLHZVDOVRSRLQWWRWKHLPSRUWDQFHRIXQZULWWHQRULPSOLHGFRQWUDFWVLQ FROOHJHV 7HDFKHUV IURP WKH PRVW VXFFHVVIXO FRXUVHV HPSKDVLVH LQ SDUWLFXODU WKHFRQWUDFWVEHWZHHQPHPEHUVRIWHDFKLQJWHDPVDQGEHWZHHQWHDFKHUVDQG WKHLUVWXGHQWV

,QWURGXFWLRQWRWKHUHVHDUFK

N :KHUH OHDUQHU SHUIRUPDQFH LPSURYHV GR WHDFKHUV WHQG WR DWWULEXWH WKH LPSURYHPHQWWRWKHLURZQDJHQF\DQGGLVFRXQWWKHLPSRUWDQFHRIIDFWRUVWKDWOLH RXWVLGHWKHLUFRQWURO"

5HVHDUFKGHVLJQ

ORFDOFRQWH[WVIURPLQQHUFLW\WRVPDOOWRZQDQGUXUDOORFDWLRQV7KHFROOHJHVWKDWWRRN SDUWLQWKHUHVHDUFKDUHOLVWHGLQ$SSHQGL[

DUUDQJHPHQWV DQG QRW WR KDYH IHOW FRQVWUDLQHG E\ WKH LQWHUYLHZ SURFHVV 7KHVH LQWHUYLHZVFRPSULVHGWKHTXDOLWDWLYHSKDVHRIWKHUHVHDUFK

6WUXFWXUHRIWKHUHSRUW

7KH DLP WKHUHIRUH RI WKLV SDUW RI WKH UHSRUW LV WR H[WHQG WKH ILHOG RI FROOHJH LPSURYHPHQW UHVHDUFK E\ DSSO\LQJ PRUH TXDQWLWDWLYH UHVHDUFK WHFKQLTXHV WR D VWUXFWXUHGVDPSOHRIFRXUVHVZLWKLQFROOHJHV

7KHUHVHDUFKGHVLJQ

5HVHDUFKK\SRWKHVHV

(DUOLHUZRUNRQWHDFKHUV¶DWWLWXGHVLQIXUWKHUHGXFDWLRQ0DUWLQH]IRXQGWKDW

,QWKHOLJKWRIWKHVHILQGLQJVDQG*UD\¶VZRUNWKHIROORZLQJK\SRWKHVHVZHUHSURSRVHG

$QDO\VLVRIWKHUHVSRQVH

:H UHFHLYHG FRPSOHWHG TXHVWLRQQDLUHV IRU GLIIHUHQW FRXUVHVSURJUDPPHV ,Q WHUPV RI OHYHO RI SURJUDPPH WKHVH VSOLW LQWR WKH IROORZLQJ SURSRUWLRQV URXQGHG WR ZKROHILJXUHV

N RI FRXUVHV ZHUH SULPDULO\ IRU ± \HDU ROGV DQG IRU VWXGHQWVDJHGSOXV

2YHUDOOILQGLQJV

6HFRQG DQG LQ WKH FRQWH[W RI PDQDJHPHQW FRQFHUQV WR LPSURYH SHUIRUPDQFH WRRPXFKVLJQLILFDQFHVKRXOGQRWEHDWWDFKHGWRDVLQJOHDQGSHUKDSVXQW\SLFDO \HDU¶VUHWHQWLRQDQGDFKLHYHPHQWUDWHVIRUDQLQGLYLGXDOFRXUVH

N ZKHUHVXFFHVVUDWHVZHUHXQVWDEOHWKLVFRXOGEHZLWKDQXSZDUG RUGRZQZDUGWUHQGRUZLWKQRGLVFHUQLEOHWUHQGHLWKHUZD\

5DWHVRIFKDQJHRYHUWKH\HDUSHULRG

N PRUDOHZLWKLQ\RXUGHSDUWPHQWRUIDFXOW\

N FRXUVHRUJDQLVDWLRQDQGDGPLQLVWUDWLRQ

5HVSRQVHVE\FDWHJRU\RIFRXUVHSURJUDPPH

7DEOH EHORZ JLYHV WKH PHDQ VFRUH IRU HDFK RI WKH W\SHV RI FKDQJH JLYHQ E\ UHVSRQGHQWV LQ GLIIHUHQW FDWHJRULHV RI FRXUVH 7KH FKDQJHV WKDW KDYH WKH PRVW SRVLWLYHLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQWDUHDVIROORZV

7LPHWDEOLQJ

)LJXUH+RZFROOHJHVLPSURYH

7KHFKDQJHVZLWKWKHPRVWQHJDWLYHLPSDFWRQUHWHQWLRQDQGDFKLHYHPHQWDUH

N VWXGHQWFRKRUW

)LQGLQJVIRUVSHFLILFFKDQJHIDFWRUV

N VFKHPHRIZRUN

N WKHLQFOXVLRQRIDSHUVRQDOOHDUQLQJMRXUQDO

+0 +LJKEDVHOLQHPDLQWDLQLQJ /0 /RZEDVHOLQHPDLQWDLQLQJ

6WXGHQWV ZHUH HQWHUHG IRU NH\ VNLOOV IRU WKH ODVW \HDUV ZKLFK KDVDGGHGH[WUDKRXUVWRWKHLUWLPHWDEOHDQGUHTXLUHGDQGH[WUD ZRUNIRUWKHPWRFRPSOHWH

&RQWH[WRIFRXUVHVSURJUDPPHV

N WKHHPSKDVLVSODFHGRQLPSURYLQJWHDFKLQJDQGOHDUQLQJ N KRZVXFFHVVHVDUHGHDOWZLWKE\WKHWHDPOHDGHU

+RZVXFFHVVHVDUHGHDOWZLWKE\FRXUVHOHDGHU

N ODFNRIVWDIIGHYHORSPHQW

7KHPRVWSRVLWLYHDVVHVVPHQWVRIFKDQJHVWRFRXUVHFRQWH[WZHUHPDGHE\WKHKLJK EDVHOLQH LPSURYLQJ DQG ORZ EDVHOLQH GHFOLQLQJ FRXUVHV 7KH VRUWV RI FRXUVH FRQWH[W FKDQJHVZLWKWKHPRVWSRVLWLYHLPSDFWZHUHLGHQWLILHGDV

FDWHJRULHVYLHZHGOHDUQLQJPRVWRSWLPLVWLFDOO\

$VVHVVPHQWDQGIHHGEDFN

N KHOSIXOQHVVRIIHHGEDFNWRVWXGHQWV N VWXGHQWV¶XQGHUVWDQGLQJRIDVVHVVPHQW N PHWKRGVRIDVVHVVPHQW

N LPSURYHPHQWVWRDVVHVVPHQWPHWKRGVDIWHUDOHFWXUHUEHFDPHDQ H[DPLQHUIRURQHRIWKHPRGXOHV

6WDIILQJ

7KHFKDQJHVWRVWDIILQJZKLFKZHUHUDWHGPRVWSRVLWLYHO\ZHUH

7KHZRUGVRIRQHUHVSRQGHQWDUHTXLWHUHSUHVHQWDWLYHRIWKHVRUWVRIYLHZVH[SUHVVHG

7DEOHVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIVWDIILQJFKDQJH E\GLIIHUHQWFDWHJRULHVRIFRXUVH

4XDOLW\V\VWHPV

7KHRQO\FKDQJHWRTXDOLW\V\VWHPVZKLFKZDVUDWHGQHJDWLYHO\ZDV

7HDFKLQJ REVHUYDWLRQV DUH FRPSOHWHG DQQXDOO\ DQG , SHUVRQDOO\ IHHOWKLVKDVDSRVLWLYHLPSDFWRQWKHWHDFKLQJPHWKRGVZLWKLQWKH FROOHJHDQGWKHGHSDUWPHQW

6HOIDVVHVVPHQWSURFHVVIRU\RXUFRXUVHSURJUDPPH

5HVRXUFHV

$OODVSHFWVRIFKDQJHVWRUHVRXUFHVZHUHHYDOXDWHGQHJDWLYHO\7KHPRVWQHJDWLYH HYDOXDWLRQFRQFHUQ

,ZRUNKRXUVDZHHNDQGVWLOOQHYHUVHHPWRNHHSDKHDG

7DEOH0HDQUDWLQJVIRUUHVRXUFHV DFFRUGLQJWRFRXUVH

6WXGHQWFRKRUW

N DPRXQWRISDUWWLPHZRUNGRQHE\VWXGHQWV N ILQDQFLDOSUREOHPVRIVWXGHQWV

7DEOH VKRZV WKH DVVHVVPHQWV RI WKH LPSDFW RI GLIIHUHQW DVSHFWV RI FKDQJH WR WKH VWXGHQWFRKRUWE\FDWHJRU\RIFRXUVH

6XSSRUWIRUVWXGHQWV

1R FKDQJHV DVVRFLDWHG ZLWK VXSSRUW IRU VWXGHQWV ZHUH HYDOXDWHG DV KDYLQJ D QHJDWLYHLPSDFW7KHFKDQJHVWKDWZHUHHYDOXDWHGPRVWSRVLWLYHO\ZHUH

7DEOHVKRZVWKHDVVHVVPHQWVRIWKHLPSDFWRIGLIIHUHQWDVSHFWVRIFKDQJHWRVXSSRUW IRUVWXGHQWVE\FDWHJRU\RIFRXUVH

6XSSRUWIRUWHDFKHUV

N PHFKDQLVPVWRDFNQRZOHGJHUHZDUGHIIHFWLYHWHDFKLQJ N VXSSRUWDYDLODEOHIRU\RXIURPZLWKLQWKHFROOHJH

6XSSRUWLVMXVWDERXWQRQH[LVWHQW

6XSSRUWDYDLODEOHWR\RXIURPZLWKLQ\RXUWHDP

7HDFKLQJ

N YDULHW\RIWHDFKLQJVWUDWHJLHV

$Q\RWKHUWHDFKLQJIDFWRU

7KHFRQWULEXWLRQWRGHYHORSLQJDFWLYLW\ERWKLQVLGHFROOHJHDQGWKH RSSRUWXQLW\WRGHYHORSP\RZQSURIHVVLRQDOOLIHRXWVLGHKDVKDGD SRVLWLYHLPSDFWRQWKHFRXUVH

5HPRYDO RI ZRUNVKRS WLPH KDV PDGH LW LQFUHDVLQJO\ GLIILFXOW WR KHOSVWXGHQWVZLWKLQGLYLGXDOSUREOHPV

2YHUDOO WKH KLJK EDVHOLQH LPSURYLQJ FRXUVHV HYDOXDWHG FKDQJHV WR WHDFKLQJ PRVWSRVLWLYHO\$WWKHRWKHUH[WUHPHORZEDVHOLQHLPSURYLQJFRXUVHVZHUHWKH OHDVWRSWLPLVWLFFRQFHUQLQJWKHLPSDFWRIFKDQJHVWRWHDFKLQJ

3HUVRQDOWXWRULQJ