6.1 Data linkage
We could explore the possibility of data linkage to validate this work by enabling linkage to health outcomes. Countries with good data linkage and the established use of HTA might be Scotland (linkage to Scottish Morbidity Records), or possibly Taiwan ie this is only likely to be available in higher income countries but still with wider learning opportunities from an HTA impact assessment applicable to other countries.
6.2 iDSI’s monitoring and evaluation framework
We understand the analytical approach will be incorporated within iDSI’s monitoring and evaluation (M&E) framework, led by Itad.
Our theory of change refers only to how and why policy decisions get implemented in practice as opposed to the wider iDSI theory of change that Itad is developing. We see this work fitting into Itad’s theory of change in terms of what do better decisions and better health mean in quantitative terms, and how do we get from better decisions to better health. This speaks well to the assumptions at this level made in Itad’s framework around the implementation of decisions, and uptake by patients and clinicians.
Unlike Itad’s framework, we are not monitoring the implementation of iDSI activities. However, the methods proposed here would involve ongoing measuring along the way of an HTA, and would require size of the eligible patient population and changes in utilisation rates in addition to usual cost and effectiveness measures in order to monitor uptake and coverage. If an impact assessment was planned, this would also
require more qualitative data to inform such an evaluation. An example of different indicators of uptake and coverage is presented below which could be evidenced by (M&E) data.
Perhaps the biggest limitation to applying this framework is knowing something about the cost-effectiveness of implementation strategies.
Figure 3: Monitoring and evaluation
6.3 The case studies
It would be valuable to be involved (ideally) from the start of an HTA process. Regarding how beneficial it is to continue with the same two case studies, we would ask HITAP and iDSI as to what might be considered to be the most useful approach. The timeframe has meant the work on these studies has been confined to desk- based research. Future work would incorporate mixed methods and the theory based approaches.
6.4 Systematic review
A more thorough review of the literature would need to be undertaken. However, as there is a systematic review due to be published shortly on impact measures and models used in HTA impact assessment (13), this should greatly help to inform future work.
6.5 Package of interventions
A limitation has been raised that whilst we are considering the impact of a single technology/intervention, it would be more realistic to measure a package of HTA. Single HTA appraisals reflect the nature of economic evaluations. Alternative options might be to consider the impact of HTA in developing healthcare packages for Universal Healthcare Coverage (47) (48, 49) and/ or to consider HTA at a more macro level in LMICs (21).
Finally, we will follow up with those who provided contact emails to our impact survey at HTAsialink.
References
1. Williams L, Brown H. Factors influencing decisions of value in healthcare: a review of the literature. Health Services Management Centre: University of
Birmingham, 2014.
2. Drummond M. Twenty years of using economic evaluations for drug reimbursement decisions: what has been achieved? J Health Polit Policy Law. 2013;38(6):1081-102.
3. Sorenson C, Drummond M, Kristensen F, Busse R. How can the impact of health technology assessments be enhanced? WHO Regional Office for Europe, European Observatory on Health Systems and Policies, 2008.
4. Goodman C. HTA 101: IX Monitor Impact of HTA 2014 [3rd:[Available from: www.nlm.nih.gov/nichsr/hta101/ta101011.html.
5. O'Donnell JC, Pham SV, Pashos CL, Miller DW, Smith MD. Health
Technology Assessment: Lessons Learned from Around the World—An Overview. Value in Health. 2009;12:S1-S5.
6. M G, Kristensen F, Nielsen C, Busse R. Health Technology Assessment and Health Policy-Making in Europe. 2008.
7. Raftery J, Hanney S, Green C, Buxton M. Assessing the impact of England's National Health Service R&D Health Technology Assessment program using the "payback" approach. Int J Technol Assess Health Care. 2009;25(1):1-5.
8. Hanney SR, Grant J, Wooding S, Buxton MJ. Proposed methods for reviewing the outcomes of health research. the impact of funding by the UK's 'Arthritis Research Campaign'. 2004;2(1).
9. Buxton M, Hanney S. How Can Payback from Health Services Research Be Assessed? Journal of Health Services Research. 1996;1(1):35-43.
10. Hanney S, Buxton M, Green C, Coulson D, Raftery J. An assessment of the impact of the NHS Health Technology Assessment Programme. Health Technol Assess. 2007;11(53):iii-iv, ix-xi, 1-180.
11. Raftery J, Powell J. Health Technology Assessment in the UK. The Lancet.382(9900):1278-85.
12. The impact of the NIHR Health Technology Assessment Programme, 2003- 2013, a multi-method evaluation. waiting to publish [Available from:
www.nets.nihr.ac.uk/projects/hta/140607.
13. A systematic review of impact measures in health technology assessment. waiting to publish [Available from: www.nets.nihr.ac.uk/projects/hta/147201. 14. Oliver A, Mossialos E, Robinson R. Health technology assessment and its influence on health-care priority setting. International Journal of Technology Assessment in Health Care. 2004;20(01):1-10.
15. Tantivess S. Policy making and roles of HTA (PhD thesis).
16. Towse A, Devlin N, Hawe E, Garrison L. The Evolution of HTA in Emerging Markets Health Care Systems: Analysis to a Policy Response. 2011.
17. Drummond M, Weatherly H. Implementing the findings of health technology assessments. If the CAT got out of the bag, can the TAIL wag the dog? Int J Technol Assess Health Care. 2000;16(1):1-12.
18. Gul, aacute, csi L, szl, oacute, Boncz I, et al. Issues for countries considering introducing the : The case of Hungary. International Journal of Technology
Assessment in Health Care. 2004;20(03):337-41.
19. Kriza C, Hanass-Hancock J, Odame EA, Deghaye N, Aman R, Wahlster P, et al. A systematic review of health technology assessment tools in sub-Saharan Africa: methodological issues and implications. Health Res Policy Syst. 2014;12:66.
20. Mathew JL. KNOW ESSENTIALS: A tool for informed decisions in the absence of formal HTA systems. International Journal of Technology Assessment in Health Care. 2011;27(02):139-50.
21. Towse A. Broader approaches to HTA: strengthening health systems to improve patient care 2014 [Available from: www.slideshare.net/OHEnews/macro-hta- presentation-ispor-beijing-towse-2014.
22. Chalkidou K, Levine R, Dillon A. Helping poorer countries make locally informed health decisions. BMJ. 2010;341.
23. Walker S, Dixon S, Palmer S, Sculpher M. Getting cost-effective technologies into practice: the value of implementation. Policy Research Unit in Economic
Evaluation of Health and Social Care Interventions, 2013.
24. Fenwick E, Claxton K, Sculpher M. The value of implementation and the value of information: combined and uneven development. Med Decis Making. 2008;28(1):21-32.
25. Hoomans T, Abrams KR, Ament AJ, Evers SM, Severens JL. Modeling the value for money of changing clinical practice change: a stochastic application in diabetes care. Med Care. 2009;47(10):1053-61.
26. Kent S, Briggs A, Eckermann S, Berry C. Are value of information methods ready for prime time? An application to alternative treatment strategies for NSTEMI patients. Int J Technol Assess Health Care. 2013;29(4):435-42.
27. Mason J, Freemantle N, Nazareth I, Eccles M, Haines A, Drummond M. When is it cost-effective to change the behavior of health professionals? Jama. 2001;286(23):2988-92.
28. Chen H. A theory-driven evaluation perspective on mixed methods research. Research in the Schools. 2006;13(1):75-83.
29. De Silva MJ, Breuer E, Lee L, Asher L, Chowdhary N, Lund C, et al. Theory of Change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials. 2014;15:267.
30. Blamey A, Mackenzie M. Theories of Change and Realistic Evaluation: Peas in a Pod or Apples and Oranges? Evaluation. 2007;13(4):439-55.
31. Stern Eea. DFID Working Paper 38: Broadening the range of designs and methods for impact evaluations. London: 2012.
32. Vogel I. Review of the use of theory of change in international development. DFID, London: 2012.
33. Greenhalgh T, Wong G, Jagosh J, Greenhalgh J, Manzano A, Westhorp G, et al. Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation. BMJ Open. 2015;5(8).
34. Pawson R, Tilley N. Realistic Evaluation: Sage Publications Ltd; 1997. 35. BetterEvaluation. Realist Evaluation [Available from:
http://betterevaluation.org/approach/realist_evaluation.
36. Kingkaew P, Werayingyong P, Aye SS, Tin N, Singh A, Myint P, et al. An ex- ante economic evaluation of the Maternal and Child Health Voucher Scheme as a decision-making tool in Myanmar. Health Policy and Planning. 2015.
37. MoH M, HITAP, WHO. A feasibility study of the Community Health Initiative for Maternal and Child Health in Myanmar. Thailand: 2010.
38. MoH M, WHO, HITAP. Mid-term review of Maternal and Child Health Voucher Scheme: Yedarshey Township, Nay Pyi Taw, Republic of Union of Myanmar.
Myanmar: 2014.
39. Kingkaew P. An ex-ante economic evaluation of the Maternal and Child Health Voucher Scheme as a decision-making tool in Myanmar.
40. Health JHBSoP. The Lives Saved Tool [Available from: http://livessavedtool.org.
41. Praditsitthikorn N, Teerawattananon Y, Tantivess S, Limwattananon S, Riewpaiboon A, Chichareon S, et al. Economic evaluation of policy options for
prevention and control of cervical cancer in Thailand. Pharmacoeconomics. 2011;29(9):781-806.
42. Yothasamut J, Putchong C, Sirisamutr T, Teerawattananon Y, Tantivess S. Scaling up cervical cancer screening in the midst of human papillomavirus
vaccination advocacy in Thailand. BMC Health Serv Res. 2010;10 Suppl 1:S5. 43. Gilson L, Raphaely N. The terrain of health policy analysis in low and middle income countries: a review of published literature 1994-2007. Health Policy Plan. 2008;23(5):294-307.
44. Termrungruanglert W, Havanond P, Khemapech N, Lertmaharit S,
Pongpanich S, Khorprasert C, et al. Cost and effectiveness evaluation of prophylactic HPV vaccine in developing countries. Value Health. 2012;15(1 Suppl):S29-34.
45. Sharma M, Ortendahl J, van der Ham E, Sy S, Kim JJ. Cost-effectiveness of human papillomavirus vaccination and cervical cancer screening in Thailand. Bjog. 2012;119(2):166-76.
46. Jit M, Brisson M, Portnoy A, Hutubessy R. Cost-effectiveness of female human papillomavirus vaccination in 179 countries: a PRIME modelling study. Lancet Glob Health. 2014;2(7):e406-14.
47. Mohara A, Youngkong S, Velasco RP, Werayingyong P, Pachanee K, Prakongsai P, et al. Using health technology assessment for informing coverage decisions in Thailand. J Comp Eff Res. 2012;1(2):137-46.
48. Tangcharoensathien V, Limwattananon S, Patcharanarumol W,
Thammatacharee J. Monitoring and Evaluating Progress towards Universal Health Coverage in Thailand. PLoS Medicine. 2014;11(9):e1001726.
49. Tangcharoensathien V, Pitayarangsarit S, Patcharanarumol W, Prakongsai P, Sumalee H, Tosanguan J, et al. Promoting universal financial protection: how the Thai universal coverage scheme was designed to ensure equity. Health Research Policy and Systems. 2013;11:25-.