Open-ended question feedback

The open-ended questions allow SMEs to give feedback which is not necessarily captured through the close-ended questions. Valuable insights are gathered through these questions, as they allow the SMEs to guide the focus and scope of the responses according to their respective backgrounds and individual perception of the framework. The criticisms, concerns and additional feedback received from SMEs which have not been addressed in the previous sections, are discussed and interpreted in remainder of this section.

Nomenclature use and ambiguity of methodology and framework steps

One of the SMEs raised a concern (as indicated in the ‘criticisms and concerns’ row in Table G.2) that some definitions were not used consistently, in addition to some methodological steps not being clearly explained in the background to the research. The most probable explanation for the observation was the fact that the background and origin section of the validation document consisted of six pages, thus it was a very condensed overview of Chapter 1 - Chapter 4. Although great care was taken to give a sufficient overview of the background and methodology, some of the more complex and detailed considerations were not included in this condensed overview. Additionally, as mentioned previously the framework was presented in less detail in the validation document than in the thesis document, which may have contributed to some additional ambiguity with regards to the framework function. As discussed, this concern is easily addressed by ensuring that the final stand- alone framework document contains a more detailed overview of the design considerations of the framework, in addition to not condensing the explanations part of the framework presentation steps and ensuring consistent description of the nomenclature.

Lack of consideration of additional intervention strategies

One of the SMEs raised a concern (as indicated in the ‘criticisms and concerns’ row in Table G.2) that additional intervention strategies such as prevention and mitigation should also be considered in the context of an infectious disease outbreak (for instance, social distancing as part of prevention). Firstly, it is worth noting that the validation document did not include all the design considerations used to establish the definitions of some of the terminology and the scope of considerations which are investigated. For instance, social distancing may relate to quarantined individuals, depending on how the intervention strategy is defined, hence it is technically already included in the research project considerations. Secondly, this concern highlights the importance of using the definitions and terms appropriately and consistently, with a view to ensure universal and consistent application of the steps and terms within the framework. It is not possible to retrospectively infer new definitions to the treatment interventions in the dataset, however, future work considerations may include further distinction of the intervention strategies to include a more diversified set of options other than

treatment and vaccination strategies, in addition to more complete descriptions of the intervention strategies.

Robustness and sensitivity of framework decisions

One of the SMEs raised a concern (as indicated in the ‘criticisms and concerns’ row in Table G.4) that the framework is perceived to be heavily dependent on the selection of considerations by the modeller, as opposed to making recommendations on what should be included in a given case. This concern relates to the robustness of the selection and guidance process, as approached from a systems modelling perspective. During the interview with the SME, this concern was addressed by explaining that the framework use is not limited to informing the selection of considerations, but also guiding the consideration of these factors in light of the outbreak context. The SME stated that the discussion during the validation interview clarified that the main guidance of the framework is not only limited to the sections that are heavily reliant on the modelling practitioner’s discretion. Only one SME voiced this specific concern and recommended that the sensitivities of the framework be evaluated. This does not reduce the importance of this concern, but, as this was only raised by one SME, it was deemed sufficient to discuss this as future work considerations in Chapter 7. In spite of this concern, the framework contribution as a valuable tool in guiding users is recognised by the SME in question (as indicated in the ‘additional feedback or comments’ row in Table G.4).

Merits of framework in the context of the modelling process and context uncertainty One of the SMEs (as indicated in the ‘criticisms and concerns’ row in Table G.5) questioned whether the activities which are supported by the framework are truly of such a frequent and repetitive nature as to warrant the development of the proposed framework. The SME, however, also stated that the framework has potential value for both the modelling practitioner and to serve as a reference for future research projects, which indicates recognition of the contributions of this framework to the literature. Although the modelling process is not generalisable for every single modelling instance, the framework serves as a valuable departure point to ensure that the modelling process is followed systematically according to a well-researched methodology. Not all steps of the modelling process are always incorporated in every modelling approach, but allowance is made for the considerations thereof, to ensure some generalisation of the modelling process.

Additional comments

One of the SMEs had no criticism or concerns (as indicated in the ‘criticisms and concerns’ row in Table G.6), but gave a detailed feedback response. Some of the critiques in this feedback response relate to design considerations of the research, which was not elaborated on in detail within the validation document. As mentioned by the SME (as indicated in the ‘additional feedback or

comments’ row in Table G.6), however, many of these critiques were addressed as part of the validation interview. The overall response indicated that the framework is received very positively and is perceived as a ‘valuable addition to the literature and practice.’