To explore the large-scale implementation of EHRS in PHCs in SA
3.5 Study Three: Evaluation of implemented EHRS in PHCs in SA from the perspective of end-users from the perspective of end-users
Since this thesis was aiming to explore the large-scale implementation of an EHRS in PHCs in SA, I decided to conduct a further study to evaluate the currently implemented EHRS, because assessing PHC readiness for implementation alone is therefore not sufficient to achieve this aim. Although three questionnaire-based studies were conducted in this thesis, this study focuses on the previously implemented EHRS in PHCs in SA (see Chapter One), whereas the two previous questionnaire-based studies were conducted to assess PHC readiness for an EHRS implementation project.
As revealed in Chapter One, previous implementations of an EHRS in Saudi PHCs have failed. Therefore, it was beneficial to investigate the barriers that may have contributed to the failure of these projects. If these barriers are overcome in future, this may enhance the chance of success of future projects. Other questionnaire-based studies focus on the readiness of PHCs for new EHRS implementation projects.
To achieve the study objective, this study also attempted to determine the barriers and facilitators to EHRS implementation. Moreover, to achieve a better understanding of the implemented EHRS, end-user satisfaction was examined. EHRS end-user satisfaction surveys have been identified as useful tools to measure the success or failure of any EHRS implementation (Yusof et al., 2008). Consequently, several studies have examined end-user satisfaction with EHRS since 1996 (Bani-Issa et al., 2016; Khajouei et al., 2011; Lee et al., 1996).
In addition to EHRS end-user satisfaction, all other factors identified in the literature related to the organisation, individual and technology were addressed and evaluated as influential factors. Moreover, to achieve its aim, this study examined the relationship between different factors that influence EHRS implementation, such as the relationship between perceived usefulness and training (Carr et al., 2010) (see Chapter Two). This questionnaire-based study utilised both closed-ended and open-ended items.
80 3.5.1 Site of the study
This study was carried out at PHCs that had previously implemented an EHRS. These PHCs were distributed across thirteen regions in Saudi Arabia (see Table 3.5.1) (MoH, 2012).
3.5.2 Population and sampling technique
In the current research work, the eligible population involved all end-users of EHRS within 150 PHCs in Saudi Arabia. These end-users within the Saudi MOH came from various backgrounds, age groups, departments and occupations, as well as genders. Such a population was selected since they were currently using or had been using an EHRS in their workplace. According to documents attached to an email sent by the IT department at the Saudi MoH, only 150 PHCs had fully implemented EHRS in SA (will be explored in Chapter Seven).
The sampling strategy applied in this study was similar to the one adopted in Study Two.
The number of PHCs with EHRS varied from province to another. For instance, in Riyadh there are twenty-one PHCs with an EHRS, while Aljouf has eight. In Stage three, a total of twenty-one out of 150 PHCs were randomly selected from within the five chosen clusters (see Table 3.5.1). Finally, the sample (n=483) was drawn from the selected twenty-one PHCs across the five selected regions.
81
Table 3.5.1: Regions of Saudi Arabia and number of PHCs in each region NO. Region Geographical
location
Number of PHCs with EHRS
Selected in this study
Number of selected PHCs
1 Riyadh East 21 Yes 5
2 Gassim Centre 12 Yes 4
3 Makkah West 16 Yes 4
4 Almadinah West 11 No
5 Alsharqiah East 10 No
6 Albaha South 9 Yes 4
7 Asir South 13 No
8 Najran South 11 No
9 Hail North 9 No
10 Alshamaliyah North 11 No
11 Jazan South 10 No
12 Tabuk North 9 No
13 Aljouf North 8 Yes 4
Total 13 150 21
I visited the head of the PHC department at the Saudi MoH headquarters and provided him with a copy of the ethical approval and facilitator letters. The head of the department then linked me with a small number of PHCs to facilitate the process of distribution of the questionnaires. According to the list obtained from the PHC department, the number of PHCs in each province with EHRS varied from eight to twenty-one, depending of the size of the province and number of residents.
All potential participants in these areas were sent an electronic copy of the questionnaire.
The following section highlights the data collection process and describes the questionnaire distribution mechanism.
82 3.5.3 Data collection instrument
The utilisation of a questionnaire to evaluate EHRS implementation is not unique to this study; as revealed in the literature review (see Chapter Two) the questionnaire was the most frequently used in the field of EHRS implementation (Gagnon, Ghandour el, et al., 2014; Hor et al., 2010; Khajouei et al., 2011). Moreover, Nguyen et al. (2014), in their systematic review, found that the most frequent and effective way to evaluate EHRS implementation is through “perception-based data collection” via the utilisation of questionnaire-based research. According to Hayrinen et al. (2008), there have been twenty-eight questionnaire-based research studies conducted to evaluate EHRS implementation. Although the questionnaire was frequently used in previous literature, this study employed modified data collection instruments used for the first time to evaluate EHRS implementation in Saudi PHCs.
In this section, I describe in detail the design of the questionnaire and its content. I developed the questionnaire after reviewing the relevant literature. Similar to Study Two, the questionnaire utilised in this study was available in both English and Arabic for the same reasons mentioned in Study Two. The questionnaire was designed for the people who were using EHRS in the Saudi PHCs. The aim of this study was therefore to evaluate the EHRS implemented in PHCs in SA.
This questionnaire was divided into three sections (see Appendix A). The first section contained a letter of assurance. The second section of the questionnaire sought information about the participant’s demographic details, specifically gender, age, position, years of experience in current workplace, years of experience using a PC, years of experience using an EHRS, the name of their PHC, region name, and if they were still using an EHRS. These demographic details were included in the questionnaire because they were identified in the literature as potential influencing factors of EHRS end-user satisfaction (see Chapter Two). Due to this importance, several studies were keen to include gender and age as essential questions in their data collection instruments to assess the level of EHRS end-user satisfaction and acceptance of system implementation and use (e.g. Bani-Issa et al., 2016; Biruk et al., 2014; Hamid & Cline, 2013; Pare et al., 2008;
Saleh et al., 2016).
83
The third section of the questionnaire was divided into three main themes. In this section, a total of sixty-four questions were included in order to evaluate the implemented EHRS, determine the main barriers and facilitators, and to establish end-user satisfaction and attitudes towards the use of EHRS. More than half of the items (n=37) were taken from a pre-existing questionnaire called “The Clinical Information System Implementation Evaluation Scale (CISIES)” (Gugerty et al., 2006). Additional items (n=27) were added to fully achieve the objectives of the study. The CISIES has been widely used in the field of ICT implementation in healthcare, and has already been implemented in various similar studies. CISIES can gather user attitudes on health information systems, and was frequently used in similar studies conducted in various countries. For instance, Hsieh et al. (2009) implemented this tool in a Taiwan health centre to analyse user attitudes toward the Mobile Electronic Medication Administration Record System (ME-MAR). It has also been used in the USA by (O'Meara, 2007) to evaluate user attitudes toward a digital documentation system in Ambulatory Surgery Centres (ASCs).
Participants were asked to rate each question using a scale that ranged from Strongly Disagree to Strongly Agree, as follows: Strongly Disagree (1); Disagree (2); Neutral (3);
Agree (4); and Strongly Agree (5). Items that were rated as strongly disagree or disagree represented dissatisfaction, and those that were rated as strongly agree and agree represented satisfaction with EHRS implementation (Gugerty et al., 2006). The items in CISIES focused on end-user perception of EHRS implementation and use, which assisted in providing sufficient understanding of the quality of the implemented EHRS. The decision to select CISIES was also made because it addressed all factors determined to influence EHRS implementation. The items in this section were categorised under four main themes. It began with seventeen questions querying the perceived usefulness of the EHRS, then further items (n=29) were included to evaluate other factors such as training and support. Sixteen items were included to determine the barriers to EHRS implementation. Finally, two open-ended questions were added for further clarification and recommendations. Although, the majority of the items in this questionnaire were taken from pre-used questionnaire, findings from semi-structure interviews were taken into consideration. For instance, some items in the barriers scale were added based on emerged barriers from the semi-structure interviews.
84
Theme One: Overall attitudes towards and satisfaction with EHRS implementation
In order to establish end-user satisfaction and attitudes towards the EHRS implementation, forty-six items were included, which were largely (n=37) quoted from CISIES. This theme allocated several factors determined to be influential factors in EHRS implementation (see Chapter Two), some of which are directly related to EHRS end-user satisfaction. For instance, perceived usefulness and perceived ease of use are considered key indicators in measuring end-user acceptance of EHRS implementation (Devine et al., 2010). In addition, effectiveness and performance of EHRS implementation can influence healthcare practitioner satisfaction with the system (Khalifa & Alswailem, 2015). The other nine items were added based on a suggestion made by previous literature to fully examine the perceived usefulness of the implemented EHRS (Altuwaijri, 2011; Bardhan
& Thouin, 2013; Chaudhry et al., 2006; Gajanayake et al., 2013; Mekhjian et al., 2002;
Menachemi & Collum, 2011).
Theme Two: The barriers to EHRS implementation
To assess the barriers to EHRS implementation, the “Barriers to EHR implementation”
scale was used (see Appendix B). This consisted of sixteen questions developed according to the most frequent barriers identified in the literature review (see Chapter Two). The aim of this scale was therefore to determine the main barriers to the implementation of EHRS and to find which barriers had the most impact. It was anticipated that the responses offered may provide some insight and clarification regarding the barriers that could result in unsuccessful EHRS implementation in PHCs in Saudi Arabia. Participants were asked to rate each item using a scale that had three response options: not a barrier, minor barrier, and major barrier.
Theme Three: General clarification and end-user recommendations regarding possible ways to improve EHRS implementation in PHCs
The aim of adding open-ended questions was to gain further information about the implemented EHRS, as well as to determine the barriers to and facilitators of EHRS implementation in PHCs. Question Twelve required the participants to mention the features of EHRS they most liked. Question Thirteen required the participants to provide
85
recommendations to improve EHRS implementation in PHCs. Part of the evaluation of the implemented EHRS was to gain further information via these open-ended questions.
These two open-ended questions were an important addition to the questionnaire as they sought to obtain rich information and suggestions on how to help the EHRS project team make better decisions. Moreover, the purpose of asking open-ended questions was to encourage the participants to write full, meaningful answers in relation to their knowledge and opinions on improving EHRS implementation.
3.5.4 Questionnaire development
The majority of items, included in this study, were from existing questionnaires. These questions were included because of their comprehensiveness and relevance to the factors identified in the literature review. To ensure all factors identified in the literature review were examined in this study, a mind-map was developed to match the items covered by the questionnaire. Additional items were added to the questionnaire to achieve all objectives of this study. For instance, sixteen items were used to determine the main barriers to EHRS implementation in PHCs in SA. The new items were also imported to the mind-map to test their appropriateness in helping achieve the objectives of the study.
Bryman (2012) suggested considering such hypotheses when designing new questionnaires. Thus, I took into account hypotheses from the literature review. For instance, the literature identified that participant demography (e.g., gender, age, and occupation) may influence their satisfaction with EHRS implementation (see Chapter Two). Therefore, I included all potential questions related to the participant demographic information to examine their impact on participant satisfaction.
Since this study was conducted at the same time as Study Two, similar development procedures were conducted to develop the questionnaire of this study (see Section 3.4.3 0). Although the same expert panels reviewed the questionnaire in the first phase of development, different feedback was received from them. The population of this study may not be able to provide all required information about the implemented EHRS, such as leadership and management, cost of the project, or project team selection and communication factors. Therefore, I asked the panel to review the questionnaire to make
86
sure all questions were answerable by the target population and to avoid irrelevant responses which may affect the questionnaire’s reliability.
Similar to Study Two, the panel of experts recommended to design the questionnaire to be available in both English and Arabic. Moreover, the barriers scale was ranked from major facilitator to major barrier, as follows: major barrier, minor barrier, not a barrier or facilitator, minor facilitator, major facilitator. However, the panel were of the opinion that a two-sided Likert scale may cause confusion in the participants. Therefore, the scale range was reduced from five to three (see previous section). In addition, since this data collection instrument was designed to evaluate the previously implemented EHRS, it was more beneficial to determine the barriers that led to failure of the project. Finally, a few spelling mistakes and translation errors were detected and corrected.
Phase Two: Following review and feedback on the questionnaire by the expert panels, it was sent to a small number (n=13) of the implemented EHRS end-users in Saudi PHCs as a pilot study to ensure that the questionnaire was clear, understandable and reliable.
Every participant involved in the pre-test was selected by me in coordination with the representatives of the PHCs by the utilisation of simple random technique (see Section 3.5.5). Volunteers from different occupations, genders, EHRS usage and nationalities were selected for the pilot study. A Cronbach’s alpha test revealed that questionnaire used in this study is statically reliable. Responses obtained from the pilot study were added to the final findings.
3.5.5 Data collection process
Similar to Study Two, an online self-administrated questionnaire was used to collect data in the current study. Since I had no direct access to the PHCs, selection was made with the coordination of the PHC department at the Saudi MoH. Therefore, once the PHCs and target population had been identified, I distributed the questionnaire for this study in the same way as Study Two. Similar to Study Two, the Department of PHCs at the MoH provided me with the contact details of representatives from the selected PHCs to contact them directly and facilitate distribution of the questionnaire. All PHC representatives were invited to a WhatsApp group to fill in the questionnaire and then forward the questionnaire link to other staff members at the selected PHCs. All participants received
87
an electronic copy of the questionnaire via a PHC representative (see 3.4.4 for more detail).
3.5.6 Data analysis
The data analysis performed for this study was similar to that of studies One and Two.
Initially, a reliability test was performed using a Cronbach’s alpha test (see Chapter Six for more details), then descriptive analysis to display all obtained data. While demographic tables included frequency and percentages, both main scales include median, percentages, total agreement and rank. As proved earlier, individual satisfaction to EHRS implementation may be influenced by characteristics such as position, gender, experience with computers, etc. (see Section 3.4.2). Therefore, a Mann-Whitney U test was used to examine whether there are any differences between males and females in terms of level of satisfaction, as hypothesised in the literature. A Kruskal Wallis test was also used to examine whether there are any differences between nurses and other PHC staff in regard to their level of satisfaction, as hypothesised in the literature. The Kruskal Wallis test was used with other demographic variables that included more than two groups, such as age and experience using a PC, to determine whether these demographic differences influenced the level of satisfaction, as suggested in previous literature (see Chapter Two).
It was obvious from the literature that certain factors that influence EHRS implementation are associated. Whereas some of the study scales represent these factors, a Spearman’s rho was used to determine any correlation between the main scales used in the study.
According to Carr et al. (2010); and Escobar-Rodriguez and Bartual-Sopena (2013), factors such as perceived usefulness and perceived ease of use are associated with training and support (see Figure 3.5.1). Therefore, a Spearman’s rho correlation coefficient test was used (Field, 2013) to examine whether there was a relationship between these factors, as revealed in previous literature (see Chapter Two). The procedures adopted to analyse the responses to open-ended questions were exactly the same as those adopted in Study Two.
88 Training
User age
Support Prior experience
with IT
Perceived usefulness
Perceived ease of use
Attitudes toward using
the EHRS
Figure 3.5.1: The relationship between factors influencing EHRS
89