Methods overview: the available data
4.5 Unit Profile Study 2011
In 2011, the Unit Profile Study was repeated (UPS 2011). The survey was again car-ried out by the EPICure Research Group, this time collaboratively within the Neonatal Economic, Staffing and Clinical Outcomes Project (NESCOP) with the following par-ticipating organisations: the Neonatal Data Analysis Unit (NDAU), based at Imperial College, London, Bliss (the charity “for babies born too soon, too small, too sick”), the British Association of Perinatal Medicine (BAPM) and the National Neonatal Audit Programme (NNAP) of the Royal College of Paediatrics and Child Health (RCPCH).
4.5.1 Preparing the questionnaire
I prepared the survey form, drafting questions and explanations based on the two pre-vious surveys (UKNSS and UPS 2006) and comments from an initial meeting between
1Personal communication with Professor K.L. Costeloe, shared with myself.
2Letter from Dr. Tucker to Professor Costeloe, provided to me by Professor Costeloe.
3Personal communication between Professor Draper and Dr Tucker, copied to myself and Professor Neil Marlow, in October 2012.
4.5 Unit Profile Study 2011
collaborators in August 20111. Additionally, I included three new questions relating to neonatal staffing coverage of junior doctors:
• Provide the total number of the following in whole time equivalents (WTE) ac-tually working or on annual leave on the medical rotas in your unit on 22nd November 2011:
1. Tier 1 (ST1-3/SHO).
2. Tier 2 (ST4-8/SpR).
3. ANNPs (in a medical role).
• Provide the total number of the following in whole time equivalents (WTE) who should be working on the medical rotas in your unit on 22nd November 2011:
1. Tier 1 (ST1-3/SHO).
2. Tier 2 (ST4-8/SpR).
3. ANNPs (in a medical role).
• On 22nd November 2011, did you have dedicated medical rotas at:
1. Tier 1 (ST1-3/SHO level)?
2. Tier 2 (ST4-8/Registrar/middle-grade level)?
Each collaborating party was given the opportunity to provide feedback, with two draft versions being circulated in October 2011.2 Further trials of the survey were performed by sending the questionnaire to several external reviewers who were asked to complete it.3 An additional question (number 16, relating to the absolute numbers of nursing staff working on the 22nd November 2011) was requested by collaborators interested in the economic analyses, and by NNAP.
1Present at the meeting were: Neil Marlow (Professor of Neonatology, IfWH, UCL), Elizabeth Draper (Professor of perinatal epidemiology, University of Leicester), Andy Cole (Director, Bliss), Neena Modi (Professor of Neonatal Medicine, Imperial College London); I was not there, but received a comprehensive summary afterwards from Professor Draper.
2Draft questionnaires were sent to the following representatives of each organisation: Stavros Petrou (Professor of Health Economics, Warwick University), Andy Cole (Director, Bliss), Neena Modi (NDAU), Elizabeth Draper (BAPM/EPICure), Neil Marlow (EPICure).
3These were: Dr. Bryan Gill (BAPM), Dr. Michael Watkinson (NNA), Dr. Maggie Redshaw (NPEU), Kim Davis (NNAP/RCPCH) in an email circulated on the 30th September, 2011.
The questions finally used were again very similar to both the previous UPS in 2006 and to the UKNSS: this time, the survey was divided into three sections, relating to activity, staffing and infrastructure (or facilities). The four pages of the questionnaire were formatted throughout to provide obvious, precise questions in a large font followed by more detailed, smaller-sized explanations of what data were being requested. Boxes were provided in which to enter responses, along with extra space for additional, free-text notes; each page was numbered and contained a separate box with the return address in it.
Two time periods were covered by the survey. First, data relating to the calendar year of 2010, or the most recent complete year for which data were available, were collected. Second, a census of current staffing was made; these data related to “the present time”, defined as Tuesday, 22nd November, 2011.
Within the ‘activity’ section were questions common to the two previous surveys.
These related to the total number of women delivering in the hospital as well as numbers of admissions, babies provided with respiratory support and total days of respiratory support provided. The ‘staffing’ section contained similar questions to the UPS 2006 and UKNSS about consultant medical staffing and nurses, as well as the additional questions noted above about junior medical staff and the absolute numbers of nursing staff on a single day. The final section, about ‘facilities’, sought information about unit categorisation (including BAPM method used), number of cots, provision of sur-gical and/or cardiac survice, as well as two new questions relating to the number of
“rooming-in” rooms available and the lowest gestational age at which babies were rou-tinely admitted into the unit (as opposed to transferred ex utero to another unit). The final version of the paper questionnaire is reproduced in appendix E.
Additionally, a customised web-based survey was designed to match the paper ver-sion. This was created using a software called “Opinio” [184] that was provided by University College London for research purposes, and required not only inputting the questions and specifying formats for the responses, but also adjusting the cascading style sheets used to display the survey. The software then created a unique survey URL for each individual invited to participate; this allowed easy tracking of who had responded (or not) and whether they’d actually finished the online survey, or merely looked at it but not pressed the “submit” button. It also facilitated using another
4.5 Unit Profile Study 2011
email address for the ‘sender’, meaning responses came direct to me if there were any issues.[184]
4.5.2 Hospitals surveyed
The list of hospital units providing neonatal care was obtained by combining the 2006 Unit Profile Study list with a publicly available list published on the Neonatal Data Analysis Unit (NDAU), based at Imperial College, London website, supplemented with information obtained from NNAP at the RCPCH. As previously, hospitals for inclusion were defined by the presence of both maternity and neonatal services on the same site (i.e. the provision of a perinatal service). One hundred and seventy hospitals were identified for inclusion.
Copies of the UPS 2011 were mailed to a named contact at each hospital during the first week of November 2011, and the email invitation to complete the online question-naire was mailed on November 15th; both of these letters were written by me but signed by Professors Marlow and Draper on behalf of the collaborating partners. Respondents were asked to return questionnaires either via the mail or using the custom web page as soon as possible after the 15th November.
Five reminder emails were sent to non-respondents over the following two months with links to the online survey. Following this, non-respondents were contacted individ-ually by telephone and email over a period of six months; the survey was declared closed on the 30th June, 2012. I was assisted in contacting hospitals that hadn’t responded by Dr. Laura McCormack, EPICure Research Manager.
Responses were received from 74 respondents via the web interface (32 completed entries, and 42 partially completed) and 134 respondents who completed the posted questionnaire. Six hospitals did not respond at all, despite multiple attempts to con-tact them; all of these were SCUs, hence would not affect analyses of provision of intensive care over time. The total number of units responding by the end of June 2012 (excluding 35 duplicate returns, three of which were duplicate paper returns from different individuals within the same hospital, the rest duplicated between the online and paper versions) was 159. This included one hospital that had not been previously identified as providing a perinatal service, Pinderfields Hospital – albeit, it replaced the service previously operating at Pontefract Hospital, and both hospitals were part
of the same NHS Trust. Twelve other units that were contacted had also merged. A flow chart of the numbers is shown in figure 4.1.
Figure 4.1: Flow chart of respondents to the Unit Profile Study (UPS) 2011.
1 7 6
1 7 0 6
1 7 1
1 6 5 6
1 5 9 6
1
U n i t s i n v i t e d t o p a r t i c i p a t e .
U n i t s w i t h o u t m a t e r n i t y s e r v i c e s .
P r e v i o u s l y u n k n o w n u n i t .
U n i t s c l o s e d d u e t o m e r g e r .
N o n - r e s p o n d e n t s .
Units available for study.
4.5.3 Data management
Postal survey returns were entered by myself and Dr. McCormack into two separate be-spoke MySQL [185] databases which I created. These data sets were then merged, with any discrepancies between them clarified by examining the original response recorded on the paper questionnaires. The online survey data were downloaded and converted into a format compatible with the processed postal data. The postal and online data sets were then appended to each other and duplicates compared, retaining the entry
4.5 Unit Profile Study 2011
with the most complete data.
4.5.4 Ensuring data accuracy
Following initial data entry and merging, assessment was made of the accuracy of data returns. This was performed using R.[186] Variables were prioritised for data checking according to their utility and perceived data integrity. Priority was given to ensur-ing that those variables common to the previous two staffensur-ing surveys were accurately recorded. Specifically, some units appeared to have been confused by questions 14 to 16: questions 14 and 15 related to the “total number of budgeted whole-time equivalent nurses who provide hands-on clinical care” – also known as the establishment figure – while question 16 asked about the (actual) numbers of staff who were working or meant to be working on the 22nd November, 2011. Confusion arose either when questions 14 and 15 were interpreted as requiring the WTE number of staff present and working on a single day, or when question 16 was completed with the number of WTEs rather than the absolute number of staff.
A further source of confusion perceived by the Neonatal Economic, Staffing and Clinical Outcomes Project (NESCOP) collaborating group was around how the num-bers of cots were reported by different units: were units reporting the total number of cots in the building, those for which they were staffed as an establishment, or only those which were actually open on the 22ndNovember 2011?1 The desire was to collect the latter of these – in order to provide an estimate of the total capacity of English neonatal services at that point in time; however, I suggested it would be still consistent with the previous surveys, as the exact detail hadn’t been specified in either of the previous two surveys.
Numeric variables were plotted as kernel density plots (for example, as shown in figures 4.2a and 4.2b, for the total number of women delivering in each hospital and the total number of babies admitted into neonatal intensive care during 2010 (or the most recent complete year for which data were available), respectively). This permitted vi-sualisation of outliers, allowing reasonable limits to be selected for further investigation and identification of units – for example, by selecting hospitals that reported greater than 7000 deliveries or 900 neonatal admissions per year. Scatter plots were used to
1NESCOP meeting, 4th March, 2013; notes available at https://nescop.org.uk/trac/wiki/
Meetings/20130304(password-protected).
Figure 4.2: Example density plots, used for cleaning and data consistency checks of data for 2010 (or the most recent 12-month period for which complete data were available), returned by English maternity hospitals in the Unit Profile Study 2011.
(a) Total number of women who delivered (“total.women).
(b) Total number of babies (“babies.number.all”) admitted.
4.5 Unit Profile Study 2011
check data consistency between variables within individual unit returns; an example with labelled outliers is shown in figure 4.3.
Categorical data were tabulated according to self-reported unit designation (i.e.
level) to identify whether any data were systematically missing or appeared erroneous (table 4.2). Histograms were used to visualise spread of data among categories.
Following unit identification, apparently erroneous or missing data were checked by comparing them with the original data returns then, where necessary, contacting the original survey respondents again. This was conducted during a three month period over the summer of 2012, commencing two weeks after the survey closed at the end of June. In total, data were updated for 48 units, with a further eleven proving impossible to check and hence were dropped from analyses by converting the responses to missing data.
Figure 4.3: Example scatter-plot of the total number of women delivering (“to-tal.women”) compared with the total number of babies (“babies.number.all”) admitted to neonatal units during 2010 (or the most recent 12-month period for which complete data were available) used in data cleaning and consistency checks for the Unit Profile Study 2011. LGI: Leeds General Infirmary; RDE: Royal Devon & Exeter Hospital.
THODSOVERVIEW:THEAVAILABLEDATA
Variable N (%) Special Care Unit Local Neonatal
Unit
Network
NICU χ2 P-value
Levels n % n % n %
2010 data supplied 159 (100) 0.9226
No 10 13.3 6 15 7 15.9
Yes 65 86.7 34 85 37 84.1
Tier 1 (SHO): presence of a 24-hr on-call rota
156 (98.1) <0.001
No 24 32 21 52.5 0 0
Yes 51 68 17 42.5 43 97.7
Missing 0 0 2 5 1 2.3
Tier 2 (SpR): presence of a 24-hr on-call rota
155 (97.5) <0.001
No 51 68 24 60 4 9.1
Yes 24 32 13 32.5 39 88.6
Missing 0 0 3 7.5 1 2.3
Year of BAPM classification 153 (96.2) 0.5489
1992 3 4 1 2.5 2 4.5
Surgical service 158 (99.4) <0.001
No 74 98.7 39 97.5 25 56.8
Yes 1 1.3 0 0 19 43.2
Missing 0 0 1 2.5 0 0
Continued on next page. . .
104
4.5UnitProfileStudy Table 4.2: (Continued.) Descriptive table of categorical variables from the UPS 2011, including
some common to those available from the UKNSS and UPS 2006.
Variable N (%) Special Care Unit Local Neonatal
Unit
Network
NICU χ2 P-value
Levels n % n % n %
Number of consultants on-call 158 (99.4) 0.0236
0-4 0 0 0 0 3 6.8
Lowest GA routinely cared for 159 (100) <0.001
<24 weeks 5 6.7 0 0 43 97.7
Of note, the three questions that I had included in the survey relating to junior medical staffing were not well completed. This appeared to be for several reasons.
First, there was confusion about whether “staff on the medical rota who provide hands on clinical care” included doctors only dedicated to neonates or also those who provided cross-cover with general paediatrics. Secondly, there was no provision for respondents who had different numbers of staff available at different times of day: for instance, some hospitals have increased numbers of doctors during the afternoon and evening periods, when there may be increased general paediatric attendances in accident and emergency (A&E) departments. Thirdly, it was unclear to respondents how they should account for “extra” doctors who may provide on-call coverage – for example, junior doctors working in community paediatrics sometimes work on-call in paediatrics at the local hospital during out of hours periods (evenings/nights and weekends).