• No results found

Effects of Morphine Analgesia on Diagnostic Accuracy in Emergency Department Patients with Abdominal Pain: A Prospective, Randomized Trial

N/A
N/A
Protected

Academic year: 2021

Share "Effects of Morphine Analgesia on Diagnostic Accuracy in Emergency Department Patients with Abdominal Pain: A Prospective, Randomized Trial"

Copied!
14
0
0

Loading.... (view fulltext now)

Full text

(1)

Effects of Morphine Analgesia on Diagnostic

Accuracy in Emergency Department Patients with

Abdominal Pain: A Prospective, Randomized Trial

Stephen H Thomas,

MD, MPH

, William Silen,

MD

,

FACS

, Farah Cheema,

MD

, Andrew Reisner,

MD

,

Sohail Aman,

MD

, Joshua N Goldstein,

MD

,

PhD

, Alan M Kumar,

MD

, Thomas O Stair,

MD

BACKGROUND: Because of concerns about masking important physical findings, there is controversy surround-ing whether it is safe to provide analgesia to patients with undifferentiated abdominal pain. The purpose of this study was to address the effects of analgesia on the physical examination and diagnostic accuracy for patients with abdominal pain.

STUDY DESIGN: The study was a prospective, double-blind clinical trial in which adult Emergency Department (ED) patients with undifferentiated abdominal pain were randomized to receive placebo (con-trol group, n⫽36) or morphine sulphate (MS group, n⫽38). Diagnostic and physical exam-ination assessments were recorded before and after a 60-minute period during which study medication was titrated. Diagnostic accuracy and physical examination changes were compared between groups using univariate statistical analyses.

RESULTS: There were no differences between control and MS groups with respect to changes in physical or diagnostic accuracy. The overall likelihood of change in severity of tenderness was similar in MS (37.7%) as compared with control (35.3%) patients (risk ratio [RR] 1.07, 95% confidence interval [CI] 0.64–1.78). MS patients were no more likely than controls to have a change in pain location (34.0% versus 41.2%, RR 0.82, 95% CI 0.50–1.36). Diagnostic accuracy did not differ between MS and control groups (64.2% versus 66.7%, RR 0.96, 95% CI 0.73–1.27). There were no differences between groups with respect to likelihood of any change occurring in the diagnostic list (37.7% versus 31.4%, RR 1.20, 95% CI 0.71–2.05). Correlation with clinical course and final diagnosis revealed no instance of masking of physical examination findings.

CONCLUSIONS: Results of this study support a practice of early provision of analgesia to patients with undiffer-entiated abdominal pain. ( J Am Coll Surg 2003;196:18-31. © 2003 by the American College of Surgeons)

Emergency medicine specialists and those in other fields have been criticized for providing suboptimal analgesia to patients with a variety of diagnoses. The explanations for underanalgesia are many, but in at least one group of patients—those with undifferentiated abdominal pain (UAP)—the withholding of analgesia has theoretic

foundation in a desire to avoid masking of diagnostic findings.

The proscription against pain relief during evaluation of patients with abdominal pain dates as far back as 1921, when Sir Zachary Cope’s1textbook on abdominal

pain cautioned against catastrophic diagnostic delays at-tributable to opiate-induced clouding of physical find-ings. In fact, concerns about administration of analgesia are sufficiently deep that they persist despite counterar-guments reported in clinical investigations, reviews, and in the last few editions of Cope’sEarly Diagnosis of the Acute Abdomen.2-13

Clinicians who withhold analgesia from patients with abdominal pain are neither inherently callous nor nec-essarily ignorant of the pertinent literature. The argu-No competing interests declared.

Received June 25, 2002; Accepted July 25, 2002.

From the Division of Emergency Medicine (Thomas, Reisner, Goldstein, Kumar, Stair) and the Department of Surgery (Thomas, Silen), the Depart-ment of Emergency Services, Massachusetts General Hospital (Thomas, Cheema, Reisner, Aman), and the Department of Emergency Medicine, Brigham and Women’s Hospital (Stair), Boston MA.

Correspondence address: Stephen H Thomas, MD, MPH, Department of Emergency Services, Massachusetts General Hospital, Clinics Building Room 115, Boston MA 02114-2696.

18

© 2003 by the American College of Surgeons ISSN 1072-7515/03/$21.00

(2)

ment about masking diagnostic findings is not wholly implausible, and on closer scrutiny the literature sup-porting early analgesia is not without substantial meth-odologic shortcomings. For example, expert statistical opinion holds that more than 1,000 patients would be necessary to power a study searching for outcomes dif-ferences, but the aggregate number of patients in all relevant published studies falls below this number.14

Given the importance of the issue and the imperfec-tions of the extant literature, we concluded that there was sufficient question about this issue to embark on an additional trial. Specifically, the objectives of the current study were to evaluate, in patients with UAP, the safety of morphine sulfate (MS) analgesia as assessed by alter-ations in diagnostic accuracy and physical examination.

METHODS

Design and setting

This was a randomized, double-blind clinical trial with two arms, MS and control. The study was conducted at Brigham and Women’s Hospital, a tertiary care Harvard-affiliated institution with an annual emergency depart-ment (ED) census of 55,000. Patients with abdominal complaints are seen primarily by resident physicians in emergency medicine and general surgery, or obstetrics and gynecology; all patients are also seen and staffed by emergency medicine-boarded attending physicians. The study was approved by the Institutional Review Board at the study center.

Time frame and participants

Study patients were enrolled in nonconsecutive fashion between August 1999 and October 2000. Other than the manner of defining the severity of abdominal pain (see below), the admission of patients into the study proceeded in a manner similar to that used in the previ-ous trials of the effects of analgesia on evaluation of patients with abdominal pain.5,6,15Because

understand-ing study group composition is critical to both

interpre-tation and generalizability of our results, details (and justification) of inclusion criteria are delineated below. To be eligible for the study, patients had to meet the following criteria:

1) Age at least 18 years. Pediatric abdominal pain encom-passes a different disease spectrum than that of adults. Additionally, the (relatively frequent) use of sedatives to enable physical examination of pediatric patients could confound assessment of association between opioid ad-ministration and study endpoints.

2) Nonpregnant status. The ED evaluation of pregnant pa-tients is characterized by a unique diagnostic approach. Additionally, pregnancy has effects on the selection of an-algesic medications. So women of childbearing age could not enter the study until a negative (urine) pregnancy test was documented.

3) Abdominal pain of less than 72 hours’ duration. This time limit, implemented in an attempt to reduce enrollment of patients with chronic pain and patients whose pain was unlikely to require operation, was based on standard texts and has been used in previous UAP analgesia studies.5,13 4) Severe pain. There is little reason to assess the effects of

opioids on physical examination in patients whose pain is not severe. But determination of what constitutes severe pain is subjective. For admission to the study, patients had to indicate that their abdominal pain was of sufficient severity to warrant (absent safety considerations) analge-sics stronger than ibuprofen or acetaminophen. The alter-native approach of allowing the physician to determine whether patients’ pain was sufficiently severe to warrant opioids was rejected for two reasons. First, investigators have demonstrated clinically significant discordance be-tween patient and physician perception of the need for opioid analgesia in the setting of undifferentiated abdom-inal pain in the ED.16Second, because patients were asked to agree to participate in a study in which they might receive a “risky” opioid injection, we believed they should be empowered to make the determination as to whether their pain was sufficiently severe that there was potential benefit from MS administration.

5) Patients were ineligible if early opioid analgesia was clearly indicated. Patients for whom treating physicians believed that early opioid analgesia was clearly indicated given cur-rent standards of care (eg, high suspicion of renal or biliary colic or pancreatitis), and for whom randomization to the control study arm would therefore be unethical, were in-eligible for the study.

6) Patients were ineligible if hypotensive. Patients with sys-tolic blood pressure less than 90 mmHg were excluded. Abbreviations and Acronyms

CI ⫽confidence interval ED ⫽Emergency Department MS ⫽morphine sulfate RR ⫽risk ratio

UAP⫽undifferentiated abdominal pain VAS⫽visual analog scale

(3)

7) Patients were ineligible if emergency diagnostic and ther-apeutic interventions precluded study enrollment. Pa-tients with abdominal pain requiring expedited diagnostic or therapeutic intervention for an imminently life-threatening condition (eg, leaking abdominal aortic aneu-rysm) were excluded because these patients would be po-tentially endangered by study entry.

8) Patients were ineligible if allergic to MS. Interventions

Patient flow in the study is depicted in Figure 1. Just after the initial physician evaluation of patients with a chief complaint of abdominal pain or related diagnosis, independent (ie, not involved in patient care) study per-sonnel discussed patient eligibility with treating physi-cians. If patients seemed eligible, study personnel intro-duced themselves to the potential study subjects, explained the goals and objectives of the project, and made final determination of eligibility. If patients were

eligible and if they provided informed consent to partic-ipate, they were asked to indicate the level of their pain on a 10-cm visual analog scale (VAS) that had the text markings “No pain” and “Severe pain” at the bottom and top of a nongraduated vertical line. Use of the VAS cor-responded to generally accepted methodology.17-19Next,

the examining physician was asked to complete the ini-tial (prestudy medication) evaluation form. This evalu-ation form consisted of the first eight questions depicted in Figure 2. The study did not define any of the elements on the study form, but rather allowed examining physi-cians to use their own operational definitions for find-ings such as “peritonitis” and “presence of surgical abdo-men.” If more than one physician examined the patient, study design allowed for multiple examiners completing forms on a single patient.

After initial study form data were entered, treating physicians were to provide study medication, titrating

(4)

Figure 2. Examining physician evaluation. LLQ, left lower quadrant; LUQ, left upper quadrant; RLQ, right lower quadrant; RUQ, right upper quadrant.

(5)

the (double-blinded) study medication (control or MS) as if it were MS 1 mg/mL. For purposes of this article, the termstudy medicationrefers to either the MS or the placebo (saline) preparation, depending on the group to which study patients were randomized. The actual med-ication was obtained from the pharmacy, which dis-pensed an unmarked vial for each patient; the identity of the vial’s contents (MS 1 mg/mL or saline placebo) was determined by a randomization table kept at the re-search pharmacy. During the 60-minute study period, treating physicians were not to provide other (ie, non-study medication) opioid analgesics, but they were free to provide other medications, such as antiemetics and H2-blockers. Doses and frequency of administration of

study medication were left to the treating physicians, but there was a study period (60-minute) maximum of 15 mL (equivalent to 15 mg of MS). After the 60-minute study period was over, repeat patient VAS and poststudy medication (60-minute) physician assessments were ob-tained. Patients were then “off study” with further anal-gesia provided at the discretion of the treating physician. The poststudy medication assessment for examining physicians consisted of the same data as in the initial form in addition to two other questions (queries 8 and 9 on Figure 2) addressing examiner perceptions about study medication identity and the effects of study med-ication administration on physical findings.

After the poststudy medication evaluation form and VAS assessments were completed, study personnel fol-lowed the course of study subjects during the remainder of their ED stays, recording information pertinent to di-agnostic testing and ED diagnosis and disposition. Hos-pital records of patients who were admitted were re-viewed. Telephone followup was obtained for all study patients. The goal was to contact patients once, 7 days after the study-related ED visit, but the protocol allowed for re-peated attempts to contact patients until followup was ob-tained. Followup queries assessed whether patients had on-going pain and whether they had visited another physician. After followup information was obtained, patient records (ie, including reports, but no radiographic im-ages) were presented to one of the study’s coauthors with recognized expertise in evaluation of the acute abdomen. The review of records occurred in a blinded fashion, such that the reviewer had access only to clinical infor-mation and not to other study-related data (eg, study medication, study medication guess by treating physi-cians, VAS scores). Although the a priori assumption was

that there would be insufficient patient accrual to adju-dicate these endpoints for statistical significance, we be-lieved such review would be beneficial for descriptive purposes and also a means to optimize patient safety (ie, to screen for any untoward outcomes potentially caused by MS administration).

Data and analysis

The study collected four general categories of data: base-line information (eg, pain duration, age, gender); initial and poststudy medication pain levels as assessed by VAS; initial and poststudy medication study form data (Fig. 2); and final diagnostic and followup information ob-tained by medical record review and patient telephone contact after the study.

For comparisons between MS and control groups, continuous data were analyzed with thettest (eg, age) and with nonparametric rank-sum testing (eg, ordinal VAS data). For delineation of whether VAS changes were clinically significant, a cutoff of score change of at least 13 mm was used.20Pearson chi-square and Fisher’s exact

(when any cell counts were 5 or less) tests were used for categoric data.

Study form data were also analyzed with risk ratios (RRs), reported with 95% confidence intervals (CIs), to assess intergroup comparative likelihood of endpoints (eg, disappearance of abdominal tenderness). But, this type of analysis, which does not incorporate the clinical picture surrounding any examination changes, fails to tell the complete story. For example, disappearance of McBurney’s point tenderness after study medication ad-ministration has quite different implications for a pa-tient with gastroenteritis as compared with one with appendicitis. So, reporting for study form data includes descriptive information.

Evaluation of diagnostic endpoints was a major goal of this study. One such endpoint was the proportion of cases where the poststudy (60-minute) diagnostic list of up to two entities included the ultimate “correct” con-dition. Definition of the correct diagnosis proceeded by an a priori plan, implemented before unblinding, and depended on the patient’s disposition and followup in-formation. For a patient discharged from the ED, who did not return for another evaluation (at any location) before the followup call, the ED discharge diagnosis was assigned as the correct diagnosis. For admitted patients, the hospital discharge diagnosis was defined as the cor-rect diagnosis. Regardless of initial disposition, if

(6)

pa-tients were seen by another physician before the fol-lowup period, the discharge diagnosis from the second presentation was considered correct. The relation be-tween diagnostic confidence (ie, high, moderate, or low) and type of study medication was also evaluated, as was the frequency with which an initial incorrect diagnosis was changed to a correct one after medication.

Based on previous work, we believed that study en-rollment would be insufficient to power assessment of morbidity or mortality.14But the study design included

study group-blinded review of patient management by an expert surgical reviewer. This reviewer, who had ac-cess to all medical records information and radiology reports (but not images), was asked to provide subjective or objective impressions about patient management.

All analyses were performed with Intercooled STATA version 7.0 (StataCorp, College Station, TX). Signifi-cance was defined at the alpha level of 0.05. Because analgesia has been argued to both obscure and enhance the abdominal examination, two-sided hypothesis test-ing was used.

RESULTS

Enrollment, baseline characteristics, and pain relief To enroll the 74 study patients, study personnel ap-proached a total of 899 patients who had triage-documented chief complaint of abdominal pain. Of the 825 patients who were ineligible, the reasons for exclu-sion (in decreasing order of frequency) were: 1) pain not

severe enough (n⫽245, 29.7% of 825); duration of pain too long (n⫽222, 26.9%); 3) treating physician deemed patient ineligible because of high suspicion of abdominal process warranting early analgesia (n⫽204, 24.7%); 4) patient refusal to participate in the study (n⫽68, 8.2%); 5) pregnancy (n⫽62, 7.5%); 6) no time for study enrollment because of need for emergency or expedited workup (n⫽14, 1.7%); and 7) morphine allergy (n⫽10, 1.2%).

As shown in Table 1, baseline characteristics of pa-tients in the control (n⫽36) and MS (n⫽38) groups were similar. The total number of examiners for the control group was 51; there were 53 examiners for the MS group. On initial examination, the proportion of control and MS groups with signs of peritoneal irritation (27.5% versus 41.5%, p⫽0.132) were similar. Al-though initial pain scores were similar in the MS and control groups, MS patients were more likely to achieve marked pain relief (see Table 1).

ED course, disposition, and ultimate diagnosis Of 74 study patients, 18 (24.3%) were admitted with planned operative intervention, 39 (52.7%) were admit-ted for observation, and the remaining 17 (23.0%) were discharged home. There was no association between study group and disposition (p⫽0.50) or ultimate need for operative intervention (p⫽0.51). Ultimate di-agnoses, as determined by medical records and followup information, are shown in Table 2.

Table 1. Patient Characteristics for Control and MS Groups

Characteristics Control group (nⴝ36) MS group (nⴝ38) p Value control vs. MS Demographics

Age, median, y (IQR) 39 (30–48) 39 (30–56) 0.709

Proportion female (%) 58.3% 52.6% 0.622

Pain and VAS data

Mean pain hours⫾SD 30.4⫾25.7 25.2⫾19.4 0.472

Initial VAS⫾SD 7.4⫾1.8 7.4⫾1.7 0.904

Median VAS change (IQR) 1.0 (0.0–3.5) 3.0 (1.0–5.0) 0.008

Proportion with VAS drop⬎12 mm* 41.7% 73.7% 0.005

Number of examiners per patient† 0.873

One 25 26

Two 7 9

Three 4 3

*Proportion with drop in VAS, from initial to 60 minute assessment, of at least 13 mm. †

Number of examiners completing pre- and poststudy medication evaluation sheets. IQR, interquartile range; MS, morphine; VAS, visual analog scale.

(7)

Followup information and repeat physician evaluation

Followup was obtained in all patients, but the study protocol-dictated 7-day followup period was often ex-tended because initial efforts to contact patients or fam-ily member failed. The likelihood of delayed followup (beyond 7 days) was similar (p⫽0.99) between MS (81.6%) and control (80.6%) patients. Control patient followup required as many as 69 days and one patient in the MS group did not have followup for 124 days.

There were four patients (three MS, one control, p by Fisher’s exact test⫽0.62) who had a repeat physician visit (in all cases, with their primary care provider) for abdominal pain within a week of the study-related ED visit. These four patients, all of whom had telephone followup at 7 days, had no pain at the time of followup and there were no untoward developments.

Surgical evaluation of decision making

Blinded review of surgical management revealed four cases (three MS, one control, p by Fisher’s exact⫽0.62) where retrospective assessment raised questions about decision-making. In none of these cases did the reviewer believe any clear errors were made and in no cases were the questions about management believed by the re-viewer to be potentially related to study medication.

A patient in whom study medication did not affect VAS (of 8.0), and who the examiner correctly assessed as having received placebo, was suspected by the treating

physicians of having borderline evidence for cholecysti-tis. She was discharged home with a plan for elective cholecystectomy, performed without sequelae 3 days lat-er; the surgical reviewer believed admission might have been warranted.

A patient who had received MS had an examination remarkable for right upper quadrant abdominal pain and a Murphy’s sign, both of which diminished (but did not disappear) after administration of study medication. The patient’s VAS decrease (3.0 to 2.8) was not notable, and examiners failed to correctly guess the study medi-cation identity. She was admitted to the hospital and underwent removal of a gallbladder pathologic only for presence of sludge; the operation was judged by the sur-gical reviewer to have possibly been premature.

Another MS patient, whose Murphy’s sign disap-peared after administration of study medication, was admitted for cholecystectomy but had a normal gall-bladder on pathologic examination. This patient, dis-cussed in more detail in the “Distribution and severity of abdominal tenderness” results section, had marked pain relief (VAS from 10.0 to 2.0), and the examiner correctly guessed the identity of the study medication.

A final case involved a patient who had a clinically significant VAS decrease (from 8.7 to 7.0) after admin-istration of study medication. The examiner incorrectly guessed that the patient received placebo. Both the pre-and poststudy medication diagnostic lists consisted of diverticulitis first and appendicitis. Subsequent CT scanning (ordered from the ED) demonstrated a diver-ticular abscess, and the patient underwent an operation. The surgical reviewer, who had access to the radiology report (but not the images), believed the operation might have been premature.

In all four of the preceding patients, diagnostic con-fidence was not changed after medication. Additionally, in none of the four patients did examiners indicate that medication had any effect on the physical examination. Changes in findings of peritonitis

This endpoint assessed whether examiner determination of the presence of peritonitis changed between initial and followup queries. For six examiners (assessing five patients), responses changed. The examiners’ ultimate diagnosis list (of up to two) contained the correct diag-nosis in five of six patients.

Disappearance of peritoneal signs was noted in one control and two MS patients. In the control patient, an Table 2. Ultimate Diagnosis for Control and MS Groups

Diagnosis Control group (nⴝ36) MS group (nⴝ38) n % n % Abdominal pain 10 27.8 11 29.0

Abdominal wall mass/hernia 1 2.8 3 7.9

Appendicitis 3 8.3 4 10.5 Biliary colic/cholecystitis 3 8.3 2 5.3 Bowel obstruction 3 8.3 1 2.6 Diverticulitis 1 2.8 8 21.1 Food poisoning 1 2.8 – Gastroenteritis 5 13.9 2 5.3

Genitourinary tract conditions 4 11.1 4 10.5

Inflammatory bowel disease 1 2.8 –

Ischemic colitis 1 2.8 –

Pancreatitis 3 8.3 2 5.3

Peptic ulcer disease – 1 2.6

(8)

initial differential diagnosis list of bowel obstruction and gastroenteritis was narrowed after study medication ad-ministration, to the single correct diagnosis of bowel obstruction (managed nonoperatively). One MS patient had a negative ultrasonographic study and an ultimate diagnosis of “abdominal pain—resolved.” A second MS patient, who had a CT scan diagnosis of “possible ab-dominal wall hernia,” was managed nonoperatively. This case, in which the examiner was a PGY3 emergency medicine resident who correctly guessed the nature of the study drug, was the only instance (out of 104 exam-inations in 74 patients) where the examiner indicated that the physical examination was impaired after study medication, though diagnostic confidence actually in-creased from moderate to high.

An MS patient who was assessed by two examiners believed that signs of peritonitis developed in the inter-val between the initial and followup examinations; both examiners correctly diagnosed appendicitis. Findings of peritonitis also developed in a control patient, who was correctly diagnosed as having an abdominal mass (pylo-ric mass found on operation).

Distribution and severity of abdominal tenderness The overall likelihood of change in severity of tenderness was similar in MS (37.7%) and control (35.3%) patients (relative risk [RR] 1.07, 95% confidence interval [CI] 0.64–1.78). Similarly, MS patients were no more likely than control patients to have a change in location of pain (34.0% versus 41.2%, RR 0.82, 95% CI 0.50–1.36). MS patients were no more likely than controls to have the combined endpoint of any change in location, sever-ity of tenderness, or both (47.2% versus 52.9%, RR 0.89, 95% CI 0.61–1.31). Patients who manifested the combined endpoint of changed location severity or both were not significantly more likely than those lacking this endpoint to have a correct diagnosis included in the poststudy medication differential list (69.2% versus 61.5%, RR 1.13, 95% CI 0.85–1.49).

Increasing localization of the abdominal examination was present in 24.5% of MS patients and 23.5% of control patients (RR 1.04, 95% CI 0.53–2.07). Patients who had localization of the examination were not signif-icantly more likely than those without this finding to have a correct diagnosis included in the poststudy med-ication differential (76.0% versus 62.0%, RR 1.23, 95% CI 0.93–1.63).

In no case was there a finding that the area of

pro-nounced tenderness moved away from, and did not in-clude, the initial area of tenderness. Such migration did occur, for areas of lesser (mild or moderate) tenderness, in seven patients, and the combined endpoint of expan-sion, migration of tenderness, or both were noted in 19 patients (18.3% of 104). This type of change in the abdominal examination was significantly more likely to occur in the control group compared with the MS group (27.5% versus 9.4%, RR 2.91, 95% CI 1.13–7.49). Patients who had expansion, migration of examination findings, or both were not significantly more likely than those without this finding to have a correct diagnosis included in the poststudy medication differential (68.4% versus 64.7%, RR 1.06, 95% CI .75–1.49).

Overall, the likelihood of complete disappearance of abdominal tenderness in MS patients as compared with control patients did not reach statistical significance (15.1% versus 3.9%, RR 3.85, 95% CI 0.86–17.37). The difference between groups was nonsignificant (Fish-er’s exact p, 0.09), probably because of low numbers of patients with this finding.

In only two patients, both in the MS group, was there complete disappearance of pronounced tenderness after study medication. In the first patient (mentioned in the “Surgical evaluation of decision making” section), bili-ary tract pathology was suspected with a high degree of confidence both before and after the administration of study medication, even though the patient’s abdominal tenderness and Murphy’s sign both disappeared after study medication administration. The examiner, who was unsure of the nature of the study medication, indi-cated that the study medication had no effect on the quality of the examination. (Another examiner of the same patient also noted disappearance of moderate ten-derness and of Murphy’s sign, and also maintained the incorrect diagnosis of biliary tract disease after study medication administration.) The patient was found on laparotomy to have a normal gallbladder (no stones) and was discharged from the hospital with a final diagnosis of abdominal pain.

In the other patient in whom pronounced tenderness disappeared, a patient for whom there was low confi-dence in the pre- and poststudy medication diagnoses of “abdominal pain—unclear etiology” was subsequently found on CT to have mild diverticulitis (uneventfully managed with outpatient antibiotics). There were eight cases in which patients had only moderate tenderness on initial examination and had no tenderness at all on the

(9)

poststudy medication examination. One of these pa-tients (the patient with the normal laparotomy for sus-pected biliary tract disease) has already been mentioned. Another two patients, also in the MS group, were char-acterized by pre- and poststudy medication differential diagnoses, including appendicitis and biliary tract dis-ease; both patients had complete resolution of pain (after MS study medication) and required only observation before final diagnoses of abdominal pain were assigned. The other five patients (two control, three MS) in whom moderate tenderness disappeared were characterized by correct pre- and poststudy medication diagnoses. Changes in diagnostic signs

Overall, there was no association between MS adminis-tration and likelihood of the combined endpoint of any change in either the presence or severity of the assessed signs (15.1% versus 11.8%, RR 1.28, 95% CI 0.48– 3.44). But, these endpoints are better considered in de-scriptive fashion to emphasize the importance of clinical correlation with any changes.

The psoas or obturator signs were assessed in less than half of the patients (48.1% of 104). Findings for these signs only changed (decreased in degree) in one control patient, who had a final diagnosis of abdominal pain of unknown etiology.

There were two patients, with three examiners, in whom Murphy’s sign disappeared. One of these patients (in whom two examiners noted disappearance of Mur-phy’s sign and a laparotomy for suspected biliary tract disease was negative) is discussed in the Results section. In the other patient (also an MS group patient) in whom Murphy’s sign disappeared, it was initially coextant with McBurney’s point tenderness; the latter persisted and Murphy’s sign did not, and the patient’s operative man-agement for suspected appendicitis was confirmed as correct. So, in all three instances in which Murphy’s sign disappeared, the poststudy medication absence of the sign was appropriate as delineated by absence of biliary tract disease on operative intervention. For the five pa-tients (two MS, three control) who ultimately were di-agnosed with biliary tract disease (biliary colic or chole-cystitis), Murphy’s sign was present in all of the initial examinations and changed after study medication in only one case—an MS patient in whom the sign was persistent but decreased in intensity after study medica-tion administramedica-tion.

McBurney’s point tenderness disappeared in two

pa-tients, both in the control group. Both of these patients had multiple examiners, some of whom did not believe McBurney’s point tenderness was present initially. Both examiners who did perceive McBurney’s point tender-ness initially, and noted its subsequent loss correctly suspected their patients to have nonoperative causes of abdominal pain (gastroenteritis in one patient, in-flammatory bowel disease in the other). For the seven patients (four MS, three control) who had appendicitis, McBurney’s point tenderness was present in all and changed in only one patient—an MS patient in whom the severity of McBurney’s point tenderness increased after study medication administration.

Diagnostic accuracy and confidence

Overall diagnostic accuracy did not differ between MS and control groups (64.2% versus 66.7%, RR 0.96, 95% CI 0.73–1.27). There were no differences between MS and control groups with respect to likelihood of any change occurring in the differential diagnosis list (37.7% versus 31.4%, RR 1.20, 95% CI 0.71–2.05). Further analysis focused on whether those diagnostic changes that did occur represented moves toward or away from the correct diagnosis.

In none of the 57 patients for whom examiners’ initial diagnostic lists included the correct diagnosis was that diagnosis excluded from the poststudy medication list-ing. So analysis for change in accuracy of the diagnostic list focused on moves toward the correct diagnosis. These moves could occur by two pathways, depending on whether the initial diagnostic list included the correct diagnosis.

If the initial diagnostic list did not include the correct diagnosis, diagnostic improvement was said to occur if the correct diagnosis was present on the poststudy med-ication differential. If the initial list did include the cor-rect diagnosis, diagnostic improvement was defined as occurring if the poststudy medication list elevated the correct diagnosis (ie, from second to first position) or excluded the incorrect diagnosis that had appeared on the initial list.

When the correct diagnosis was not included on the initial list, the poststudy medication diagnostic list in-cluded the correct diagnosis more often in MS as com-pared with control patients but the difference was not significant (15.1% versus 3.9%, RR 3.85, 95% CI 0.86–17.3). The second means of moving toward the correct diagnosis, narrowing of an initial list of two

(10)

di-agnoses to a poststudy medication listing of a single cor-rect diagnosis, occurred with similar frequency in MS and control patients (7.6% versus 5.9%, RR 1.28, 95% CI 0.30–5.45). For the combined endpoint of either mechanism of moving toward the correct diagnosis, the higher rate of occurrence in MS as compared with con-trol patients was not statistically significant (22.6% ver-sus 9.8%, RR 2.31, 95% CI 0.88–6.09).

MS and control study groups had similar likelihoods of increase in diagnostic confidence (13.2% versus 9.8%, RR 1.34, 95% CI 0.46–3.97), decrease in diag-nostic confidence (5.7% versus 3.9%, RR 1.44, 95% CI 0.25–8.28), or any change in diagnostic confidence (81.1% versus 86.3%, RR 0.94, 95% CI 0.79–1.11). There was no association between study group status and final diagnostic confidence (p⫽0.285) and no as-sociation between final diagnostic confidence and diag-nostic accuracy (p⫽0.339).

Unblinding

The a priori plan for unblinding analysis was to assess for association between study group status and examiner response to the study form query as to identity of the study medication. This analysis revealed that unblinding did not occur (p⫽0.472). The proportion of examin-ers indicating “cannot tell” as the answer to the study drug query was similar (p⫽0.649) between control (35.3%) and MS (39.6%) patients. Because there were no cases of study-medication side effects, in no case was unblinding performed during the study period. Effects of study medication on quality of physical examination

In most patients (88.7% MS, 96.1% control), examina-tion quality was rated as unchanged. In only one case (MS patient), discussed in the Results section “Changes in findings of peritonitis,” was examination quality im-paired as judged by the examiner. Overall, there was no association between study group and perceived impact of study medication on examination quality (p⫽0.320).

DISCUSSION

In 1921, Sir Zachary Cope1wrote in his classic treatise

on the evaluation of acute abominal pain, “If morphine be given, it is possible for a patient to die happy in the belief that he is on the road to recovery, and in some cases the medical attendant may for a time be induced to share

the elusive hope.” Over the years, the Cope text’s pro-scription against early pain relief was a major contribu-tor to a practice—withholding of analgesia because of fears of masking examination findings—which devel-oped substantial clinical inertia despite a lack of support-ing literature. No studies have demonstrated conceal-ment of abdominal findings with reasonably dosed opioids (Cope’s warning was based on contemporary practices of intramuscular administration of up to 30 mg morphine).21Indeed, recent editions of Cope’s text13

la-ment the practice of analgesia denial and policy state-ments from physician (eg, American College of Emer-gency Physicians) and regulatory (eg, AEmer-gency for Healthcare Research and Quality) panels, and evidenced-based medicine reviews have concluded that early analgesia is effective and does not cloud diagnostic findings.7,22,23

On the surface, then, it would appear that the ques-tion of safety has been asked and answered. So why another study on use of analgesics in abdominal pain? There were two fundamental and intertwined reasons for the execution of the current study. First, the voices of skeptics have not been silenced, and there is strong in-dication that publication of studies directed at this ques-tion has not effected a widespread reversal in clinical practice. Recent assessments of attitudes and clinical practices of general surgeons in both the UK and the US have revealed that 38% to 67% believe that analgesia risks masking of diagnostic findings.2,3 Unfortunately,

this reluctance of consultants to provide analgesia has understandable effects on practices of emergency physi-cians. For example, a 1998 survey reported that 80% of US emergency medicine respondents withheld opioid analgesia pending surgical assessment.24Finally, a 2002

editorial review by Burdick25lamented the persistence of

the traditional proscription against analgesia.

In isolation, nonevidence-based argument against an-algesia, however persistent, would be an insufficient ba-sis on which to conduct a placebo-controlled trial. But, attempts at a decisive response to skeptics soon reveal the second impetus for execution of the current study: pro-vision of early analgesia is based on a body of literature that even sympathetic reviewers do not construe as de-finitive.12Although a detailed scrutiny of the literature is

beyond the scope of this discussion, brief characteriza-tion of the relevant studies is necessary to frame the current investigation’s contribution to the debate.

(11)

opi-oids in abdominal pain was reported in 1986.15 The

surgical research group found no evidence for obscura-tion of physical examinaobscura-tion findings, but the study was characterized by limitations such as equal rates of anal-gesia in the control and experimental groups. The next trial, published in 1992, reported improved analgesia rates in the experimental (morphine) group and also found no evidence suggesting danger associated with early pain relief.6 Study limitations included potential

for interobserver variability in assessment of physical ex-amination findings because the pre- and postmedication examinations were performed by different physicians.

In 1996, a group of US emergency physicians ad-dressed some of the shortcomings of the earlier litera-ture. Pace and Burke5evaluated morphine analgesia with

a design in which pre- and postmedication physical ex-aminations were performed by the same physician. Re-lief of pain did not affect the physical examination find-ings, though this endpoint was assessed with only a single query about the presence of peritonitis. The au-thors found that pain relief did not impair diagnostic accuracy, though the study design allowed a large num-ber of differential diagnoses to be listed.

The important distinction between physical examina-tion and clinical outcomes endpoints was emphasized in a paper published by LoVecchio and colleagues.4This

paper, reporting the effects of administering morphine to patients with acute abdominal pain, could arguably be used to support either side of the analgesia debate because some changes in physical examination were found in the patients receiving analgesia. The study’s conclusions about analgesic safety were limited by the fact that patients had all been evaluated by surgeons who formulated a definitive plan before study enrollment. Another recently reported study also found mixed re-sults with respect to analgesia and its effects on physical findings.26

Perhaps the optimal solution to the ongoing abdom-inal analgesia debate is for the decision-making surgeon to promptly evaluate ED patients with abdominal pain and then prescribe analgesia as needed. Unfortunately, as is noted in the Cope text and by other experts, there are often considerable—and in many cases unavoidable— delays between patient arrival and evaluation by the re-sponsible surgeon.10,13,15 The reasons for this vary

de-pending on whether the hospital setting is academic (eg, multiple layers of housestaff ) or community (eg,

rela-tively fewer surgeons). In either case, prompt evaluation by a surgeon is not invariably possible.

The likely persistence of obstacles to rapid attending surgical evaluation, in combination with the fact that patients with abdominal pain comprise a major propor-tion of ED visits, means that (safe) improvement of an-algesia in these patients would represent a substantial increment in overall quality of emergency care.27So we

believed there was room for another study covering both diagnostic and physical examination endpoints. The strength of the current study’s assessment of differential diagnosis is that it demanded a narrow differential (max-imum of two entities) and also assessed diagnostic con-fidence. Study strengths related to physical examination endpoints were the inclusion of both general (eg, pres-ence of surgical abdomen) and specific examination endpoints commonly used in clinical practice. Perhaps most important, the a priori plan for reporting of results called for correlation of examination changes with the clinical picture; this allowed for adjudication of whether any examination changes that did occur led the examin-ers toward (or away from) the correct diagnosis.

In fact, by every analysis—diagnostic, examination, and clinical correlation—the study’s results are consis-tent with a conclusion that there is no untoward effect of early administration of MS. The study’s assessment of diagnostic listing after MS revealed no difference in pro-portion of patients with correct diagnosis, and there were no instances where MS administration led examin-ers away from the correct diagnosis. Similarly, changes in physical examination were no more likely with MS than they were with placebo; importantly, such changes that did occur did not mask physical findings. Overall, the assessments could be summarized by a statement that there was no evidence supporting the contention that MS administration was deleterious in any way.

Though the study’s results are compelling on initial review, they should be considered in light of method-ologic limitations. Some limitations, perhaps the most important, are related to subject enrollment scheme (ie, convenience sample) and accrual. The study enrollment scheme was convenience sampling, rather than a consec-utive series. The reason for this was that study personnel, who had to be physically present in the ED during pa-tient enrollment periods, were not available around the clock. As such, study patients were preferentially en-rolled during weekdays and afternoon and evening shifts rather than weekends and overnight shifts.

(12)

Nonconsec-utive sampling, which appears to have been used in all other abdominal pain analgesia studies except for that of Attard and coworkers,6risks selection bias. This is said to

occur when enrolled subjects differ, in some means rel-evant to the study’s endpoints, from other eligible sub-jects who were not enrolled. Although incomplete sam-pling is a fatal flaw of some study types (eg, response bias in a survey study), there is little reason to suspect that results of this randomized, placebo-controlled double-blind study would be subject to major bias resulting from nonconsecutive sampling. But, the chance of un-anticipated bias cannot be ignored.

Another aspect of the study which, at least on initial perusal, appears as a weakness is the enrollment number of only 74 patients. In fact, this study’s enrollment is the largest of any of the studies on this subject conducted in the US and the number of “cases” was further increased to 104 by assessment of multiple examiners for individ-ual patients.4,5,25 In any event, based on the work of

others, it was known during study planning stages that accrual necessary for outcomes endpoint assessment would be beyond our resources.14So we concentrated on

assessment of the diagnostic and physical examination endpoints that were amenable to assessment in a study of the projected size of this one. Practically speaking, the study’s inclusion criteria assured that enrolled patients would be representative of the population—those with severe abdominal pain of unclear etiology—potentially eligible to receive opioid analgesia. So, the study’s enroll-ment number reflects, among other things, a higher acu-ity than the average patient with abdominal pain; 77.0% of study patients were admitted and 23.4% underwent operation. In fact, as outlined in the Results section, study personnel approached more than 10 patients for every 1 who was enrolled.

Though the study’s analysis was facilitated by replace-ment of outcomes endpoints with assessreplace-ment of accu-racy in the differential diagnosis, this approach was not without problems. The first was that the differential di-agnosis listing was somewhat artificial. Only two diag-noses were allowed on the list; emergency physicians’ working diagnosis lists, especially in the initial 60 min-utes of a patient evaluation, can reasonably contain more entities. But, use of a short differential diagnostic list increased the power of the study to detect important shifts in the differential toward or away from the correct entity, and this constraint was applied equally to the two study groups.

A second issue related to the diagnostic list was that both the pre- and poststudy medication diagnostic lists were generated in the absence of relevant laboratory (eg, liver function tests) and radiographic (eg, ultrasonogra-phy, CT) data. In some cases basic laboratory (ie, elec-trolytes, complete blood count) and plain radiography results might have been available to examiners complet-ing the poststudy medication data forms, but in no case was advanced imaging (ie, ultrasonography or CT) available. As is the case for the constraint on number of diagnostic entities, this artificial increment in difficulty of listing correct diagnosis applied equally to both study groups. Additionally, although specialized imaging (eg, appendix-protocol CT) is frequently and increasingly used to make a definitive diagnosis for patients with abdominal pain, the study design forcing examiners to indicate poststudy medication differential diagnosis be-fore these imaging results emphasized the role of the history and physical examination in determining differ-ential diagnosis. In fact, though the study was not de-signed to address this issue, it should be emphasized that the history remains critical to the process of evaluating abdominal pain. There are doubtless some cases where the history would be of sufficient suggestive power to (consciously or subconsciously) affect interpretation of physical examination findings. This study was not able to assess this possibility.

A third issue pertinent to the use of the diagnostic list was the method of definition of the correct diagnosis. Unavoidably, there was subjectivity in cases where there was no clear diagnostic result from imaging or operative intervention. In some cases where the ED diagnosis dif-fered from the final (eg, medical record or followup phy-sician) diagnosis, it could have been the former that was correct. But, this type of problem was most likely to occur in patients with nonoperative pathology—a group for whom abdominal pain analgesia is arguably of less concern.

An additional methodologic issue is that study data were assessed in something of an artificial fashion to maximize chances of identifying diagnostic differences in the control and MS groups. This approach increased the use of intergroup comparisons of the study’s end-points of interest, but limited the appropriateness of generalizability of other results. For example, based on the points just discussed, the study’s results with respect to diagnostic accuracy (eg, frequency of correct

(13)

diagno-sis) are not intended to reflect the true ability of exam-ining physicians to determine patient diagnoses.

Like the diagnostic endpoints, the study’s physical examination endpoints are imperfect, but there is little reason to suspect bias related to study group. For exam-ple, after the manner of similar previous studies, the definitions of “surgical abdomen” and “peritoneal signs” were left to the examiner’s usual practice.5The inevitable

interexaminer inconsistency resulting from this method-ology would not be expected to impact validity of using a single examiner’s pair of examinations to assess study medication effect.

Study design allowed for administration of study medication, and pain relief, over a period of 1 hour. This methodology is similar to that of most previous studies, which have allowed for reassessment of parameters as early as 15 to 30 minutes and as long as 60 minutes after study medication administration.4,5,25,28 Though this

study had a strength of allowing titration of study med-ication, there was a related weakness that the titration and analgesia period was limited to an hour. In fact, investigators were initially interesting in pursuing a study design that allowed for titration of study medica-tion over 2 or more hours, a time frame more accurately reflective of the interval often necessary for establishing an ED diagnosis (ie, for supplementing the history and initial physical examination with laboratory data and radiography results). But, in reviewing the proposal for the study, the institutional Human Studies Committee believed strongly that a study period should not exceed an hour because of the potential dangers of opioid ad-ministration and also the need for patients to be able to be “off protocol” relatively quickly so they could receive analgesia as clinically indicated.

The 60-minute time frame for the study’s assessments (before and after study medication titration) had pre-dictable effects on the evaluation of study endpoints. Specifically, examiners completing the evaluation form after the titration period benefited from availability of a history, but had few laboratory data and no specialized imaging results. In effect, the examiners had to indicate their differential diagnosis and examination findings without benefit of information which, in real life, would have a clear effect on differential diagnosis and a poten-tial (subconscious) effect on interpretation of the phys-ical examination. The end result of the study’s magnifi-cation of the physical examination component to the clinical evaluation was enhanced internal validity,

achieved at some cost to external validity. Through the imposition of the 60-minute time constraint (and lack of critical test results such as CT), the study gained in its ability to focus on the true effects of analgesia on the endpoints assessed. Clinicians were asked to formulate differential diagnoses without benefit of information that plays an important role in evaluation of patients with abdominal pain of unclear etiology, and only two physical examinations were assessed. External validity was compromised, because diagnostic imaging and re-peat examinations frequently play vital roles in the eval-uation of the patient with abdominal pain of unclear etiology. Because the study’s goals did not include assess-ment of overall accuracy of diagnoses or physical find-ings, the external validity issue is not a major weakness. Furthermore, the impact of the study’s results is en-hanced by the fact that no adverse effects of analgesia were found even given the constraints imposed by the study design. In other words, in actual practice, where full laboratory and radiography results would be avail-able, there would be much less likelihood of opioid-associated adverse outcomes from misdiagnosis than any such risks identified in this study setting.

Though analysis failed to identify statistically signifi-cant unblinding, there remains potential for residual ef-fects from unblinding in individual circumstances. Given the widespread familiarity with the clinical effects of opioids, one could reasonably suspect that such un-blinding could occur with frequency. But, our findings to the contrary are consistent with results from previous studies of similar design.5,15Overall, any alterations in

diagnostic workup resulting from residual unblinding (or similarly, from lowering of the threshold for ordering diagnostic tests for all study patients) should have had minimal impact on most study parameters because these were assessed well before results of diagnostic tests were available.

In conclusion, this study builds on those in the liter-ature that uniformly suggest the appropriateness of an-algesia provision to patients with undifferentiated ab-dominal pain. This practice should be implemented as a cooperative clinical approach between emergency spe-cialists and surgeons. Neither this study nor others in the literature support a practice of opioid analgesia followed by early ED discharge. What is supported by these stud-ies, and emphasized by the frequency with which ab-dominal pain patients are encountered, is that emer-gency physicians and surgeons appear to have little

(14)

reason to ignore the fundamental clinical imperative to relieve suffering.

Author contributions

Study conception and design: Thomas, Silen, Reisner Acquisition of data: Cheema, Aman, Goldstein, Stair Analysis of data: Kumar

Critical revision: Silen Statistical expertise: Thomas Obtaining funding: Reisner Supervision: Thomas, Stair

Acknowledgment: The authors wish to acknowledge the co-operative efforts of the Brigham and Women’s Hospital de-partments of Emergency Medicine and Surgery, and also ex-press appreciation for the assistance of the entire Emergency Department staff at the Brigham and Women’s Hospital.

This paper is dedicated to the memory of Dr C David Godley, surgical colleague at Massachusetts General Hospital, who enthusiastically collaborated on previous work leading to this study and who personified principles of humane and compassionate patient care.

REFERENCES

1. Cope Z. The early diagnosis of the acute abdomen. New York: Oxford University Press; 1921.

2. Tait IS, Ionescu MV, Cushieri A. Do patients with acute abdom-inal pain wait unduly long for analgesia? J R Coll Surg Edinb 1999;44:181–184.

3. Graber MA, Ely JW, Clarke S, et al. Informed consent and general surgeons’ attitudes toward the use of pain medication in the acute abdomen. Am J Emerg Med 1999;17:113– 116.

4. LoVecchio F, Oster N, Sturmann K, et al. The use of analgesics in patients with acute abdominal pain. J Emerg Med 1997;15: 775–779.

5. Pace S, Burke TF. Intravenous morphine for early pain relief in patients with acute abdominal pain. Acad Emerg Med 1996;3: 1086–1092.

6. Attard AR, Corlett MJ, Kidner NJ, et al. Safety of early pain relief for acute abdominal pain. Br Med J 1992;305:554– 556.

7. Mackway-Jones K, Harrison M. Towards evidence based emer-gency medicine: analgesia and assessment of abdominal pain. J Accid Emerg Med 2000;17:128–129.

8. Vermeulen B, Morabia A, Unger PF, et al. Acute appendicitis: influence of early pain relief on the accuracy of clinical and US

findings in the decision to operate—a randomized trial. Radiol-ogy 1999;210:639–643.

9. Brewster GS, Herbert ME, Hoffman JR. Medical myth: analge-sia should not be given to patients with an acute abdomen be-cause it obscures the diagnosis. West J Med 2000;172:209–210. 10. Paris PM. No pain, no pain. Am J Emerg Med 1989;7:660–661. 11. Editorial. Analgesia and the acute abdomen. Br Med J 1979;2:

1093.

12. Yealy DM, O’Toole KS. Challenging dogma—analgesia in acute abdominal pain. Acad Emerg Med 1996;3:1081–1082. 13. Silen W. Cope’s early diagnosis of the acute abdomen. 20th ed.

New York: Oxford University Press; 2000.

14. Lee JS, Stiell IG, Wells GA, et al. Adverse outcomes and opioid analgesic administration in acute abdominal pain. Acad Emerg Med 2000;7:980–987.

15. Zoltie N, Cust MP. Analgesia in the acute abdomen. Ann R Coll Surg 1986;68:209–210.

16. Thomas SH, Borczuk P, Shackelford J, et al. Patient and physi-cian agreement on abdominal pain severity and need for opioid analgesia. Am J Emerg Med 1999;17:586–590.

17. Todd KH. Clinical versus statistical significance in the assess-ment of pain relief. Ann Emerg Med 1996;27:439–441. 18. Menegazzi J. Measuring pain at baseline and over time. Ann

Emerg Med 1996;27:433–435.

19. Ho K, Spence J, Murphy MF. Review of pain-measurement scales. Ann Emerg Med 1996;27:427–432.

20. Todd KH. Patient-oriented outcome measures: the promise of definition. Ann Emerg Med 2001;38:672–674.

21. Hughes TJ. Opiates in acute abdominal pain (letter). Br Med J 1979;2:1145.

22. American College of Emergency Physicians. Clinical policy: critical issues for the initial evaluation and management of pa-tients presenting with a chief complaint of nontraumatic acute abdominal pain. Ann Emerg Med 2000;36:406–415. 23. Evidence Report/Technology Assessment No. 43. Making

health care safer: a critical analysis of patient safety practices. AHRQ Publication 01-E058; 396–400. Rockville, MD: Agency for Healthcare Research and Quality; 2001.

24. Wolfe JM, Lien D, Lenkoski C, Smithline HA. Analgesic ad-ministration to patients with acute abdomens: a survey of Emergency Physicians [abstract]. Acad Emerg Med 1998;5: 528.

25. Burdick WP. Abdominal pain. In: Burdick WP, Cone DC, Cydulka RK, et al, eds. Yearbook of emergency medicine. St. Louis, MO: Mosby, Inc; 2002;36:151–170.

26. Kim MK, Strait RT, Sato TT, Hennes HM. A randomized clinical trial of analgesia in children with acute abdominal pain. Acad Emerg Med 2002;9:281–287.

27. Stanniland JR, Ditchburn J, de Dombal FT. Clinical presenta-tion of acute abdomen: study of 600 patients. Br Med J 1972; 3:393–398.

28. Mahadevan M, Graff L. Prospective randomized study of anal-gesic use for ED patients with right lower quadrant pain. Am J Emerg Med 2000;18:753–756.

References

Related documents

Preference for specificity Only specific effects attributable to the intervention are therapeutically valid Context independence There is a "true" magnitude of efficacy, or

Considering situation of variables of research: To assess and evaluate the variable of knowledge management level in study wholesalers industry, the four aspects (possession

The organizers brought together immigration officials, child welfare profes- sionals, and refugee advocates, in an effort to build partner- ships and improve awareness of the problem

The results showed a strong relationship between supervisors’ communicator competence and their task and relational leadership styles, with supervisor communicator competence being

In each case we conducted a 50-minute focus group of 3-5 mathematics teachers exploring the strategies they considered significant for retaining girls in mathematics; a focus

100 Billion searches are conducted. every month

17 TOURISM INTELLIGENCE 5 core units Regional Information Systems Statistical System Dynamic Pricing Monitor Mobility on destination Social Media Monitor Tourism

Fresh leaves of distinctive plants like Lagenaria sicereria, Acalypha indica, Melia dubia, Phyllanthus emblica and the gel of Aloe vera were collected.. The