• No results found

MONAHRQ Host User Guide. Version 1.0

N/A
N/A
Protected

Academic year: 2021

Share "MONAHRQ Host User Guide. Version 1.0"

Copied!
63
0
0

Loading.... (view fulltext now)

Full text

(1)

 

 

MONAHRQ Host User Guide

Version 1.0 May 27, 2010

             

(2)

TABLE OF CONTENTS

INTRODUCTION... 2

PART I: DOWNLOADING MONAHRQ ... 3

Check that Your System Meets Hardware and Software Requirements... 3

     Step‐by‐Step Installation ... 4 

PART II: BUILDING YOUR MONAHRQ WEBSITE... 6

Preparing Your Data ... 6

Comparing Quality Indicators between MONAHRQ and SAS ... 7

List of AHRQ Quality Indicators available in MONAHRQ ... 7

Building Instructions – Running the Software... 19

PART III: CUSTOMIZING WEB PAGES... 48

Introduction to MONAHRQ Website Architecture... 48

Folder Structure of the Website ... 48

Adding/Removing Content... 49

APPENDIX: REPORTING QUALITY OF CARE TO CONSUMERS ... 52

FREQUENTLY ASKED QUESTIONS... 62

    INDEX OF TABLES Table 1: Preparing Your Data... 9

Table 2: Present on Admission Coding ... 18  

Table 3: Navigation and Content Page Structure18……….49

 

(3)

INTRODUCTION

MONAHRQ is a free Windows-based software product that enables host users—such as State and local data organizations, chartered value exchanges, hospitals, and health plans—to input their own raw inpatient hospital administrative data and generate a data-driven Website. This tool was developed by the Agency for Healthcare Research and Quality, the Federal Government’s lead agency for health care quality in the United States. MONAHRQ is based on two of AHRQ’s most popular and widely respected tools, the Quality Indicators (QIs) and HCUPnet, and it incorporates several other AHRQ tools as well.

This innovative tool allows users to navigate through a series of simple steps to ultimately generate powerful Web-based reports. This Website, with reports in HTML-pages, will help health care professionals and consumers make informed decisions about health care quality and performance.

MONAHRQ analyzes, summarizes, and presents the information in an evidence-driven, easy-to- understand format that can be easily hosted on an organization’s Website and is instantly ready for use internally by the organization or externally by consumers and other decision makers.

This user documentation is designed to help host organizations create their own MONAHRQ-generated Website. The instructions provided in this document will guide users through the process step-by-step to facilitate the installation and implementation of MONAHRQ specific to the needs of each host

organization. The document is organized by “user type” to guide the implementation process and allow for a team of specialized personnel to work together while crafting the tool according to the needs of each organization.

• Download. Information for System Administrators regarding system requirements and instructions for downloading MONAHRQ and any necessary supplemental software (e.g., SQL).

• Build. Details for Programmers on how to prepare data and load it into MONAHRQ. This process will involve mapping your data elements, uploading hospital data, conducting analyses, and understanding various implications.

Host. Guidelines and suggestions for Web masters on hosting the tool on your organization’s Website. This involves learning the many customization options available to users, evaluating output pages, and ultimately refining the pages to meet the organization’s primary data interests.

MONAHRQ can be used in a variety of ways to meet the interests of your organization. Here are some examples of how MONAHRQ can be used:

• Within your organization to generate reports and statistics you use internally

• To create a limited-access Website for member organizations

• To create an open-access Website to be used by consumers and other decision makers to compare various facilities in an area or to present health care outcomes in a geographic region.

In addition, this document provides guidance to host users in reporting hospital quality. This memo was developed by Shoshanna Sofaer for users of the AHRQ QIs. It not only provides guidance on quality reporting, but also describes the context for development and testing of the QI consumer pathway in MONAHRQ.

The choice is yours as to how you would like to use MONAHRQ – your data on your Website.

For more information, please visit http://www.monahrq.ahrq.gov or contact us by email at

[email protected].

(4)

PART I: DOWNLOADING MONAHRQ: INFORMATION FOR A SYSTEM ADMINISTRATOR This section provides directions for installing MONAHRQ and describes system requirements.

MONAHRQ needs .NET and SQL to run successfully, and these instructions will help you determine if you need to download a free version of SQL or .NET. We also provide some helpful hints for system administrators.

Important: MONAHRQ CANNOT be downloaded onto a server. Please load it directly onto your PC.

The HTML pages created by the software can later be moved to a server to display on your organization’s Website.

Before you begin, make sure that you have the appropriate Administrator permissions to install software on your computer. If a system administrator downloads the software on behalf of another user, ensure that the appropriate access privileges are granted. This is only required during the installation. You should also make sure your Windows operating system has the latest Service Pack and updates applied.

If you have any previous versions of MONAHRQ on your computer, you need to uninstall using Add/Remove Programs.

1. Open the Windows Control Panel and use the Add or Remove Programs utility to get a list of software programs installed on your PC. You can access the Control Panel from the Windows Start button via the "Settings" option.

2. Scroll down the alphabetical list of Programs until you get to MONAHRQ; select MONAHRQ and then select the Remove button.

3. When the program has been removed, close all windows.

4. Restart the system before installing the updated MONAHRQ.

This section will walk you step-by-step through the process of:

1. Determining whether your system meets hardware and software requirements for MONAHRQ.

2. Installing Microsoft .NET Framework (if needed).

3. Installing Microsoft SQL Server Express (if needed).

4. Installing the AHRQ Quality Indicator Risk Adjustment.

5. Installing MONAHRQ.

Check That Your System Meets Hardware and Software Requirements

MONAHRQ runs on computers with Windows XP, Windows Vista, Windows Server 2003, or higher operating systems. The following software is required before installing MONAHRQ and can be downloaded from the MONAHRQ download Website (http://www.monahrq.ahrq.gov ):

• Microsoft .NET Version 2.0 or higher.

• Microsoft SQL Server Express if a local database is used.

Approximate disk space requirements for the three components are:

• Microsoft .NET – 300 MB.

• Microsoft SQL Server Express – 600 MB.

• AHRQ Quality Indicator Risk Adjustment – 2 MB.

• MONAHRQ – 50 MB.

• MONAHRQ data – Requirements vary depending on the number of discharges you wish to

process. About 100 MB is typical, but this can be up to 4GB.

(5)

MONAHRQ is a single-user desktop application that requires a SQL Server database to store data. It has been tested on two versions of Microsoft SQL Server (2005 and 2008). Each of these versions of Microsoft SQL Server has several editions ranging from the free edition that is provided (SQL Server Express) to the Data Center version. Microsoft SQL Server can be installed on your PC or accessed over a network. Most users prefer to use SQL Server 2005 Express Edition installed on their PC unless local IT policies prohibit this setup. If you have an especially large dataset, it will be more efficient to use the full SQL Server rather than the free Express Edition.

Existing SQL Server

If you choose to use an existing SQL Server, contact your database administrator for the connection host name, login, and password that will be required by the MONAHRQ installer.

Free SQL Server

Microsoft SQL Server 2005 Express Edition is the current free database from Microsoft. This edition can be downloaded from the same site as MONAHRQ.

Step-by-Step Installation

Step 1

Check that the Microsoft .NET 2.0 Framework is installed on your computer. Open the Windows Control Panel and use the Add or Remove Programs utility to get a list of software programs installed on your PC. You can access the Control Panel from the Windows Start Button via the "Settings" option. Scroll down the alphabetical list of Programs until you reach Microsoft programs. The image below shows .NET Framework with a service pack. Please note that any service pack level will work.

If you do not have the Microsoft .NET 2.0 Framework installed, then download the correct install package (32-bit Microsoft .NET 2.0 Framework or 64-bit Microsoft .NET 2.0 Framework). Most users have a 32-bit version of Windows on their computers, which will use the 32 bit link. If the computer uses a 64-bit version, then the 64-bit version of Microsoft .NET 2.0 Framework must be installed. Select and save the correct version and then run the file.

To check the version of Windows, right-select the My Computer icon on your desktop and select

Properties. You can also select My Computer from the Start Menu and select View System Information.

A pop-up box displaying your version of Windows will appear. If it does not say 64 bit, then your system

is 32 bit. Below is an example of the Properties dialog box for a system that uses the 32-bit version.

(6)

  Step 2

Because the software will need to set up a Microsoft SQL Server database, an instance of Microsoft SQL Server 2005 will need to be accessed. If you have access to a copy running on a database server in your network then that instance can be used. If not, a free version of Microsoft SQL Server Express can be obtained from the MONAHRQ Website. As with the .NET framework, the 64-bit version will need to be used if you are running a 64-bit version of Windows. Step 1 describes how to check your version of Windows. Select and save the correct version and then run the program.

  Step 3

The third step is to download and install the AHRQ Quality Indicator Risk Adjustment. MONAHRQ requires that this file be downloaded to correctly calculate the analyses. Select and save the AHRQ Quality Indicator Risk Adjustment file.

Step 4

The final step is to download and install MONAHRQ. The install package will prompt you with several questions. If you are using a network copy of Microsoft SQL Server, you will need to know the correct network name of the instance. If you set up a local copy of Microsoft SQL Server Express, you can use the default answer when prompted.

The install process will first load the MONAHRQ software on your PC, then access the Microsoft SQL Server instance and create your MONAHRQ database, and finish by loading baseline data into the database. The process will take up to 15 minutes, depending on the speed of your PC. Progress meters keep you informed on the progress of the setup process.

Additional Information

If the person installing the MONAHRQ software is not the person who will be using MONAHRQ or if there will be more than one MONAHRQ user on the machine, then the System Administrator will need to add users to the "MONAHRQ" database. This can be done with a remote SQL Server Manager or by

installing a local copy of the SQL Server Management Studio Express Edition and using it to add the required users.

You can download the free SQL Server Management Studio Express Edition from

(7)

PART II: BUILDING YOUR MONAHRQ WEBSITE: INFORMATION FOR A PROGRAMMER This section will provide programmers with the critical information they need to prepare the data and run the software. This easy step-by-step process is demonstrated in a series of basic instructions. We also include detailed screenshots and data tables so that you can incorporate your organization’s inpatient administrative data the way you want to present it.

More specifically, this process will involve:

• Understanding the data elements needed for the analyses, as well those that are optional.

• Knowing the implications of missing data elements and how this will affect output pages.

• Developing custom dictionaries.

• Building a Website for your organization.

Preparing Your Data

MONAHRQ uses hospital inpatient administrative data that provides demographics on the patient and the provider, diagnosis codes, procedure codes, and information about the admission, payer(s), and

discharge. The software is designed for processing one calendar year of data at a time. The software will walk you through a very simple, “point and click” process for mapping your data elements and value codes. The software is designed to be easily usable on raw (i.e., source) administrative data.

The software accepts three common formats for your data:

1. Text (comma-separated values, CSV).

2. Microsoft Access table.

3. Microsoft Excel spreadsheet.

Two key formatting issues are:

• Each row of data represents a separate discharge record.

• Each column of data represents a single variable for all discharges.

CSV files use commas to separate the data values. If you have commas within any data values (for example, "Private, incl. HMO"), you will need to put double quotes around each data element. An exception is the variable “Total Charge” (refer to attached table). Many data elements in hospital inpatient data have leading zeros; if you are working from Excel, we recommend that all appropriate fields/cells be formatted as text to ensure full conversion of the data.

Input data have specific meaning according to the coding conventions in your organization. The data need to be mapped to the specific meaning used by MONAHRQ. The data elements in MONAHRQ are based on the coding specifications used in the State Inpatient Data (SID) in the Healthcare Cost and Utilization Project (HCUP), which are similar to the Uniform Bill (UB-92/04), but not identical.

MONAHRQ’s Crosswalk Screen provides the opportunity to map your variable values to the values used in the software. Present on Admission is the only variable with values that are automatically mapped.

Please review Table 2 to ensure that your data are coded correctly. You may prepare your dataset in advance by using names and codes that match those in MONAHRQ so that the software will

automatically recognize data element names and value codes.

Because MONAHRQ is designed to recognize HCUP formatted data, if you are using HCUP formatted

data, most data elements and data values will be mapped for you when you load the software. HCUP

(8)

to Table 1. If your data elements used the same names and coding values as shown in Table 1, the process of identifying and mapping data elements will be quicker. Table 1 also identifies which data elements are required and what happens if an optional element is missing.

When you prepare your data, it is not necessary to create "dummy variables" or fill in missing values.

Your input file may contain extra data that are not required; you do not need to remove extra variables.

Any variables that are not used in the Data Mapping Screen will not be imported with your data.

Comparing Quality Indicators between MONAHRQ and the SAS Version

If you would like to compare the quality indicators produced by MONAHRQ with the SAS version of the QIs then you will need to do the following:

• First run your data through MONAHRQ and save your MONAHRQ generated dataset (screen 14 has directions).

• MONAHRQ assigns a DRG for each discharge. The MONAHRQ assigned DRG should be used when you run the QI SAS programs. In the MONAHRQ generated dataset, there are two DRG data elements. When the DRG value is equal to or less than 24, the data element DRGVER should be used for the SAS program. When the DRG value is greater than or equal to 25, the data element DRG_VER should be used for the SAS program.

• We also recommend that you subset the data loaded into the SAS programs to those discharges that were included in MONAHRQ (some discharges may be excluded from MONAHRQ).

• Note that MONAHRQ and SAS round statistics to different levels.

For additional information about the AHRQ SAS QI programs, please visit http://www.qualityindicators.ahrq.gov/.

List of AHRQ Quality Indicators Available in MONAHRQ

The following indicators are included in MONAHRQ:

Prevention Quality Indicators (17 indicators)

• Diabetes, short-term complications (PQI 1)

• Perforated appendicitis (PQI 2)

• Diabetes, long-term complications (PQI 3)

• Chronic obstructive pulmonary disease (PQI 5)

• Hypertension (PQI 7)

• Congestive heart failure (PQI 8)

• Low birth weight (PQI 9)

• Dehydration (PQI 10)

• Bacterial pneumonia (PQI 11)

• Urinary infections (PQI 12)

• Angina without procedure (PQI 13)

• Uncontrolled diabetes (PQI 14)

• Adult asthma (PQI 15)

• Lower extremity amputations among patients with diabetes (PQI 16)

• Prevention Quality Indicator Composite - Overall

• Prevention Quality Indicator Composite - Chronic Conditions

• Prevention Quality Indicator Composite - Acute Conditions Inpatient Quality Indicators

1. Mortality Rates for Medical Conditions (4 Indicators)

• Acute myocardial infarction (AMI) (IQI 15)

• Congestive heart failure (IQI 16)

• Stroke

(9)

2. Mortality Rates for Surgical Procedures (5 Indicators)

• Esophageal resection (IQI 8)

• Abdominal aortic aneurysm repair (IQI 11)

• Percutaneous transluminal coronary angioplasty (IQI 30)

• Carotid endarterectomy (IQI 31)

• Hip replacement (IQI 14)

3. Hospital-level Procedure Utilization Rates (3 Indicators)

• Laparoscopic cholecystectomy (IQI 23)

• Incidental appendectomy in the elderly (IQI 24)

• Bi-lateral cardiac catheterization (IQI 25) 4. Area-level Utilization Rates (4 Indicators)

• Coronary artery bypass graft (IQI 26)

• Percutaneous transluminal coronary angioplasty (IQI 27)

• Hysterectomy (IQI 28)

• Laminectomy or spinal fusion (IQI 29) 5. Volume of Procedures (4 Indicators)

• Esophageal resection (IQI 1)

• Abdominal aortic aneurysm repair (IQI 4)

• Percutaneous transluminal coronary angioplasty (IQI 6)

• Carotid endarterectomy (IQI 7) Patient Safety Indicators

1. Hospital-level Patient Safety Indicators (13 Indicators)

• Death in low mortality DRGs (PSI 2)

• Iatrogenic pneumothorax (PSI 6)

• Postoperative hip fracture (PSI 8)

• Postoperative physiologic and metabolic derangements (PSI 10)

• Postoperative respiratory failure (PSI 11)

• Postoperative pulmonary embolism or deep vein thrombosis (PSI 12)

• Postoperative sepsis (PSI 13)

• Postoperative wound dehiscence in abdominopelvic surgical patients (PSI 14)

• Accidental puncture and laceration (PSI 15)

• Birth trauma -- injury to neonate (PSI 17)

• Obstetric trauma -- vaginal delivery with instrument (PSI 18)

• Obstetric trauma -- vaginal delivery without instrument (PSI 19)

• Death in surgical inpatients (PSI 4)

2. Area-level Patient Safety Indicators (7 Indicators)

• Foreign body left in during procedure (PSI 21)

• Iatrogenic pneumothorax (PSI 22)

• Selected infections due to medical care (PSI 23)

• Postoperative wound dehiscence in abdominopelvic surgical patients (PSI 24)

• Accidental puncture and laceration (PSI 25)

• Transfusion reaction (PSI 26)

• Post-operative hemorrhage or hematoma (PSI 27)

(10)

Table 1: Preparing Your Data   

 

MONAHRQ Variable

Name Age

Sex

Hospital ID

Year

Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Age in years at admission

Gender of

patient:

male/female

Data source hospital number

Calendar year

of patient's discharge

Required

Required

Required

Required

If this data element is missing, the discharge record will not be loaded.

If this data element is missing, the discharge record will not be loaded.

Data element used to facilitate data exploration and as a stratifier for provider-level indicators (in the QI reports section). If this data element is missing, the discharge record will be not be loaded.

Data element used to apply the proper fiscal year coding (e.g., ICD-9, CPT) and to assign the APR™ DRG Grouper used. Discharge year should be within the range of 1997 to present year. If this data element is missing, the discharge record will be not be loaded.

Source value

1: Male 2: Female Other values mapped to

<Exclude from dataset>

Source value

Source value, YYYY

Numeric. Convert to years; if age <365 days, set value to 0. If variable does not exist, it should be calculated from Admission Date and Date of Birth.

No data preparation needed.

Source values, alpha or numeric, will be mapped to accepted numeric value (1, 2) or excluded during data load.

No data preparation needed.

Source values, alpha or numeric, accepted.

Numeric: YYYY

Discharge year should be within the range of 1997 to

present year.

(11)

MONAHRQ Variable

Name Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Discharge Quarter

Calendar quarter of the patient’s discharge

Required Data element used to apply the proper fiscal year coding (e.g., ICD-9, CPT) and to assign the APR™ DRG Grouper used. If this data element is missing, the discharge record will be not be loaded.

1: January-March 2: April-June 3: July- September 4: October- December

If data element does not exist, it should be calculated from discharge date. Value must be numeric (1, 2, 3, 4) with no leading alpha characters.

Principal Diagnosis

ICD-9-CM diagnosis code without decimal points.

Diagnosis 1 is the principal diagnosis.

Required If this data element is missing, the discharge record will be not be loaded.

Source value;

string value more than 5 characters will be shortened.

Decimal points, if any, must be removed before loading data. Do not remove leading or trailing zeros. Similarly, you should not include additional digits when they are not required. Diagnosis codes are always 3, 4, or 5

characters long. For example, a diagnosis code of 005.89 would be coded as 00589 in MONAHRQ

Key Unique case

identifier

Optional If this data element is not available, users cannot link the discharge records in the Patient-Level Report back to the input data file.

Source value Maximum length: 20 characters

Age in Days Age in days at admission (coded only when the age in years is less than 1)

Optional Used in the inclusion and exclusion criteria for indicators addressing neonates or neonatal conditions and in the Pediatric Quality Indicators (PDIs). If this data element is missing (and age is 0), generally, an alternative specification applies.

Age in days only applies for age <

1. If value is greater than 365, value will be changed to Missing.

Numeric: 0-364

(12)

MONAHRQ Variable

Name Race

Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Source values, alpha or numeric, can be mapped to accepted numeric value (1-6) or excluded during data value mapping.

Race of patient Optional Used to stratify the AHRQ QI rates. Records with this data element missing will be retained and the value set to Other. The rates and utilization paths will not be stratified by race if the data element is completely missing.

1: White 2: Black 3: Hispanic 4: Asian or Pacific Islander

5: Native American 6: Other 0: Missing 99: Retain value

<Exclude from dataset >

Primary Payer Expected Optional Used to stratify the AHRQ QI 1: Medicare Source values, alpha or numeric, can be mapped to primary payer rates and utilization

statistics. Records with this data element missing will be retained and the value set to Other.

2: Medicaid 3: Private/HMO 4: Self-pay 5: No charge 6: Other 0: Missing 99: Retain value

<Exclude from dataset >

accepted numeric value (0-6, 99) or excluded during data

value mapping.

(13)

MONAHRQ Variable

Name Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Patient State/County Code

Federal Information Processing Standard (FIPS) State/county code of patient’s residence

Optional If this data element is missing, the discharge record will be excluded from area rate calculations and the Website Wizard cannot create maps by showing rates of preventable hospitalization by area. If patient codes are not available, hospital’s codes can be loaded. We

recommend that you analyze the area rates at the State or metro area level. Otherwise, patients who reside outside the same county as the hospital will be included in the numerator but not the denominator. The larger the geographic unit of analysis, the less likely it is that this situation will occur. If the hospital FIPS codes are used instead of the patient FIPS codes, the area rates must be interpreted with caution.

Source value We recommend that you use the patient FIPS

State/county code. FIPS codes may be obtained at

http://www.census.gov/popest/geographic/codes02.html.

(14)

MONAHRQ

Variable Required/ Ramifications of Default Element

Name Description Optional Exclusion Coding Data Preparation

Discharge Disposition of Optional Used in the inclusion and 1: Routine No data preparation needed.

Disposition patient exclusion criteria for several 2: Short-term Source values, alpha or numeric, will be mapped to Prevention QIs (PQIs), hospital accepted numeric values (0-7, 20) or excluded during Patient Safety Indicators 3: Skilled nursing data load.

(PSIs), and Inpatient QIs facility

(IQIs). For indicators that 4: Intermediate rely on this field, records care

with this data element 5: Another type of missing will be excluded facility

from the denominator. 6: Home health care

7: Against medical advice

20: Died in the hospital 0: Missing

<Exclude from dataset>

99: Missing

<Exclude from dataset>

Admission Type Admission type Optional Used in the inclusion and 1: Emergency No data preparation needed.

exclusion criteria for several 2: Urgent Source values, alpha or numeric, will be mapped to PQIs, PSIs, and IQIs. For 3: Elective accepted numeric values (0-6) or excluded during data indicators that rely on this 4: Newborn load.

field, records with this data 5: Trauma center element missing will be 6: Other

excluded from the 0: Missing denominator. <Exclude from

dataset>

(15)

MONAHRQ Variable

Name Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Admission Admission Optional Used in the inclusion and 1: Emergency No data preparation needed.

Source source exclusion criteria for several

PQIs, PSIs, and IQIs. For indicators that rely on this field, records with this data element missing will be excluded from the denominator.

room

2: Another hospital 3: Another facility, including long- term care 4: Court/law enforcement 5: Routine/birth/

other 0: Missing

<Exclude from dataset>

Source values, alpha or numeric, will be mapped to accepted numeric values (0-5) or excluded during data load.

Length of Stay Number of days from admission to discharge

Optional Used in the exclusion criteria for several PSIs and PDIs;

not used in PQIs or IQIs. If this data element is not available, indicators that rely on this field will be excluded from the denominator. In the utilization pathway, statistics by length of stay will be excluded if the data element is missing.

Source value Calculate if needed, from discharge data and admission date. Same-day stay should be set to 0.

Days on Number of days Optional Optional data element is Source value Mechanical the patient passed directly to the APR™

Ventilator spent on a mechanical ventilator

DRG Grouper. If this data

element is not available,

value will be set to default in

the grouper software.

(16)

MONAHRQ Variable

Name Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Birthweight in Grams

Birthweight for newborns

Optional Optional data element that is passed directly to the APR™

DRG Grouper. If this data element is not available, value will be set to default in the grouper software. This field is not used as

stratification criteria; ICD-9- CM diagnosis codes are used to indicate birthweight.

If value greater is than 7,000, value will be changed to Missing because higher values are considered invalid birthweights.

Total Charge Total charge associated with hospital stay

Optional If this data element is not available, cost savings associated with reducing the level of potentially avoidable hospitalizations will not be included in summary report, costs and charges will be excluded from the utilization path, and cost will be excluded from the rates.

Source value.

Must be integer (i.e., whole numbers only).

Must be integer: remove dollar signs and decimals (i.e., whole numbers only).

Diagnosis Code 2 – Diagnosis Code 35

Codes 2-35 are secondary diagnoses and would include any External Cause of Injury codes (E- codes).

Optional Used in the inclusion and exclusion criteria for multiple indicators. The number of reported codes will affect rates.

Source value;

string value more than 5 characters will be shortened.

Decimal points, if any, must be removed before loading data. Do not remove leading or trailing zeros. Similarly, you should not include additional digits when these are not required. Diagnosis codes are always 3, 4, or 5 characters long. Secondary diagnosis codes may include External Cause of Injury codes (E-codes).

Principal Procedure

ICD-9-CM Procedure Codes without decimals.

Procedure code 1 is the principal procedure.

Optional Used in the inclusion and exclusion criteria for several indicators.

Source value.

String value more than 4 characters will be shortened.

Procedure codes are always 2, 3, or 4 characters. As

with diagnosis codes, you should remove any decimal

points and you should not add or remove characters on

the left or ride side of the code.

(17)

MONAHRQ Variable

Name Procedure Code 2 – Procedure Code 30

Days to Procedure 1 – Days to Procedure 30

Custom Stratifier 1 - Custom Stratifier 3

Present on Admission 1 - Present on Admission 35

Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

ICD-9-CM Procedure Codes without decimals.

Procedure codes 2-30 are secondary procedures.

Days from admission to procedure.

Procedure 1 is the principal procedure;

procedures 2-30 are secondary procedures.

Custom stratification values

Flag indicating whether diagnosis was present on admission

Optional

Optional

Optional

Optional

Used in the inclusion and exclusion criteria for multiple indicators. The number of reported codes will affect rates.

Used in several PSIs and PDIs. If this data element is not available, an alternative logic applies.

Custom stratifiers can be used in the reports section of the software (e.g., a user could stratify by type of hospital – teaching or nonteaching). This data element has no effect on the generated HTML pages.

Present on admission (POA) data elements may eliminate false positives from PSI results. IMPORTANT: If you use these present on admission fields, a different set of risk adjustment covariates and reference population rates will be applied.

Source value.

String value more than 4 characters will be shortened.

Source value. It is expected that the number of Days to Procedure

variables agrees with the number of procedure codes present.

1 = present at the time of inpatient admission 0 = not present at the time of inpatient admission

Procedure codes are always 2, 3, or 4 characters. As with diagnosis codes, you should remove any decimal points and you should not add or remove characters on the left or ride side of the code.

If the data element does not exist, it should be calculated from the admission data and the procedure date(s).

Present on Admission flag (POA) should be included for all records or none of them. Mixing records with and without POA could adversely affect the expected rates.

Please see Table 2 for a more detailed coding

explanation for POA.

(18)

MONAHRQ Variable

Name Description

Required/

Optional

Ramifications of Exclusion

Default Element

Coding Data Preparation

Patient ID Patient ID or medical record number for identification purposes only

Optional None It is recommended

that you DOT NOT USE this field unless required for external analysis.

It is recommended that you DO NOT USE this field.

Date of Birth Patient date of birth for identification purposes only

Optional None It is recommended

that you DOT NOT USE this field unless required for external analysis, MM/DD/YYYY

It is recommended that you DO NOT USE this field, MM/DD/YYYY.

Admission Date Date of patient admission for identification purposes only

Optional None It is recommended

that you DOT NOT USE this field unless required for external analysis, MM/DD/YYYY

It is recommended that you DO NOT USE this field, MM/DD/YYYY.

Discharge Date Date of patient discharge for identification purposes only

Optional None It is recommended

that you DOT NOT USE this field unless required for external analysis, MM/DD/YYYY

It is recommended that you DO NOT USE this field,

MM/DD/YYYY.

(19)

Table 2: Present on Admission Coding  

HCUP Data

ICD-9-CM Guidelines ICD-9-CM Description Element HCUP Description Y - Yes Present at the time of 1 Diagnosis present at

inpatient admission. admission.

N - No Not present at the time of 0 Diagnosis not present at

inpatient admission. admission.

U - Unknown Documentation is insufficient 0 Diagnosis not present at to determine if condition is admission.

present on admission.

W – Clinically undetermined Provider is unable to clinically 1 Diagnosis present at determine whether condition admission.

was present on admission or not.

E - Unreported/Not used Exempt from POA reporting. 1 Diagnosis present at admission.

1 - Yes Present at the time of 1 Diagnosis present at

inpatient admission. admission.

0 - No Not present at the time of 0 Diagnosis not present at

inpatient admission. admission.

Blank Missing Blank Missing

(20)

Building Instructions – Running the Software

This section includes screenshots of the software with helpful hints and background information under each screenshot. This will walk you through the process of loading your data, conducting analyses, and generating a Website.

NOTE: The building instructions use Maryland as an example State. These data do NOT reflect Maryland discharges – the discharge and hospital data have been randomly generated for testing purposes only.

Screen 1 – AHRQ MONAHRQ Welcome Page

1. Once MONAHRQ has been loaded and the data have been prepared, you will be able to build the Website. On the first screen, select View Session Log before choosing an option on the Task Menu.

This log will help you or MONAHRQ technical support identify any errors or problems you may have while creating the Website. Please note that you will not be able to access the log once the program, also called a wizard, starts.

Begin by viewing the log session and then select the Database Manager. You must identify or create a

SQL database before beginning the data import process.

(21)

Screen 2 – Database Manager

     

2. You must first create your MONAHRQ SQL database. If you are using SQL Express, the server name and authentication is prefilled for you. If you are using SQL Server, you will need to alter the server name and enter a username and password.

Once you have provided the database information, select Create or Overwrite to create a new database.

When the process has finished, you may select Done.

Each time you generate a different MONAHRQ Website that you would like to potentially alter at a later time, we suggest you create a new SQL database. For example, if you would like to create a MONAHRQ Website for 2006 and another Website for 2007, you would create distinct SQL databases (e.g.,

MONAHRQ_2006 and MONAHRQ_2007). When you would like to alter a previous Website, you will type in the name of the database and select Switch Connection and then Done.

Once you have created a database, you will return to the MONAHRQ Welcome Page and can select

Import Data to continue. Follow the data steps in the order shown on the left side of the page. If you

have a particularly large dataset with greater than 400 hospitals then you will need to select the Program

Options before proceeding to the Import Data Wizard, screen 25 provides more detail.

(22)

Screen 3 – Import Data Wizard

3. This screen explains the Import Data process – select Begin to continue.

(23)

Screen 4 – Select Input File

4. Select the Browse button to locate the discharge data file to be loaded. Once you have found the appropriate file, check an option in the Import Data File Options (Specific to File Type) box:

• If applicable, check First row contains column headings.

• If you are unsure of data format, check Values are enclosed in quotes.

There are three types of files that MONAHRQ currently accepts: CSV, XLS (Excel), and MDB (MS Access). Users have experienced difficulty using Excel files due to how Excel handles character fields and leading zeroes; we recommend that you confirm that the Excel file has maintained the original data values before loading the file into MONAHRQ.

Then select an option in the Data Mapping and Crosswalk box:

• If this is the first time you are loading the data (i.e., you do not have a previously created data mapping file from MONAHRQ), select Data Layout Unknown.

• If you previously loaded your data and created a data mapping file in MONAHRQ, select Browse to locate the .qim mapping file.

o If you are using a .qim file, you can check Skip data validation and mapping screens.

Once you have completed this page, select Next. You can return to the previous page by using the Back button, which appears on the bottom of this page and subsequent pages.

 

(24)

Screen 5 – Check File Readability

 

5. Select a file that contains one calendar year of administrative data. MONAHRQ only allows one calendar year of data to be analyzed at a time. If you have fiscal data that span two calendar years and would like to include all records in your analysis, the values in the source data for the variable Year will need to be manipulated before loading the data. You may alter the fiscal data to reflect either the later or former calendar year (e.g., 2006-2007 fiscal year data would need to be coded as either 2006 or 2007).

MONAHRQ will check to ensure that the data are legible and that each row has the same number of columns. On the Check File Readability screen, verify the file selection shown. If correct, select the Start button. The data will automatically start loading. You may select Stop to terminate the process (the Start button will change to Stop once the checking process begins).

When the check is complete, the Status message will read Finished.

Select Next to continue.

(25)

Screen 6 – Data Mapping

6. Once the data have been loaded, you will be asked to map your dataset to the MONAHRQ variable names. MONAHRQ’s Crosswalk Screen (above) provides the opportunity to map your data elements to the data elements used in the software. While a sample of your dataset is provided on the screen, it is useful to either know your element names or have access to your data dictionary.

Drag and drop the MONAHRQ Variable boxes to the Input Variable columns (right to left). All of the required MONAHRQ variables must be linked to an input file variable. MONAHRQ will not run without all required variables. The optional fields are not required, but as many variables as possible should be mapped to optimize the output. MONAHRQ has been programmed to “automatically guess” some of the mapping options, so it is important that you check these to make sure they are correct.

Data elements in your discharge data that have the same name as MONAHRQ data elements will

automatically be mapped for you. Variable names used in MONAHRQ are the same as those that appear in HCUP’s State Inpatient Databases (SID). If a variable is mapped incorrectly, simply drag the mapped variable to the correct input file element or drag it back to the MONAHRQ variable column on the right side of the screen.

Please refer to Tables 1 and 2 for a complete listing of variables names, descriptions, and coding.

Note that Present on Admission (POA) is automatically mapped by MONAHRQ.

Once you have finished mapping elements, select Next.

(26)

Screen 7 – Variable Mapping

7. On the Summary of Variables screen, it is important to focus on the number of unmapped required variables. Unmapped Required MONAHRQ Variables should have a value of zero. If there was a data load error or if you did not crosswalk all of the required variables, it will be another number.

Once the Unmapped Required MONAHRQ Variables number is at zero and the number of variables in the input file match, you may select Save Report to create an .rtf file of information on the screen.

Note that the POA value mappings are provided on this screen.

Select Next to continue.

(27)

Screen 8 – Check for Data Errors

8. To check for errors within the mapped dataset, select the Start button. You may select Stop to terminate the process (the Start button will change to Stop once the checking process begins).

When the check is complete, Status changes to Finished.

Select Next to continue.

(28)

Screen 9 – Data Errors Report

9. The Data Errors Report will show the number of records affected by data errors. If a data error occurs in a required field and affects a large number of records in your data file, the Web pages produced may be incomplete.

Some errors may be acceptable. For example, if the Age in Days element is greater than 365, the software uses the Age variable. For other elements, the acceptability of an error is based on host user discretion, such as if the error only affects a small number of records or if it occurs in a variable that is not required for the analysis. Finally, some errors may require research and/or manipulation of the input data file. If you manipulate the input data file, you will need to start the data load from the beginning. Below are four common errors and guidelines on checking them:

1. Required field empty – record will not be loaded.

Verify that the count is a small percentage of your discharges. If the error affects a large number of records, make sure that the variable mapping was correct (use the Back button).

2. Diagnosis Codes/Procedure Codes: Invalid value. Valid codes must be at least 3 characters.

Verify that the count is a small percentage of your discharges and investigate the input data values. For example, how are missing values identified?

3. Birth Weight Grams: Value less than 200.

Value will be changed to 'Missing' and/or Value greater than 7,000.

4. Age in Days: Age is greater than zero.

Age in Days only applies for Age less than 1 year. If the value is greater than 365 days, it will be changed to 'Missing.’

To correct errors, use the Back button to return to the Data Mapping screen to review and correct the

mapping of MONAHRQ variables to input file variables. Once the results are to your satisfaction, select

Save Report if you would like to create an .rtf file of information on the screen

(29)

Screen 10 – Crosswalk Values

10. Once the data elements are loaded, the values for each element will need to be determined. Use your own data documentation to indicate the value label. We recommend that each input value be reviewed to ensure the correct value meaning was assigned to your data.

If your data are formatted in the HCUP standard or you have altered the data according to Table 1, the software will crosswalk values and meanings for you. You should still review the values and meanings for accuracy.

Please note that Present on Admission is automatically mapped by MONAHRQ. Refer to Table 2 for detailed coding information.

Once all variables have been coded, continue by selecting Next.

(30)

Screen 11 – Load Discharge Data

11. To begin loading your discharge data, select the Start button. You may select Stop to terminate the process (the Start button changes to Stop once the load process begins).

Depending on the number of records, the data load process can take a longer amount of time. When the loading process is complete, the Status changes to Finished.

Select Next to continue.

(31)

Screen 12 – Data Load Report

12. Once your data have finished loading, you will be taken to a Data Load Summary page. Warning messages are shown in red and green font and indicate inconsistencies with the loaded data that may affect the quality indicator calculations. To ensure that your data will load correctly, you’ll need to review all messages in red.

In addition, the number of records with Required field empty – Rows not loaded should be low. If there are substantial amounts of missing data for any given (or combination of) variables, the overall number of discharges will decrease accordingly. For analyses with small populations, the results may be statistically unreliable.

Select Save Report to create an .rtf file of the Data Load Summary information.

Select Next to continue.

(32)

Screen 13 – Generate Indicator Flags

   

13. To run the analyses on loaded data, select the Start button. You may select Stop to terminate the process (the Start button will change to Stop once the load process begins).

Please note that the analyses may take a significant amount of time to run (this varies given the size of the input file – but it may be multiple hours). You may leave MONAHRQ running in the background of your computer and perform other tasks during this time. Certain queries (especially those related to the QI risk adjustment) may take more than an hour to process on a large dataset and it may falsely appear that the software has stalled. For most datasets, the analyses will run within 6-7 hours. If you have a large data set, it may take 25-30 hours to run the analyses and compute the risk-adjustment. When the analyses are complete, the Overall Progress line will read “All 1090 queries completed.”

Select Next to continue.

(33)

Screen 14 – Save Data and Mapping

14. MONAHRQ allows users to save the data, but this step is not required. If you would like to save your MONAHRQ data in a comma-separated values (CSV) format for use outside the tool, select the variables you would like to include and select Save Data To File (CSV). This dataset will contain the QI flags (a binary coding for each QI) that show if discharges were included or excluded from the specific measures.

This can be particularly helpful if you are interested in exploring how the QIs are calculated and the criteria used for indicator inclusion by discharge.

Please note that MONAHRQ can only load data with fewer than 200 variables. If you are saving data for future use with MONAHRQ, it is recommended that you use the default options as shown. If you choose all checkbox options, the dataset created will have more variables than can be loaded in MONAHRQ.

If you would like to save your data variable and value mappings, select Recognize Columns by Position or Recognize Columns by Column Name. This will create a .qim file for future use with MONAHRQ. It is recommended that you use default options as shown above. When you have selected an option, select the Save Mapping button.

Select Next to continue.

(34)

Screen 15 – Define Hospital Groupings

15. Users may define hospital groupings by Dartmouth Atlas Hospital Service Areas (HSAs), Dartmouth Atlas Health Referral Regions (HRRs), by a single region, or by customized regions. Customized regions may be loaded from a CSV file or you may identify the custom regions by manually mapping hospitals to regions. Begin by selecting a State from the Choose your state dropdown box. Then select the button indicating how you would like to group hospitals into regions.

If you would like to manually define regions, type the name into the Region Name field and select Add Named Region. Repeat this process until all regions have been added. If you would like to remove a region after adding it, select the region and select the left arrow. If you chose to Load Regions from File, refer to Screen 15B for detailed instructions.

If you would like to learn more about Dartmouth Atlas HRRs or HSAs, visit http://www.dartmouthatlas.org/.

Select Next to continue.

(35)

Screen 15B – Define Hospital Groupings; Load Regions From a File

     

15B. If you chose to load your regions from a CSV formatted external file in Screen 15, you will be taken to this screen to load the file.

There are four fields on each line of the CSV file. The first field must be a number that is the Region ID.

The next field is the title of the region and must be enclosed in double-quotes if commas appear in this field. The third field is the two-letter State code. The fourth field must be a Y or N to indicate if the region is selected for reporting. This last value can be changed on the Website Builder Wizard.

A checkbox allows the user to clear existing contents.

Select Browse to find the appropriate file and then select Load File.

Once this step is complete, select Close. 

(36)

Screen 16 – Define Hospital Groupings

16. This screen allows you to alter the hospital assignment to region. If you chose Dartmouth HSAs or HRRs (as in the above screen), the hospital will already be assigned to a region; however, you may reassign to a different region if you would like. If you chose to load custom regions (manually or with a file), you may use the Region dropdown box to assign each hospital to a region. The County Name and Region dropdown boxes are prefilled; all you need to do is select your mapping choice. You may also edit the hospital Name and Zip Code.

You may assign a Centers for Medicare & Medicaid Services (CMS) provider ID manually (or by using the option to load from a hospital file), which will allow you to assign the all-payer (based on HCUP

methodology) cost-to-charge ratios using CMS data from the Medicare Cost Reports. Once the CMS provider ID has been provided, select the Assign Cost to Charge Ratio button. We strongly suggest that you review the assigned cost-to-charge ratios and make any appropriate adjustments – these ratios do not limit the range of acceptable values. A ratio of zero (0) will be treated as missing on the Web pages. Alternatively, you may manually (or by using the option to load from a hospital file) assign custom cost-to-charge ratios. In the Website Wizard, you will select to display costs or charges as available in the Web pages.

If you would like to randomly assign a masked hospital name, select Mask Hospital Names – this option will reassign all hospitals to a blinded or masked name (e.g., Hospital 1, Hospital 2). If you would like to unmask, select Unmask Hospital Names (which will appear once you have chosen to mask). Select Display Hospital List for a crosswalk of the original hospital names and the masked hospital names.

You may want to print this list for future reference or to provide limited access to the MONAHRQ- generated Website.

As an alternative, you may load hospital data from a previously created external file that maps the hospital identifier in the discharge data to hospital demographic data. If possible, load the information from a CSV file that lists the Hospital ID, FIPS county code, hospital name, ZIP Code, cost-to-charge ratio (if desired), region (if desired), and CMS provider ID (if desired). The Dartmouth Atlas HSAs will

automatically assign county names and regions. To do so, select the Load From File button at the

bottom of the screen. A window will pop up as shown on 16B.

(37)

Screen 16B – Define Hospital Groupings; Load From a File

16B. Once the hospital groupings have been defined, the Load Hospital Table screen will appear. This section provides host users the opportunity to apply demographics to each hospital in the data, such as facility names, counties, ZIP Codes, cost-to-charge ratios, CMS provider ID, or regions. Information must be in a CSV-formatted file. Select the Browse button to locate the hospital file to be loaded.

Detailed instructions for the CSV file format are provided on the software screen.

Select options on how to load the file. We recommend always checking the Overwrite existing hospital table entries box. Overwriting is important if you are loading a hospital table for a different dataset where the hospitals may be different.

If you would like to use all hospitals in your dataset, select Load entire file, including those with 0 discharges. If you prefer to only include hospitals with discharges, choose Cleanup hospital table.

Once this step is complete select Load File. You will get a message listing the number of records

loaded. Then choose Close to return to the previous screen, where data will be loaded automatically.

(38)

Screen 16C – Define Hospital Groupings; Return to Edit Hospital Table

16C. Once you have loaded the hospital demographics from a file, you will return to the Edit Hospital Table. This table will now have the information from the loaded file prefilled. You may edit the facility name, ZIP Code, cost-to-charge ratios, and CMS provider ID. We recommend that you review the county and region assignment for accuracy; some users prefer to slightly alter the assignments.

You may assign a CMS provider ID manually, which will allow you to assign the all-payer (based on HCUP methodology) cost-to-charge ratios using CMS data from the Medicare Cost Reports. Once the CMS provider ID has been provided, select the Assign Cost to Charge Ratio button. We strongly suggest that you review the assigned cost-to-charge ratios and make any appropriate adjustments – these ratios do not limit the range of acceptable values. A ratio of zero (0) will be treated as missing on the Website. Alternatively, you may manually (or by using the option to load from a hospital file) assign custom cost-to-charge ratios. In the Website Wizard, you will select to display costs or charges as available in the Web pages.

If you would like to randomly assign a masked hospital name, select Mask Hospital Names – this option will reassign all hospitals to a blinded or masked name (e.g., Hospital 1, Hospital 2). If you would like to unmask, select Unmask Hospital Names (which will appear once you have chosen to mask). Select Display Hospital List for a crosswalk of the original hospital names and the masked hospital names.

You may want to print this list for future reference or to provide limited access to the MONAHRQ- generated Website.

Once you have finished altering this page then select Next to continue.

(39)

Screen 17 – Data Load Completed

17. Once the data load process is complete, a notification screen will appear.

To continue, select Close Window.

(40)

Screen 18 – AHRQ MONAHRQ

18. After you have completed the data load and analysis process, you will return to the MONAHRQ home screen. From this screen, you may choose to generate a Website or reports.

The reports are those found in the AHRQ WinQI software. For more information about the WinQI reports, please visit: http://www.qualityindicators.ahrq.gov/software.htm.

The following screens will show you the process for creating a MONAHRQ Website. Select Website

Builder Wizard on the left hand side of the page to continue.

(41)

Screen 19 – Welcome Screen

19. The Website Wizard Welcome screen briefly outlines the process of building your MONAHRQ Website.

Select Begin to continue.

(42)

Screen 20 – Data Selection Options

20. On the Data Selection Options screen, you will make several selections that will affect the generated Web pages.

First, you should select a State and year for reporting. If you would like to report on discharges that reside (based on the Patient State County Code, PSTCO, data element) in a different State, you should select Other as your State. This selection will allow you to analyze any border crossings that occur within your dataset and report those findings in the Website; if you would like to restrict reporting to one State (based on PSTCO), you should select the appropriate State.

You should enter a phrase to describe the year of data analyzed by MONAHRQ (e.g., in 2006, from June 2006 to May 2007). This phrase will appear throughout the generated Website.

If you would like to suppress small discharge cell sizes or hospital display thresholds, you may enter the threshold number (e.g., enter 15 in the discharge display to suppress any discharge cell sizes with 15 or fewer cases).

You may select the denominator you would like to use in the rates and utilization pathways as 1,000, 10,000, or 100,000. It may be more appropriate to use larger denominators for larger datasets.

If you either loaded custom cost-to-charge ratios or used the embedded CMS-based cost-to-charge ratios, you should select that the ratios are valid. You may then choose to display costs or charges on the generated Web pages.

You may also remove regions from the Web pages by changing the value in the Selected column from Y

to N.

(43)

Screen 21 –Quality Indicators

21. This screen provides you with topics from the AHRQ Public Reporting Template and Quality Indicators. All of the indicators for each topic are preselected. Remove the check from the box for indicators you do not wish to show on your MONAHRQ Website. All items endorsed by the National Quality Forum (NQF) are marked (*). You can learn more about the AHRQ Public Reporting Template and Quality Indicators development by reviewing the memo provided by Shoshanna Sofaer in this user guide.

Select Next to continue.

(44)

Screen 22 – Preventable Costs

22. This screen provides a set of tabs for reporting Potentially Preventable Hospitalization

Information in the Website for each indicator. All of the items are preselected. Remove the check from the box for indicators you do not wish to show on your MONAHRQ Website. All items endorsed by the National Quality Forum (NQF) are marked (*).

Select Next to continue.

(45)

Screen 23 – Build Website – Generate Web Pages

23. The Generate Web Pages screen allows you to choose the look and feel of your MONAHRQ-

generated Website. To begin, select the Browse button to locate the folder to store the Web pages once produced.

Next, check the pages you would like to have included on your Website in the Pages To Generate section. If you generate utilization and rates pages, you may choose to compute the medians by checking the Compute Medians box. Note that the median computing process may take some time to complete (this option may increase processing time by 50%).

MONAHRQ will use default settings for the QI benchmarks. The default setting is to use the input file average to classify hospitals as better than average or worse than average (this classification includes a basic significance test). If you use the default benchmarks, the Area Description you provide on this screen will be used to describe the data. You may choose to use custom benchmarks by selecting the Set QI Benchmarks button. A window will pop up and allow you to set new benchmarks, as shown in 24).

If you want to change the default settings for the font, Web page colors, or page styles, you can choose new options in the Style Sheet Options section by using the dropdown box and selecting the button that corresponds to the desired layout. For examples of the possible layouts, select the Show Style Models button. The fluid style will adjust based on the web site user’s computer settings and type of browser while the fixed style will remain more constant across different computers and browsers.

To change the color, select the Color Chart button for examples of the colors available. Cut and paste

one of the codes, including the # symbol, into the color field box. Alternatively, you may use the color

References

Related documents

o Select input email data file format and First Row detail o Select the raw email address data file you want to import o Enter the name to save this email list as.. • Create a

The scope of this work is to investigate the effects of the natural phenology of exten- sively managed semi-natural grasslands combined with their management practices on

Even though children in the current study continued to make gains in their bilingual vocabulary knowledge and grammatical development, the correlations between total vocabulary

The lower the level of law enforcement, the higher the likelihood of drug flows to a country; yet countries exposed to significant flows of drugs might react with

of the programs. 172 7.8 Average percentage of affected nodes versus affected test cases. 173 7.9 Crosscutting change analysis. 173 7.10 Box-plot of the percentage of faults detected

Each chapter focuses on textual links between speciesism and the oppression of particular human groups based on gender, sexuality, and race, arguing that each novel offers new ways

The sensitivity of these factors with respect to the damping ratio is discussed and from where it can be concluded that friction coefficients have linear relationship with

The Massey litigation was brought by parents and guardians of students who had transferred to better schools while Normandy School District was labeled as unaccredited. 68