O M NG A OS
1PERFORMING A
POST-IMPLEMENTATION SYSTEM
REVIEW
REVIEW
JAN S. HERTZBERG MANAGING DIRECTOR GRANT THORNTON WILLIAM G. KISTNERVICE PRESIDENT & CHIEF AUDIT
V C S N & C U
EXECUTIVE
NORTHWESTERN MEMORIAL HEALTHCARE
HEALTHCARE
AHIA 32ndAnnual Conference – August 25-28, 2013 – Chicago, Illinois
A POST-IMPLEMENTATION SYSTEM REVIEW –
LEARNING OBJECTIVES
LEARNING OBJECTIVES
Describe the benefits of performing a
post-2
Describe the benefits of performing a post
implementation review
Discuss an approach to execute the review
Discuss an approach to execute the review
Identify potential issues that may arise during a
post-implementation review
post implementation review
Content
Background – scope, growth drivers, software delivery
3
g p , g , y
approaches, success factors, costs/benefits
Options for Internal Audit's Involvement in Software
I l i
Implementations
Scope of Software Integration
p
g
Large-scale software integration can include many hospital departments and functions into
4
g g y p p
a system that can serve all those departments’ particular needs. Software may include Enterprise Resource Planning (ERP), Electronic Medical Records (EMR) and other systems spanning:
Core Services - Patient Registration, Emergency, In-Patient, Lab, OT Services, Blood
Bank, Pharmacy, Electronic Medical Records
Support Services – Patient Billing, Appointment Scheduling, Diet & Kitchen, Linen &
Laundry, Bio-Medical Waste, Transportation
Back Office – HRMS, Payroll Management, Accounts/Financial Management,
Growth Drivers
Adopt Best Practices
5
p
Improve Patient Care
Reduce Operational Costs
Software Delivery Approaches
y pp
Less and less of the IT infrastructure and
applications are
6
Cl d C ti
Various vendor solutions between
co-app ca o s a e "owned" by the
customer Variations here include only
portions of a business process that is hosted
Hosted
Fully Outsourced
•Business processes and systems performed by external vendor
•All systems and business t l t f Cloud Computing •Shared infrastructure and services •No ownership of software or hardware
•All processes are defined by vendor
•Support, security and
Many companies use vendor facilities for
backup and
location and full managed services Co-location •Hardware and software at off-site d t t •Referred to as managed services •Offered by many software vendors and third parties
•Processes and controls setup by vendor
•Lower cost alternative to
support personnel part of the vendor
•Fully responsible for support, maintenance, upgrades, etc.
•Standardized system and business processes
•Little or no flexibility
•Referred to as BPO
pp , y program changes by the vendor
•Sometimes referred to as "on demand" or "utility computing" backup and development On Premise •Dedicated data center and infrastructure •Software and data center
•Data center may or may not be owned
•Customer ownership of hardware and software •Identical IT and business processes as on premise full outsourcing •No internal IT staff
•Business support remains at customer
•Referred to as BPO
Large software vendors may offer solutions at every level and niche
players fill in spots across the
Software and hardware located at customer location
•On-site support staff
on premise
•Business users see no changes
•Security process is similar
players fill in spots across the continuum
Success Factors
Defined business and IT strategic plans
7
g p
Approved software project business case
Assigned project sponsors with cross functional team
IT governance structure focused on ongoing project review and
timely issues resolution and decision making
Strong Project Management Office (PMO)
Strong Project Management Office (PMO)
Comprehensive change management, communications and training
approach
Software Implementation Costs
p
Financial costs
8
Business performance will dip during implementation before being boosted by
software
People costs
T i i kill d l
Training - new skills and roles
Turnover - change aversion vs. change acceptance
Manage both software duties and day-to-day operations
P
Process costs
Need to instill a process discipline
Technology costs
A hi d b i l lid f d i f f
Architectures need to be in place - a solid foundation for software
Integration - software systems have strong conceptual links with every major
Software Implementation Costs and
B
fit
Benefits
9rmance
P
e
rf
o
r
Benefits
Software combines the needed functions of the applications that a hospital requires
d i h i i l
10
and integrates them into a single system
Streamline processes and eliminate departmental silos Improve the efficiency and effectiveness of employees Gain control and visibility over the entire business Gain control and visibility over the entire business Make better, more proactive decisions
Data integration
Reduces storage of redundant datag
Avoids re-keying and reformatting data from one system into another Reduces updating and debugging obsolete software code
Improves communication links between systems to automate data transfer
Software allows organizations to balance patient demands impacted by multiple
interrelated items and multiple hospital locations
Options for Internal Audit's Involvement in
Software Implementations
Software Implementations
Steering Committee Participation - provides advice on internal
11
g p p
controls to the project sponsors and brings project management issue to their attention
F ll P j T P i i i f ifi d i
Full Project Team Participation - performs specific duties to
ensure that individuals have performed the required
tasks/prepared adequate documentation/
Periodic Review & Consultation - project team members contact
Internal Audit during any phase of the project for advice on
i l t d ti l t t l
Options for Internal Audit's Involvement in
Software Implementations continued
Software Implementations. continued
Implementation/Conversion Review – reviews training schedule,
12
p / g ,
user/application documentation, testing plan for adequacy, acceptance test results, implementation/conversion plan, backup/recovery plans quality assurance methodology backup/recovery plans, quality assurance methodology
Post Implementation Review – conduct an audit to determine if
system is operating as intended, controls over input/output are / adequate, users are trained, anticipated benefits have been realized, proper Systems Development Lifecycle (SDLC)
methodology was followed methodology was followed
Post Implementation Review Approach
Scoping Questions13
Benefits Realization
• Have the intended objectives of the system been realized
• Have the actual costs and benefits been compared against budget • Was the implementation process effective and appropriate
• What were the causes of time overruns, cost overruns, or quality or performance issues, if any • What were the productivity and performance improvements resulting from the system
Control • Have business processes and internal controls been implemented
Considerations • Have user access controls been implemented in accordance with organizational policy • Have users have been appropriately trained
• Is the system maintainable and can it be further developed effectively and efficiently • Have available features and procedures, as relevant, been implemented
Compliance • Is the implementation compliant with relevant statutory requirements and organizational policies • Is the implementation compliant with management's control objective framework and guidelines, as
relevant
Northwestern Memorial HealthCare
“ We are an academic medical center where the patient comes first ”
•894-bed nationally recognized d i di l t
… We are an academic medical center where the patient comes first.
14
academic medical center hospital
•205-bed community hospital in Lake Forest IL
Lake Forest, IL
•14 ambulatory sites in Chicago and suburbs
•Primary clinical affiliate of Northwestern University’s Feinberg School of Medicine
Feinberg and Galter pavilions Prentice Women’s Hospital
•NMH #6 U.S. News & World Report Honor Roll and NLFH #19 in Chicago
More than 10,000 People Come Together for Our
P ti t th i F
ili d th C
it
Patients, their Families and the Community
7,099
Employees
157,099
Employees
2 537
47,177
Inpatient Admissions$51.8M
Charity Care2,537
Physicians12,415
Deliveries$279.4M
Community Benefit800+
Volunteers83,805
ED Visits$
y591,238
Outpatient Registrations750
Chicago Cares VolunteersSummary of NMHC Installations
16
NMH – Registration/Scheduling – Implementation of
NMH Registration/Scheduling Implementation of
a pre-service interactive registration and scheduling
(“Reg/Sched”) system.
(
g/
) y
Implementation included:
32 departments with over 1,100 users
Converted approximately 71,000 appointments Required training of 736 employees
Summary of NMHC Installations, continued
17
ERP – Replacement of an existing system - included
p
g y
Human Resources, Supply Chain, and Financial Systems.
As part of the implementation, the following were
converted converted:
1,200 vendors
1,800 employees
YTD payroll balances and benefits
7,900 procurement items
500 purchase orders
50 active capital projects
Warning Signs
g g
18
Viewed as an “IT” projectp j
Lack of strong executive sponsorship
Lack of formal and disciplined project management
Lack of proactive change management
Lack of milestones or missed milestones
B d t
Budget overruns
Lack of adequate training
Lack of adequate testing (load and user acceptance)
Lack of adequate testing (load and user acceptance)
Too much customization
Key Issues – Common Themes
y
19
Area of Key Finding Attribute Reg/Sched ERP
Type of Project, per PMO Tracking System IT IT
Type of Project, per PMO Tracking System IT IT
Steering Committee Members and Responsibilities
Specified in Charter No Yes
Advisory Committee Members and Responsibilities
Specified in Charter No Yes
Steering Committee involved business stakeholders PMO Scope
Coordination Steering Committee involved business stakeholders
representing affected entities No Yes
Key issues were reported in PMO tracking system No No Issues were tracked by the project team outside the PMO
ki Yes Yes
Quality Management
tracking system
Schedule Health status reported by PMO tool reflected
slipped dates No N/A
Detailed requirements prepared and compared to the
solution's capabilities during the Selection Process No N/A
y g
solution s capabilities during the Selection Process
High‐level analysis performed as part of Selection Process Yes N/A Customer reference checks performed by business
stakeholders No N/A
Procurement and Vendor
Management
User signoff prior to Integration Testing Yes No
User signoff after Integration Testing No Yes
Key functionality delivered late, i.e. less than month
before go‐live Yes No
Key Roles and Responsibilities
y
p
20
Executive sponsor – Leadership and commitment
Executive sponsor Leadership and commitment
Business area – Owner
IT Support
IT – Support
Supply chain – Governance/oversight
O id l
S
Outside consultants – Support
Internal Audit Role and Value
P
iti
Proposition
21
Have a seat at the table → become an ally to business
owners, IT, senior management and audit committee by providing an independent perspective
p g p p p
Pre-implementation and ongoing monitoring
Proactive focus on accountability
Post-implementation reviews
Questions & Thank You!
Save the Date
S
b 2 2 2
September 21-24, 2014
33
rdAnnual Conference
Austin, Texas
23