1.Readiness A ssessm ent 2. E V M S O n- S ite Rev ie w C A R Issuance 3. Follo w U p Rev iew 4. C ertification C AP P roposal C A P A cceptance 5. S urv eillance Com plaint or Finding Im ple m entation R ep orted C A P Im plem entation
Figure F-1, EVMS Schedule Milestones Graphic Representation
The most complex area is the detailed planning and execution of the on-site review. This area has not yet been addressed to any level of detail in this Guide and is addressed below.
6. Detailed Planning and Execution of Onsite Reviews: This discussion focuses on the following:
• Organization and composition of a typical EVMS certification review team. • Functioning of the team immediately prior to, during, and following the on-site
review.
DOE G 413.3-10 Appendix F
DRAFT XX-XX-08 F-3
• EVMS Certification Review Team: A typical EVMS certification review team is shown in Table F-1.
Table F-1; Typical EVMS Certification Review Team OECM Over-Sight: OECM EVMS Lead
Review Director: Independent EVMS Review Contractor Review Chief: Technical Consultant
Functional Team Interview Team
No. Areas Members 1 2 3 Acctg
EVMS Contractor X Other X Organization Other X A Analysis & Management Reports Same as above EVMS Contractor X Other X B Planning & Budgeting Other X
Acctg Oversight Org X
Acctg Oversight Org X
C Accounting
Acctg Oversight Org X
EVMS Contractor X
Other X D Revisions
Other X
The functional areas shown Table F-1 represent the five process groupings into which the 32 ANSI/EIS Standard 748-A Guidelines are categorized. Functional area leads are designated by the bold text, and all but accounting come from the independent review contractor organization. Each of the functional team members is also assigned to an interview team, wherein the interview teams are constructed to have representation from all but the accounting team. The accounting team also conducts interviews, and although it is assigned to a few of the interviews conducted by the other three interview teams, in large part it conducts its interviews of accounting personnel separate from the other three interview teams. In addition to the interviews of the contractor personnel conducted by the three indicated interview teams and accounting team, the OECM EVMS lead, review director, and review chief conduct interviews of contractor senior management personnel.
Appendix F DOE G 413.3-10
F-4 DRAFT XX-XX-08
• Functioning of the Review Team: In general, the review team plans and executes its work prior to, during, and following the on-site review, as follows: - Prior to arriving on-site, team members review contractor EVMS
documentation that is supplied by the review director to assess compliance of the EVMS processes and procedures with ANSI/EIS Standard 748-A.
- During the on-site review, review team members conduct interviews and data traces to determine if personnel are doing business consistent with their EVMS procedures. Also, at the end of each day the entire team assembles to discuss potential problem issues. Further, the OECM EVMS lead, review director, and review chief meet at the end of each day of interviews with the contractor in order to ensure that there are no misunderstandings in what the review team believes it is observing and to ensure there are no surprises at the end of the week when the review director presents CARs to the contractor. - Following the on-site review and acceptance of a contractor CAP, the review
chief and necessary functional leads respond to direction from the review director to assess the acceptability of the contractor CAP, and to support further reviews to ensure CAP implementation.
During all of the above activities, the OECM EVMS lead provides DOE oversight. Technical accountability (e.g., determining which issues represent non-compliances with ANSI/EIS Standard 748-A) rests with the review director. Although final authority rests with this position, in most cases consensus is obtained from the entire review team as to which issues represent non-compliances.
• Staffing of the Review Team and Roles and Responsibilities: To ensure consistent interpretation of the ANSI guidelines, the OECM EVMS lead, the review director, and the review chief are generally the same people.
The review director and the functional leads for all areas except accounting are from the independent contractor organization. The technical consultant to OECM functions as the review chief.
The accounting team is comprised primarily (and most times exclusively) of personnel from the organization that has accounting over-sight responsibility for the contractor being reviewed. This is usually DCAA, or a DOE field services office (e.g.,
Albuquerque Service Center).
To the extent that it is possible, the remaining positions should be filled by qualified personnel to include contracted personnel and DOE personnel but on an exception basis may be filled by other personnel. For example, these other personnel sometimes are contractor personnel who support other projects that are within the purview of the same PSO/NNSA office that is being reviewed (but are not assigned to projects that are the subject of the planned review). Potentially beneficial results, and potentially conflicting
DOE G 413.3-10 Appendix F
DRAFT XX-XX-08 F-5
results, can result from the determination of the review team members, and the review director has the authority to accept or not accept the proposed review team members. In cases of disputes, OECM may need to resolve the issue with the affected PSO/NNSA Office.
Staffing of the review team should be identified by roughly one month prior to the start of the on-site review and—consistent with the above guidelines—is the responsibility of the following organizations/personnel:
• The independent review contractor (for the positions of review director, and the three functional leads for functional Teams A, B, and D shown in Table F-1). • The technical consultant (for the position of review chief).
• The review director (to establish contact with, and obtain the commitment of, the required Accounting personnel), and
• The cognizant DOE PSO/NNSA EVMS point of contact for the balance of the positions, when appropriate.
To summarize plus expand on the above discussions, Table F-2 describes the EVMS review team member roles and responsibilities.
Appendix F
DOE G 413.3-10
F-6 DRAF
T
XX-XX-08
Table F-2; EVMS Certification Review Team Member Roles and Responsibilities Organization/
Team Position Review Team Roles and Responsibilities
OECM/ EVMS Lead
1. Provides OECM over-sight to ensure effective planning and execution of review.
2. Ensures required contracts and contract funding are in place to provide necessary contractor support. 3. Works with review director and PSO/NNSA points of contact to ensure all review team positions staffed.
Independent Contractor/ Review Director
1. Establishes final review agenda with contractor point of contact, and makes review team member assignments. 2. Obtains contractor EVMS documentation and provides to review team members roughly 30 days prior to review. 3. Makes final determination as to which issues represent findings (i.e, CARs), and at end of review week provides
contractors with completed CARs and CIOs.
4. Interfaces with contractor to obtain, review/approve, and verify implementation of an acceptable CAP.
Technical
Consultant/
Review Chief
1. In preparation for the review, providing the review director a written summary of the planning /readiness assessment meeting, to include areas warranting further review and possible improvement, and number of interviews required (which impacts necessary number of review team members).
2. Assists all team members in their performance of their jobs (e.g., writing CARs, CIOs, etc).
3. Assisting the review director as may be directed (e.g., drafting more complex CARs, and collecting and reviewing /editing CARs and CIOs; also, assessing CAP acceptability and implementation).
Independent Contractor/ Functional Leads
1. Provide direction and guidance to review team members regarding conduct of interviews, writing of CARs and CIOs, and writing of guideline writeups and functional area summaries.
2. Review/approval of all documents relating to their functional area.
3. In also functioning as interview team leads, working with their assigned interview team members to plan and direct the conduct of the interviews.
DOE/Other Support Team
Members, As Appropriate
1. During and following review, assisting functional leads as directed by them; e.g., summarizing daily issues, and drafting CARs, CIOs, Guideline Writeups, and Functional Summaries.
All Team Members
DOE G 413.3-10 Appendix G
DRAFT XX-XX-08 G-1 (and G-2)