I. Program Overview. Organization Name/Program Name: Program Leader Name/ Position/Contact information , Phone

I. Program Overview Organization Name/Program Name:

Gravity Recovery and Interior Laboratory (GRAIL) Project of NASA’s Discovery Program

Program Leader Name/ Position/Contact information – E-mail, Phone

Maria T. Zuber, GRAIL Principal Investigator, MIT, 617-253-6397, JPL Caltech, 818-354-2023, [email protected]

Program Category o System level R&D/SDD program or project

Program Background: What is this program all about? (No more than one page).

Describe:

 The overarching need for this program

 History of the program

 The product that is created by this program

 Scope of work – original & updated

 Expected deliverables

 Current status of the program

From its inception in early 2006, the GRAIL Project was developed to map the structure of the lunar interior from crust to core to reveal the internal constitution and evolution of the Moon. This objective was accomplished by producing maps of the lunar gravity field that are enabling scientists to explore a planetary interior at unprecedented resolution. Technologically, GRAIL was the first dual spacecraft mission to execute

precision formation flying around another planetary body, measuring distance changes between the spacecraft to tenths of a micron per second. The GRAIL mission Principal Investigator is Dr. Maria Zuber from the Massachusetts Institute of

Technology. The GRAIL project is managed by the Jet Propulsion Laboratory (JPL) with Lockheed Martin (LM) contracted to provide the spacecraft. The science instrument was developed by JPL. After its successful launch on

September 10, 2011, twin GRAIL orbiters were placed into a polar orbit on December 31, 2011 and January 1, 2012. After a succession of 19 maneuvers the two orbiters settled into a precision formation to begin science operations in March 1, 2012 with an average altitude of 55 km. The primary mission was completed in June 2012. On the basis of a competitive proposal, GRAIL was approved for an Extended Mission, which was completed in Dec. 2012. Each GRAIL orbiter contained a Lunar Gravity Ranging System (LGRS) instrument that conducted dual-one-way ranging measurement to precisely measure the relative motion between them which in turn was used to develop the lunar gravity field map. The instrument was a modified version of the instrument used on the Gravity

Recovery and Climate Experiment (GRACE), which is currently mapping Earth’s gravity. GRAIL’s twin spacecraft had heritage derived from an experimental U.S. Air Force satellite (called XSS-11) and the Mars Reconnaissance Orbiter (MRO) mission, both developed by LM. Judicious use of heritage from these successful missions allowed GRAIL to provide breakthrough science at a reasonable cost and risk. GRAIL also included the signature education/outreach effort

MoonKAM (Moon Knowledge Acquired by Middle school students), led by the late Dr. Sally K. Ride, the first American woman to fly in space. MoonKAM was the first planetary imaging experiment dedicated solely to education, and allowed students to target images of the Moon using project-provided graphical software. Commanding was accomplished by supervised undergraduates at the University of California, San Diego. During its Extended Mission phase, the GRAIL

spacecraft mapped the Moon from the eye-opening average altitude of 22 km, and during the mission’s endgame the average altitude was further decreased to 11 km.

I. VALUE CREATION =20 POINTS Value:

What is the value, competitive positioning, advantage, and return created by this program to your:

• Customers – National interests, war fighter • Company – Strength,

bottom line, and shareholders • Scientific/technical

value (particularly for R&D programs) Excellence and Uniqueness: What makes this program unique? Why should this program be awarded the Program Excellence Award?

GRAIL completed it mission on time, on spec and with a healthy budget reserve and is now in its data analysis phase. The customer for GRAIL is NASA, the scientific community and the public, all of who have so far benefitted from GRAIL’s prudent technical and cost management. GRAIL achieved its minimum success measurement requirements, a year ahead of schedule. The mission produced the highest-resolution and highest-accuracy gravity map for any terrestrial planet, including Earth. GRAIL’s gravity model of the Moon has improved our knowledge over previous models by as much as six orders of magnitude. This data is being used by the GRAIL science team to understand the role of impact cratering,

volcanism and tectonics on lunar evolution. The mission also revealed the deep interior structure of the Moon, and this data will be used to understand the mechanism of deep moonquakes and to determine evidence for a molten outer core. The gravity field will also enable precision targeting and landing for future robotic and human exploration. In its Extended Mission GRAIL mapped the shallow crust of the Moon to unprecedented detail, and the data is being analyzed to determine the existence of subsurface reservoirs of ice at the lunar poles and subterranean lava tubes that could shield future astronauts from solar storms. GRAIL’s MoonKAM investigation received wide recognition for its innovative approach to encouraging STEM (Science, Technology, Engineering, and Math) education. At the end of the mission over 200,000 student images were acquired, transmitted to Earth, and downloaded to American classrooms for scientific analyses guided by curriculum materials

developed by Sally Ride Science. The experiment engaged over 100,000 students in nearly 3000 classrooms nationwide.

III.ORGANIZATIONAL PROCESSES/BEST PRACTICES:(HOW DO YOU DO THINGS)=30 POINTS

Describe how your program has identified its operational and business opportunity, and manages this opportunity throughout the program’s life cycle.

competitively selected the GRAIL mission under the Discovery Program for solar system exploration in December 2007. The Step-1 proposal contained the Level-1 requirements and these did not change through the course of the mission. This stability in the top level requirements throughout the life cycle of

GRAIL was a significant factor in the mission launching on time and under budget. This was achieved by the management expertise of the GRAIL PI who is responsible for all aspects of mission success. As a “hands-on” PI, she participated

prominently in all aspects of project development, including risk management, reviews and financial decision-making.

Strategic:

Strategic Supply Chain Integration and Cost

Effectiveness Management: - Describe how your program is integrating its supply chain to assure visibility and adapting long-term cost effectiveness up and down the supply chain.

Contract surveillance was conducted using a combination of insight and oversight techniques, founded on a philosophy that JPL involvement must be value-added and reflective of the project’s risk-management program. During the formulation phase, JPL and Lockheed Martin (LM) management agreed that each organization would use its proven spaceflight development practices. In other words, JPL would not force LM to do things the JPL way, nor would LM pressure JPL to work to LM procedures. The surveillance approach centered on the following: (1) Document Submittals—per JPL’s Standard Subcontract Data Requirements, with some items negotiated to allow LM equivalents and/or to be delivered in place; (2) Reviews—Monthly Management Reviews (MMRs) with mutual participation in each other’s many peer reviews; and (3) People—colleague-to-colleague telecons; JPL representatives on site at LM during key activities. People saw themselves as “the GRAIL team” and operated in a quasi-badgeless manner.

Strategic:

Operational Integration and Systems Engineering – Describe the challenges faced by your program in terms of integrating the system into its operational environment and its impact on systems engineering planning and management.

GRAIL has completed its data collection phase and is now in its data analysis phase. After traveling for nearly four months on a low-energy trajectory to the Moon, the twin spacecraft were inserted into lunar orbit on New Year’s Eve and New Year’s Day 2012. In Jan. 2012 a series of circularization maneuvers brought the orbiters into co-planar near-circular polar orbits. In Feb. 2012, a distant rendezvous (~75 km) was achieved and the science instruments were turned on and science mapping commenced a week earlier than planned. Mission management, mission planning and sequencing, and navigation were conducted at JPL. Lockheed Martin, the flight system manufacturer, operated the orbiters from their control center in Denver, Colorado. The orbiters together performed 78 propulsive

maneuvers to complete their science phase mapping. Execution of these maneuvers, as well as the payload

checkout and calibration activities, proceeded optimally due to extensive pre-launch operations planning and testing. During the course of the 465 day mission, over 2800 command files were radiated to the spacecraft with only 3 minor command errors for an error rate of less than 0.1%. During the mission, there were no safe-mode entries by either spacecraft. The key to the operations success of GRAIL was, number one, the people, and because detailed timelines for product interchange were used between the operations teams. Further benefits derived from the use of proven

procedures from previous JPL/LM planetary missions.

Operational:

Planning, Monitoring, and Controlling -

Describe your planning and resource allocation processes. How do you monitor and review your program’s

progress and make corrections to keep the program on track?

To plan, monitor, and control the project, GRAIL used a process called the Technical, Schedule and Cost (TSC) Control Board, chaired by the PI, which reviewed and approved the original project baseline, and then decided all top-level

technical and schedule changes, and reviewed and approved all requests for use of project cost reserves. Only the PI could approve release of cost reserves, a practice that had a powerful effect on encouraging team leads to solve their own problems and bring forward only requests that were truly needed, well justified, and reasonable in amount relative to the scope of the problem. Not all requests were approved. On the other hand, the PI would make periodic calls for staff to propose ideas for risk-reduction activities, and value-added investments were usually supported. This resulted in the Project returning to date to the Government $11M of unused funds.

Operational:

Supply Chain and Logistics Management -- What processes, tools and

relationship-building methods have you used to develop, refine and improve supply chain and stakeholder integration? Please indicate also methods used to analyze/fact-find regarding supplier proposals. This is one of the most imperative needs of our industry – please provide specific details and data that assisted you in gauging the effectiveness.

For GRAIL the principal surveillance tools changed in importance over time, but the most important tools were the personal interactions. A “24-hour rule” allowed LM a short period of time to troubleshoot an issue prior to notifying JPL. LM management could get involved and the issue could be triaged without pressure from the customer, but LM was obligated to provide a candid report to JPL the very next day, even if the concern was not yet under control. Tools

emphasized over time were: (1) Early—completion of trade studies, drawings, procurement specs (tangible indicators of progress); (2) Middle—delivery of hardware elements and software modules (direct impact on project critical path); and (3) Late—weekly Assembly, Test and Launch Operations (ATLO) schedule updates and at least daily status telecons (keeping everyone current in a very dynamic environment).

Operational:

System Integration, Testing & Reviews -

Describe the activities and

GRAIL entered ATLO with 40 days of margin for LM Colorado activities (flight system assembly, integration, functional test, environmental test, and transportation to the

your system integration, and testing. How did you conduct system design and technical reviews?

launch processing facility) and 25 days of margin for Florida activities (Launch System assembly and integration, launch-vehicle-to-spacecraft integration, fueling, and launch). These planned margin days excluded second-shifts, weekends, and holidays. In practice, work-to-ship and work-to-launch were tight, with judicious application of cost reserves to support second-shift and overtime augmentations. But when the launch provider requested extra processing time because GRAIL was the last east coast Delta rocket launch, the spacecraft team reworked the development schedule to deliver both spacecraft to the launch site a week early.

On GRAIL reviews were centrally managed by the project office with an innovative approach using a project Review Captain to supervise the preparation effort for each of the project-managed reviews. The preparation activity included coordination of pre-reviews (some required by the institution, others self-initiated), interaction with GRAIL’s standing review board, outline reviews (invented by GRAIL), dry runs (review practice sessions), and support services for multiple sites. (Some reviews were held by videocon, others by a call-in capability using the Meeting Place application.) In parallel, much attention was paid to commissioning and managing NASA- and JPL-required gate products and control plans, including negotiating, assigning, tracking, reviewing, obtaining signatures, and submitting.

Operational: Risk / Opportunity Management

Describe the processes used to identify both risks and opportunity and to assure potential for both is addressed effectively Please indicate any forward-leaning processes to support.

As a single-string, dual-spacecraft mission, GRAIL’s guiding philosophy was low-risk implementation, so proactive risk management was integral to project success. Monthly Risk Board meetings included the PI, Project Manager (PM), key personnel, and relevant support staff. Before life cycle reviews, the meetings were extended to walk through every risk. At least twice during project development, special risk identification sessions were convened by the Project Systems Engineer. Culturally, anyone on the project could recommend a risk, and most were proactive in this regard. Risk identification was continually encouraged by all members of the GRAIL leadership team. The project Risk List was tied to the project Liens List, discussed above. An important point to note is that GRAIL used risk management as a value-added technique for managing the work to go, rather than for doing the minimum to meet a NASA Agency reporting requirement.

Team Leadership:

Team Culture and Motivation Describe how you created your team spirit and culture, and accomplished entire team

From the early days of GRAIL, staffing was designed with the following outcomes in mind: a) designate a larger-than-the-norm number of staff as proposed key personnel, all of whom brought substantial flight project experience to the team;

member motivation. Given the economic environment and changes in the global marketplace, how did you assure your team changed swiftly and with agility?

b) include several veterans of the heritage projects, GRACE, XSS-11 and MRO; and c) retain all key personnel until their areas of responsibility were completed. Key personnel were supported by qualified staff at JPL and LM, augmented by contractor support staff when the need arose. Under the heading of “Team Competency,” one participant noted: “Have never seen a program with as high a percentage of talent-level at all levels of the organization—huge effect on small team

efficiency.” The shared sense of purpose and the importance of the mission engendered strong espirit de corps among the GRAIL team. When inevitable technical hurdles arose the team re-iterated its mantra: “We’re going to the Moon”!

Team Leadership: Lessons Learned and Knowledge Management Describe how you collect lessons learned and best practices, and how they are shared with your team and company to improve

performance. Also how are you capturing expertise and

knowledge to assure

availability over the life of the program?

GRAIL used JPL’s institutional processes for lessons learned and knowledge management. These were recorded in a database system for meeting minutes, design reviews, design analyses, etc. The lessons learned were shared with the

institution as a whole via briefings and memoranda throughout the life of the project, and especially at decision gates such as preliminary design reviews, critical design reviews, etc. The results of these were then folded in documents such as JPLs Flight Project Practices and Design Principles that all projects at JPL/LM are expected to follow, thus ensuring that lessons learned are actually learned (i.e. closed loop system). In addition, GRAIL’s lessons learned were folded into JPL’s proposal process and used to help win the InSight project, currently in development to explore Mars.

Team Leadership: Leadership Development How do you develop team’s skills and build future leaders

Both JPL and LM have institutional mentoring processes that are used to proactively seek out and build the skills for future leaders. Once identified, these individuals’ careers, education, training, special assignments, etc. are tracked, monitored and nurtured to ensure the best candidates become future leaders.

Best (& Next) Practices: Identify your program’s specific Best Practices that you believe are unique, and could be shared with others and become industry’s Next Practices.

Every project is unique, so what worked for GRAIL is not a precise recipe for other space flight projects. However, it is recommended that other projects being conceived consider the GRAIL experience and select and tailor those items most applicable to a new mission concept. Lessons to consider include:

(1) Architect a sound concept from conception. Propose what you are going to launch, and launch what you proposed. Limit technology development to what is absolutely needed to achieve the mission objectives; have limited and clean

interfaces; and use capabilities-based requirements to gain large technical resource margins. Support the development activity with healthy cost and schedule reserves.

Any change has a flow-down effect, which gets only greater the further down in the project architecture that it needs to be accommodated).

(3) Conduct a comprehensive formulation (pre-commitment) effort. Perform early prototyping of any open technology developments (or significant adaptations). Conduct penetrating inheritance reviews. Make it your goal to have no liens going into implementation.

(4) Be agile in implementation (final design and development). Aggressively identify technical problems, and respond to them quickly. Practice proactive risk management; empower all members of the team to identify risks and work them to resolution.

(5) Recruit an excellent team, both key personnel and support personnel. Entice them to remain on the project until their work is done. Consciously incorporate teambuilding events; a

“badgeless” team will join together to overcome the tough times that are inherent in any project.

(6) Practice focused project management - not hands-off, but not micromanaging. Begin with a realistic schedule. Use life cycle reviews as control milestones for the entire team; everyone has to deliver products compatible with their colleagues’, and at the right time. Focus on “closure,” (e.g., complete trade studies in Phase A, complete open paper in Phase D).

(7) Watch subcontracts. Place them early. Monitor

performance using early warning indicators. Offer help and encouragement as needed. Order electronic parts early.

IV. ADAPTING TO COMPLEXITY:(HOW DO YOU DEAL WITH YOUR PROGRAM’S UNIQUE COMPLEXITIES)= 20 POINTS

Identify the Program’s Market Uncertainty level – How new is your product to your market and users, based on the definitions below. Then describe how you deal and address this specific uncertainty:

GRAIL is a NASA-funded not-for-profit mission to explore the Moon and as such the project team had to deal with all the political uncertainty and turbulence existing in the government today. To combat this market uncertainty, the project team developed a system concept “Derivative” in order to achieve the strict cost-cap nature of NASA’s Discovery Program. The fact that GRAIL to date has returned $11 million to the government in the course of the life of the project while meeting or exceeding all mission objectives speaks well to how well the team performed.

Identify the Program’s Technological Uncertainty using the definitions below. Then describe how you deal and address this uncertainty:

- Low-tech: application of

mature, well-established technology

GRAIL was a medium technology project. As discussed earlier, the spacecraft was based on the MRO and XSS-11 missions. The process used to reduce the uncertainty in development was

described in an excellent article by Frank Morring, Jr. in the Aug. 29, 2011 issue of Aviation Week which says, “...Although the twin Grail orbiters have a lot of heritage from other spacecraft, managers stress that the benefit is as much in the people who

- Medium Technology:

existing technology modified to meet new design requirements

- High-Technology:

recently developed new technology

- Super High-Technology:

non-existing technology that needs to be developed during the program.

worked on the project as in the hardware itself…” The same approach to reducing technology uncertainty was also applied to the LGRS instrument which had heritage from the GRACE project. Another key factor was that GRAIL had an excellent birth in that during Phase A the project completed the following activities: conceptual design, management plan, Phase B work plan, network schedule, and parametric cost estimate.

Additionally, JPL established an instrument testbed for early system analysis using GRACE residual hardware and GRAIL prototype electronics. By the time of the NASA site visit in August 2007, prior to mission selection, a working prototype of the GRAIL instrument was demonstrated that met the science measurement requirements. Thus the project retired the mission’s greatest technical risk – making the science measurement with the required accuracy – during Phase A.

Identify the level of your System Complexity using the definitions below. Then explain how you are dealing with this level of complexity: - An Assembly performing a

single function.

- A Sub-system fitting within a larger system. - A System – a collection of subsystems performing multiple functions. - An Array – a “System of Systems”; a widely dispersed collection of systems serving a common mission.

The GRAIL mission in the context described here is an Array, or a system of systems that are widely dispersed including the twin spacecraft utilizing NASA’s Earth-based Deep Space Network (DSN). To deal with this level of complexity, GRAIL used a higher-than normal amount of system engineering expertise because experience has shown that the quality of Systems

Engineering (SE) is directly related to mission success. SE was a major attribute of the GRAIL management plan because it

provided early identification and management of problems that could lead to technical risk and cost growth. On GRAIL SE includes spacecraft and payload hardware and software as well as mission design, ground data system and mission operations system development and launch vehicle development. SE was used for requirements management, technical resource and margin management, as well as project-wide verification and validation.

Identify the Pace and Urgency of your team’s effort using the definitions below. Then describe how you deal with the program’s pace requirements:

- Regular timing – no

specific time pressures. Fast/Competitive – time to market is important for competitiveness.

- Time Critical – there is an

absolute and critical-to-success deadline. - Blitz – there is a crisis

element driving the need for immediate response

GRAIL was a Time Critical development project because of its fixed planetary launch period. A critical success factor in achieving the launch of GRAIL on time was LM’s and JPL’s extensive experience in ATLO, especially the fact the LM

Program Manager, had an ATLO background. The JPL/LM team knew from the school of hard knocks what items required special attention, how long tests and other actions actually took to perform, where margin would be best placed, and what flexibilities existed in the ATLO flow. Per plan, there were sufficient spares to keep making forward progress, and hardware could be swapped between the two GRAIL spacecraft or from flight spare or testbed status to test out the flight and ground software. Full-functionality ATLO test units (ATUs) were used temporarily while flight units were completing unit test; then

penalty (regression) testing was performed at the spacecraft level. Workmanship tests were performed on both spacecraft, whereas performance and stress tests were, in some cases, divided between the two. The project fully completed its proposed no-excuses Incompressible Test List tests, as well as the highest-priority Risk-Reduction Tests.

Other Complexities & Uncertainties -

Describe other complexities and unknown factors faced by this program and how you addressed them.

The GRAIL project was subjected to 8 audits during its

implementation phase. Auditors were from a number of federal agencies and organizations. While the GRAIL team concurred with the importance of compliance and realized that each audit had a legitimate purpose, the sum of these audits and their timing had the effect of taking the project off-line during these audits. It was not uncommon for different auditors to ask the same

questions and request the same data, often during critical mission stages. As a byproduct of GRAIL’s technical, schedule, and cost success, the audit organization sometimes specifically sought out GRAIL or was steered to GRAIL by NASA. The Review Captain coordinated the response to all the audits. This individual was instrumental in making the audit/review process for GRAIL very efficient.

V. METRICS (HOW DO YOU MEASURE PROGRAM’S PERFORMANCE)=30 POINTS

(Note: We are not looking for $ results, but the relative percentage achieved. In particular indicate what specific metrics and data you are using that drive the program beyond standard measures of schedule, budget, and performance, and which have contributed to your program’s focus and its success.)

Customer - How do you measure the impact of your program on your customer and your customer’s satisfaction? Include a description of your metrics, as well as numerical evidence.

We were in daily contact with our NASA customers to gauge their satisfaction with the project. They had a standing invitation to all project meetings so that they could be fully aware of GRAIL’s status. NASA was also the decision authority to transition from phase to phase and to determine launch readiness. A more formal measure of GRAIL’s impact was a recent GAO audit entitled “NASA: Assessments of Selected Large-Scale Projects, GAO-12-207SP, March 1, 2012, see: September 10, 2011, both on schedule and within cost…” Another measure of customer satisfaction was during a recent NASA honors award ceremony in which 22 individual awards were provided to GRAIL team members and 29 GRAIL group achievement awards were bestowed upon the team. Numerous other team and

individual honors have been awarded to the GRAIL team members for mission-related achievements.

Performance - How do you measure your

program’s performance in traditional terms such as schedule, budget, requirements, and business results?

Monthly Management Reviews (MMRs) were used as a measure the project’s performance in terms such as schedule, budget requirements, risks, etc. The MMR purpose and guidelines were established as part of the Technical, Schedule and Cost (TSC) Control Plan. It was an informational meeting and a project-wide review of all system and subsystem progress vs. plans,

schedule/cost status, issues and risks. The information from the MMR was used as input to subsequent meetings (design teams, trade studies, risk board, TSC board, etc.). Based on this process and the hard work of team members, GRAIL met all its Level 1 science requirements and completed its mission on schedule and under budget.

Preparing the Future - How do you measure and assess the long-term contribution of your program to the

corporation/organization?

The JPL institution has recognized GRAIL’s contributions to scientific, schedule, and cost performance and has collected lessons learned to be applied to future projects. These lessons were

incorporated into JPL’s proposal process and were used during the last Discovery proposal submission. This helped lead to JPL winning the InSight mission in the Discovery-12 competition, a mission to study the interior of Mars. Future scientific

contributions will be measured by publications and citations in the scientific and technical literature. Education impacts will be assessed from monitoring educational development and career choices and trajectories of students in the MoonKAM program.

Team - How do you measure and assess the impact of your program on your team development and employee

satisfaction?

The fact that the majority of team members who started with GRAIL remained on the mission is arguably the most tangible measure of job satisfaction. A quantitative measure comes from surveys in which team members admitted to being highly satisfied with their experience on GRAIL. Many members of the team are currently working together on the next Discovery Program project called InSight.

Unique Metrics - Describe any unique metrics you are using to measure your program’s progress and how do you focus it for outstanding success.

Safety and Mission Assurance (SMA) held a full-partner role on GRAIL. In addition to the traditional responsibilities for

requirements compliance and oversight, the SMA team was heavily involved in independent assessment/implementation support for the critical technical issues and anomalies experienced during

development and operations and was a key factor in the success of the mission. During the course of the mission over 99.99% of the science data that could have been collected was actually collected. This is an unprecedented data collection rate and is a unique metric for the data needed for GRAIL’s ultimate customer, the science community, and speaks to the application of GRAIL’s several innovations to improve overall project performance. Finally, the resolution errors in GRAIL’s gravity models are currently a factor of five better than the mission requirements and the errors in the gravity model are as much as a factor of 1000 better than proposed. Further improvements in both resolution and quality are anticipated as data analysis continues.