The conclusion of the two-part story regarding the Federal Aviation Administration’s testing of the En Route Automation Modernization system (ERAM) addresses reports to governmental commissions assessing the program and the effect its problems and delays have on the entire NextGen program.
On April 21, 2010, U.S. Department of Transportation Inspector General Calvin Scovel, III presented a report entitled, “Challenges in Meeting FAA’s Long-Term Goals for the Next Generation Air Transportation System,” to the Committee on Transportation and Infrastructure Subcommittee on Aviation for the House of Representatives.
Scovel warned about a number of reported problems regarding key NextGen technologies: “Central to this effort is the successful implementation of ongoing modernization projects that will provide platforms for new NextGen capabilities for enhancing capacity. These platforms include the $2.1 billion En Route Automation Modernization (ERAM) program.
“Delays with this and other projects will have a cascading effect on NextGen plans now and well into the future. One critical step to avoid risks with NextGen’s cost, schedule and capabilities is addressing gaps in partner agencies’ research and development efforts and long-term budgets and plans. We identified actions that can be taken now to strengthen the multi-agency approach, better leverage federal research projects, and prevent duplicative efforts,” he reported.
Scovel’s report detailed the background for the NextGen program, stating that “in 2003, Congress mandated that the FAA establish the Joint Planning and Development Office (JPDO) and that it create and carry out a plan for implementing NextGen by 2025. The JPDO was also required to coordinate diverse research efforts of other federal agencies, including the Departments of Defense, Commerce, Homeland Security, and the National Aeronautics and Space Administration (NASA). While the initial planning for NextGen focused on implementing improvements through 2025, FAA has recently refocused and emphasized improvements that can be implemented in the near and midterm, defined as 2012 and 2018.”
Scovel said, “While the FAA does not believe the system to be fundamentally flawed, it has postponed the in-service and operational readiness decisions for ERAM at Salt Lake City by six months, both originally planned for December 2009. In a footnote to this comment, it states that ‘an in-service decision (ISD) authorizes deployment of a system into the operational environment. It occurs after demonstration of initial operational capability at the key test site.
“The decision establishes the foundation for operational readiness to be declared at key sites and subsequent sites following completion of joint acceptance and inspection by the operating service organization and certification of compliance with information security requirements.
“The in-service decision is based on testing to verify performances and operational readiness. For ERAM, the Operational Readiness Decision (ORD) is the final operational readiness certification that is required for the system to become operational and no longer require retention of the HOST Computer system as a back-up.”
Scovel continued, “We have not assessed the severity of the problems with ERAM, but FAA officials are concerned about the ERAM transition at larger, more complex sites like Chicago and New York. These locations have unique airspace and operational issues that will require adaptation of the system’s software to accommodate their needs.
“FAA officials acknowledge that it is unlikely that all 20 systems will be fielded nationwide and controlling traffic on a regular basis by December 2010 as planned. FAA must take steps to ensure that problems with ERAM are resolved and make realistic adjustments to the program’s schedule. FAA must also assess what trade-offs in capabilities and adjustments to deployment plans and budgets are needed. Prolonged problems with ERAM will directly impact the implementation of NextGen efforts now and in the future, including key NextGen systems such as Automatic Dependent Surveillance-Broadcast (ADS-B) and Data Communications.”
In a footnote, Scovel explained that “ADS-B offers surveillance, like radar, but with more precision. It provides air traffic controllers and pilots with more accurate information to help keep aircraft safely separated in the sky and on runways. Data Communications will provide comprehensive data connectivity, including ground automation message generation and receipt, message routing and transmission, and aircraft avionics requirements. It is expected to automate repetitive tasks, supplement voice communications, and enable ground systems to use real-time aircraft data to improve traffic management efficiency,” Scovel reported.
In a Jan. 15, 2011 FierceGovernmentIT (www.fiercegovernmentIT.com) article, writer David Perera reported “En Route Automation Modernization (ERAM), a radar-based tracking system of high altitude flights that’s a key part of the FAA’s air traffic control modernization effort, won’t reach nationwide operational readiness until August 2014 and will cost $330 million more than expected, an FAA spokeswoman said. Asking not to be attributed, the spokeswoman said in an interview that the FAA has accepted the conclusions of an Oct. 15, 2010 MITRE study that call a May 2010 internal FAA plan to delay final ERAM rollout until January 2012 ‘unrealistic.’
“FierceGovernmentIT obtained the MITRE report through a Freedom of Information Act request. [MITRE Corporation and Massachusetts Institute of Technology/Lincoln Laboratory Report, Independent Assessment of the ERAM Program, Oct. 15, 2010].
“The spokeswoman said the agency will follow MITRE’s recommendations, which call for staggering a determination of ERAM operational readiness at the 20 continental U.S. air route traffic control centers in three waves, with three-month ‘risk gaps’ between them. ‘Air traffic controllers are already using ERAM to handle traffic 24 hours a day in Salt Lake City and Seattle and testing at other facilities continues to go well,’ the spokeswoman added.”
Perera reported that, “Report authors blame delays in the project on deficiencies in project oversight, stakeholder coordination, and resource commitments, and from insufficient discovery of software problems in ERAM’s operational test environment, resulting in releases of software at key sites that have not been ready for sustained operational use. According to the MITRE report, FAA officials realized that ERAM would be late and presented to FAA Administrator Randy Babbitt in May 2010 a ‘corrective action plan’ that called for final ERAM operational readiness in January 2012.”
“Report authors heap scorn on the plan presented to Babbitt, however. ‘With few details and no supporting data or rationale for the proposed actions and revised schedule and budget, it could not be considered a comprehensive plan,’ the report states.”
Perera concluded his story by writing, “The report also faults the FAA for not having followed best practices for information technology contracting, such as having settled on the scope and cost ceiling of work called for in a contract line item number before embarking on that work.”
In the MITRE/Lincoln Laboratory study, which is available online at FierceGovernmentIT, under “Comparison of Acquisition Management Best Practices Compared to ERAM Current Practices,” it states, “The contractor, in most cases, is served by increasing the scope of the effort. They will focus on efforts that will directly maximize incentives, sometimes to the detriment of the contract. For example, ERAM currently has CPIF CLINS [cost-plus-incentive-fee, and contract line item number] that incentivize the contractor to keep costs below a specified target cost for development work.
“This CLIN was intended to minimize cost over-runs; however, the CLIN rewards the contractor a higher fee for under-running contract costs. In this example, a contractor could utilize cost-cutting techniques that could harm performance but maximize profits, such as by hiring junior-level personnel or delaying the schedule. Furthermore, the CPIF award amounts are not determined until the end of the CLIN period-of-performance, which in most cases is two or three years. This is too long of a period to truly offer a current and continuous incentive to the contractor,” the MITRE report stated.
“Lastly, the scope and cost ceiling on the CLINS are frequently modified to accommodate unanticipated changes on the ERAM contract. The FAA needs to be vigilant about thorough negotiations at CLIN award to lock down the scope of work and cost, as well as continuously monitor contract performance to ensure the contractor is not growing the effort for its gain. In the future, the FAA should structure the CPIF CLINS so that they are tied to pre-determined cost, schedule, and performance goals that are evaluated at regular intervals (for example, quarterly or bi-annually) to offer a continuous incentive for the contractor, and tie incentive fee payouts to the diverse set of factors that relate to program success, according to the report.
“Lockheed Martin, the contractor for ERAM, delivered the system six months ahead of schedule to the FAA, and the FAA states that it accepted the system after it exceeded the required pass rate of 90 percent, with a score of 92.9 percent.”
“When contractors (Lockheed Martin) delivered the software code to the FAA for final acceptance testing in March 2008, it appears that an incomplete version of the product was presented to the technical center initially,” Scovel told the Committee on Science, Space and Technology Subcommittee on Space and Aeronautics for the U.S. House of Representatives on Feb. 16, 2011.
“It was lacking some of the key software code, that portion being specifically that would enable key interfaces between the test sites in Salt Lake and Seattle and other air traffic facilities. The center “never had an opportunity to test” those interfaces,” said Scovel.
“Although ERAM passed testing at FAA’s Technical Center and achieved government acceptance, testing at initial operating sites in Salt Lake City and Seattle revealed significant software-related problems that have pushed schedules well beyond original completion dates and increased cost estimates by hundreds of millions of dollars. These problems include interface issues between the key sites and other air traffic facilities, radar processing failures, errors that tag flight data to the wrong aircraft, and hand-off problems between controllers.
“To compensate for these problems, controllers relied on workarounds that increased their workload and fatigue and diverted them from managing air traffic. As a result of these issues, the FAA postponed its plans to fully deploy ERAM at the initial sites – originally scheduled for December 2009,” Scovel said.
“Last March, FAA placed a moratorium on further operational ERAM testing at the two initial sites to fix the more than 200 problems identified, reassess its efforts, and develop a new course of action. FAA has since resumed testing, and senior FAA officials state that they are improving system stability, continuing testing at additional sites, and seeing progress in conducting continuous operations without the need to fall back to the legacy system.”
“FAA now plans to complete ERAM in 2014 – a schedule slip of four years – with the next major milestones focused on getting the Salt Lake City and Seattle sites fully operational. However, FAA and its contractor plan to add new capabilities while attempting to resolve problems identified in earlier software versions, which could cause further schedule delays. Updated software releases have already exhibited new problems, including inter-facility interface issues that lock up the system and a significant software failure that resulted in Seattle falling back to the legacy system for several weeks,” Scovel reported.
Scovel told the commission, “while the FAA estimates that delays with ERAM will translate into an additional $330 million to complete deployment, our work and a recent MITRE analysis suggest that the total cost could be as much as $500 million. Scovel noted that cost escalations of this magnitude in today’s fiscally constrained environment will affect FAA’s capital budget and crowd out other projects, and the FAA will incur additional costs to sustain aging equipment longer than planned and retrain controllers on both the legacy and ERAM systems.”
“A driving factor behind potential future delays and additional cost overruns will be ERAM’s performance at large locations, like Chicago and New York. The MITRE analysis cautions that FAA’s initial corrective action plan for ERAM was not comprehensive and that additional time and resources will be necessary to accommodate site-specific operational differences,” Scovel stated.
“Continued problems with ERAM will impact other NextGen efforts. Our work has shown critical interdependencies between ERAM and three of five NextGen technologies that are key to fundamentally changing how air traffic is managed. These three technologies have already been allocated more than $500 million to integrate and align with ERAM.”
“Prolonged delays with ERAM could also impact future software enhancements for new NextGen capabilities, such as flexible and dynamic airspace that will allow controllers to shift segments of airspace to other controllers based on weather and changes in traffic patterns. These future enhancements are currently estimated to cost $1 billion.”
Scovel stated that in June 2010, “we reported that while FAA is working to coordinate with the Department of Commerce, DOD, DHS and NASA on NextGen plans, it has yet to make critical design decisions or address research and development gaps with these partner agencies that will affect NextGen’s cost, schedule and performance.”
He told this commission that there is “still a lack of coordination since that 2010 report between the FAA and partner agencies, and that unresolved issues include integrating weather information into advanced automatic systems, determining joint surveillance requirements to track aircraft, incorporating Unmanned Aircraft Systems (UAS), and assessing NextGen’s human factors input.
“The FAA has not made key decisions about the design of the long-term NextGen System, and continuing to delay these decisions will slow NextGen’s overall progress and impact NASA’s and other agencies’ research and development efforts,” said Scovel.
In closing, Scovel told the commission that the “FAA recently completed an initial acquisition workforce plan to address recommendations in the National Academy of Public Administration (NAPA) study – an important first step. However, the plan requires more development and clarification to be useful. For example, the plan does not specify how or when FAA will actually secure the necessary skill sets and expertise. We have work underway to examine FAA’s plans for determining its acquisition workforce needs and progress in addressing them – including an assessment of FAA’s oversight of its System Engineering 2020 support services contracts worth $7 billion.”
On May 12, Scovel appeared before the Committee on Appropriations Subcommittee on Transportation, Housing and Urban Development, and Related Agencies, U.S. Senate, regarding the FAA’s Fiscal Year 2012 Budget Request: Key Issues Facing the Agency.
Scovel reported to this committee that “uncertain design decisions put NextGen’s costs and schedule targets at risk, similar to the comments made to a house committee in April.
“FAA’s most recent NextGen Implementation Plan provides a framework for what NextGen will resemble in the 2015 to 2018 timeframe, and broadly outlines the linkages between FAA and stakeholder investments. While the plan is responsive to the task force, it does not outline NextGen capabilities, timing and costs, which FAA committed to in previous plans and budget requests to Congress.
“For example, the plan does not discuss how delays in critical design decisions will affect NextGen performance. Delayed decisions include (1) division of responsibility delegated to pilots in the cockpit and to controllers and FAA ground systems for tracking aircraft; (2) level of automation needed to support division of responsibility, ranging from today’s largely manual flight management to a primarily automated system with little controller involvement; and (3) the number and locations of air traffic facilities needed to support NextGen.”
Regarding ERAM, Scovel reported that “unresolved technical problems with the program have resulted in delays and cost increase, citing more than 200 software-related problems at test sites, and the resulting postponement of the testing at additional sites. He said that the FAA is requesting $120 million for ERAM in its fiscal year 2012 budget request and now plans to complete ERAM in 2014, a slip of four years. He cited plans by the FAA and its contractor to add new capabilities while attempting to resolve problems identified in earlier software versions, which could cause further schedule delays.”
It’s unknown how testing is progressing at the Los Angeles Center in Palmdale and how Los Angeles International Airport air traffic is being affected. The Air Traffic Controllers Union is working on the issue with the FAA, the employer of the air traffic controllers. The FAA’s planned deployment of ERAM to en route centers has been moved to 2014, with additional costs of close to $500 million, according to Scovel. With the FAA’s acceptance of the flawed program “six months ahead of schedule,” that cost will now reportedly be paid by the FAA.