Mandy Smithberger
Author
House Acquisition “Reform” Bill Guts Key Weapons Development Oversight Provisions
Report Quick Links
Weakening Operational Testing and Evaluation
Increasing Systems Engineering and Developmental Risks
Committing Taxpayers to Risky Multiyear Procurements
The Project On Government Oversight (POGO) has spent nearly 35 years investigating waste, mismanagement, and abuse in the Department of Defense’s (DoD’s) weapons acquisition system. The Agile Acquisition to Retain Technological Edge Act (H.R. 1597), sponsored by House Armed Services Committee Chairman Mac Thornberry (R-TX) and Ranking Member Adam Smith (D-WA), correctly assesses many of the causes of failures of the acquisition system. Unfortunately, a number of bill’s so-called solutions will actually make the problem worse.
Some of these provisions undercut nascent improvements to defense acquisition identified by the Government Accountability Office (GAO), the Department of Defense Inspector General (DoD IG), and the Defense Contract Audit Agency (DCAA). Specifically, this legislation contains provisions to weaken key officials who act as checks on program management to ensure that there is complete developmental and operational testing and sound systems engineering. It also dilutes requirements to save money and increase performance through competitive prototyping and responsible use of multiyear procurement contract authority.
Weakening the officials responsible for testing and systems engineering through unnecessary requirements and demotion of their authority will increase the likelihood of waste and losing our technological edge, as risky programs are approved before technology is mature and taxpayer resources are wasted on systems that are never fielded. Simultaneously, this bill strengthens the hand of program managers, creating the same effect. Below are some of the provisions we find most troubling.
Weakening Operational Testing and Evaluation
The proposed language in Section 701 requires the Director of Operational Test and Evaluation to consider the potential for cost increases and schedule delays, which could undermine the effectiveness of the office of the Director, Operational Test and Evaluation (DOT&E). Specifically, as POGO pointed out when this anti-reform provision was offered by the House Armed Services Committee in last year’s National Defense Authorization Act, these superfluous considerations only compound the systemic psychosis of program advocates that testing should be delayed, if not altogether avoided, as much as possible. Rather than testing, it’s too much concurrency between testing and production that commits the Department to acquiring immature technology, which slows programs, increases costs, and results in fielding unworkable systems.
A study conducted by the Congressional Budget Office (CBO) confirmed that concurrency between testing and production contributes to weapon system cost growth, and in highly concurrent programs, contributed to schedule delays. Concurrency has been specifically cited as causing cost increases and schedule delays for the C-17, the Littoral Combat Ship, and the F-35 Joint Strike Fighter. Cost considerations or schedule delays also could have impeded testing done by DOT&E that discovered problems missed in developmental testing in the Warfighter Information Network-Tactical system and the Expeditionary Fighting Vehicle. The bottom line is that testing does not cause overruns and delays; it reveals the true costs of concurrency, which are increased costs and usually more delays.
Many of the provisions in this legislation are focused on increasing the speed of acquisition to try to pursue a technological “edge” and on addressing defense contractor complaints that contracting officers and program managers are overly focused on cost. Adding cost and schedule considerations to DOT&E, one of the sole Pentagon offices focused on operational performance, is particularly inappropriate since it would sacrifice safety and performance for near-term “savings” that would inevitably and ultimately result in more cost problems, greater combat risks for our troops, and reduced weapon program effectiveness. The fact that effective operational testing of weapons uncovers problems before ramping up production—thereby saving money and lives in the long-term—seems to be lost on those who are focused on short-term corporate cash flow and career advancement.
The proposed language for DOT&E to consider delays in schedule is also unlikely to achieve its desired ends to improve speed in the acquisition system. A joint study conducted last year by the Under Secretary of Acquisition, Technology, and Logistics and DOT&E found that testing was the least common reason for delay and that “all programs that had problems in test conduct also had at least one other reason that contributed to delay.” As with cost, the vast majority of schedule delays were caused by fixing problems that must be addressed before the program could move forward. It’s preposterous to blame testing for increased delays when testing discovers problems.
It’s also important to remember that Congress strengthened DOT&E as part of the Weapon Systems Acquisition Reform Act of 2009 (Public Law 111-23, WSARA) due to the Department’s past failures to follow sound acquisition practices. A 2008 Defense Science Board Task Force on Developmental Test and Evaluation found that over two-thirds of Army systems failed to meet their reliability requirements in operational testing, and that in almost every case those programs failed to meet the criteria for successful developmental testing. Weakening DOT&E only increases the likelihood of delivering systems that are not ready for combat.
DOT&E plays a pivotal role in aiding acquisition officials and Congress in making informed decisions about our capabilities and how to best allocate resources for our national security. Former Defense Secretary Dick Cheney said that independent weapons testing “saved more lives” during Operation Desert Storm than perhaps any other single initiative. It would be far more constructive to enhance DOT&E’s ability to conduct even more realistic and thorough testing with expanded resources and thoroughly protected independence than to reduce them. Congress should remove language directing DOT&E to restrict realistic, adequate operational testing. Retaining this language would only exacerbate the very cost increases and schedule delays the House purports to be trying to fix.
Increasing Systems Engineering and Developmental Risks
Disciplined systems engineering and developmental testing are key components to the success of a major defense acquisition program, and inadequate systems engineering can be a primary cause of program failure and cancellation. This is why both of these offices were strengthened as part of WSARA. In 2008, the Defense Science Board found that many systems failed initial operational test and evaluation due to insufficient development testing and a lack of a disciplined systems engineering approach. HASC’s Defense Acquisition Panel also found that investing and prioritizing systems engineering and developmental testing is essential:
The early stages of an acquisition program are in many ways the most critical. It is in the early stages that investments must be made in systems engineering, in acquiring technical data rights to support competition and system sustainment, and in robust developmental testing. While all these elements are critical for overall program success and best accomplished early, they are all too easy to defer when funding is limited.
It is also concerning that Section 601 eliminates annual reports from both of these offices, smothering the effectiveness and ability of these offices to transmit their findings and consequently reducing congressional oversight and discipline in the acquisition system. Congress, the press, and the public should be deeply troubled by this effort to eliminate these reports from public scrutiny. The GAO has described how the recent reforms to the Director of Systems Engineering, required by WSARA, have resulted in appropriate influence over the acquisition process, though the Director of Developmental Testing has experienced more struggles in becoming effective.
Recent work by these offices underscores their importance. Assistant Secretary of Defense for Systems Engineering Stephen Welby issued important reports on F-35 capability deficiencies, including how software development difficulties could trigger delays of up to nine months for key F-35 program milestones. His office also aided Congress in determining whether the new VXX was positioned to avoid repeating the mistakes of the failed VH-71 program it was replacing (see also Inside Defense’s “Sikorsky VXX Deal Could Net $150 Million More In Fees But Fewer Aircraft,” no link available). He’s also guiding the Department’s open systems architecture initiativesto reduce costs through effective competition (closed systems typically mean that only the incumbent contractor can compete for follow on contracts). Assistant Secretary of Defense for Developmental Test and Evaluation C. David Brown, along with DOT&E, is working to improve the cyber security testing of DoD weapon systems by stressing these requirements as part of a “Shift Left” initiative. Given the current vulnerabilities of our weapon systems, reducing the role of his office to reducing these cyber threats creates unnecessary risks.
These offices play important roles in making sure that programs do not begin with unacceptable and unaffordable levels of risk. The resulting reports, and their public nature, is a vital contribution to oversight. POGO believes that any reforms to the roles these offices play should enhance, not weaken, their authority.
Reducing Real Competition
One of the most heralded initiatives in this bill is the proposal to reduce the use of competition and competitive prototypes in acquisition. It’s a foolish and destructive proposal. Real competition between genuine prototypes has consistently resulted in better weapons at a lower price in the few occasions it has been tried. Specifically, competitive prototypes were used with great success in the F-16, A-10, and Joint Direct Attack Munition (JDAM) programs—and perhaps most famously in the Great Engine War. It’s also a practice that could have greatly benefited the F-35 and F-22 programs, had the letter and the spirit of the idea been followed.
When this requirement was included in the Weapon Systems Acquisition Reform Act, the Center for Defense Information criticized the vague “national security” waivers put in place. GAO reviews of the use of these waivers found that many of these waivers were issued without adequately considering the potential for competitive prototypes to reduce costs and increase performance. Specifically, the GAO found that the Army failed to adequately analyze how competitive prototypes could reduce development risks in the Armored Multi-Purpose Vehicle, the Department of Defense did not consider potential benefits for the Enhanced Polar System, and that the Air Force only considered the most expensive prototyping opportunities before issuing a waiver for the Combat Rescue Helicopter.
At this point, the Department has only paid lip service to competitive prototypes, though Better Buying Power 3.0 also includes increased use of prototyping. Congress should be pushing the Department of Defense to properly implement competitive prototyping, not continuing to reduce the feeble attempts at competition. Congress should also establish, adequately fund, and maintain operational units in the Services to evaluate and affect genuinely competitive prototypes.
Committing Taxpayers to Risky Multiyear Procurements
Used properly for mature, well-run programs, multiyear procurement can result in cost savings for the Department of Defense. Unfortunately, it has often been used to commit the Department to risky, immature programs. POGO is concerned that Section 707, which further weakens the already permissive requirements for multiyear procurement of major defense acquisition programs, will create significant cost risks and increase the likelihood of committing the Department to immature, risky programs.
A 2006 POGO investigation found that Congress authorized multiyear procurement of 60 F-22A fight jets despite the program’s inability to establish the statutorily required substantial savings. A 2008 GAO study examining past instances of multiyear procurement authority found unit cost growth increased 10 to 30 percent. Concerned that multiyear procurement authority was being used irresponsibly for programs like the F-22, Senator John McCain (R-AZ) offered language to the FY 2008 National Defense Authorization Act to clarify the amount of savings required to ensure that this authority would not be used “to insulate poorly performing systems from effective congressional oversight.” This language was later watered down to include a broad waiver to give programs a loophole from these requirements.
It appears that the change to weaken multiyear procurement authority is based upon a false myth perpetuated by the defense industry that multiyear procurement authority, and particularly the substantial cost savings requirements, are overly stringent and mandate cost savings of 10 percent or higher prior to any approval.
The current statutory requirement is designed to encourage the Department to use best practices and has been interpreted and embraced by Defense Department leadership accordingly. Jamie Morin, the head of the independent Cost Assessment and Program Evaluation (CAPE) office, similarly told the Senate that he interpreted the “substantial savings” requirement as a reasonable, but not absolute standard, and that consideration to authorize multiyear procurement “should include the potential trade-off between cost savings and reductions in acquisition and budget flexibilities.”
POGO believes that the Department’s interpretation of this authority is correct and that changes to this language are not necessary at this time. Congress should also consider asking the GAO to update its previous study to determine whether multiyear procurement authority truly results in efficiencies and cost savings to determine if further restrictions on this authority should be made.
Conclusion
Legislation is only one step in reforming weapons acquisition. The defense procurement process is also in desperate need of discipline, aggressive congressional oversight, and accountability. Standards for appropriate levels of design maturity should be clearly defined to meet missions and requirements. Waivers from key procurement rules should be used rarely and be difficult to obtain. Any waivers from procurement rules should also be made available to both Congress and the public. All technologies should be mature before committing to production. The Department should implement real competitive prototyping to create competition that furthers innovation. Finally, initial operational test and evaluation should be complete before significant production begins.
Mandy Smithberger 
Oversight in your inbox
Sent Saturdays
