Independent Weapons Testing Comes Under Fire, Again

Sometimes, history has an odd way of repeating itself—over and over and over, again.

In 1983, Congress mandated that operational testing—the most important and objective form of combat weapons testing—be protected permanently from the pressures and influence of the Pentagon’s procurement bureaucracy. That law, establishing an independent Director of Operational Test and Evaluation (DOT&E) who reports directly to the Secretary of Defense and Congress, was the most significant and useful reform to military procurement policy over the past forty years.

The defense industry, through its numerous allies in Congress, fought desperately then—and is still fighting desperately now—to stymie the law’s intent. Their main argument is that independent weapons testing causes massive cost increases and schedule delays for major weapons systems. This specious argument has spearheaded recurring congressional and industry assaults on independent operational testing in the decades since the creation of DOT&E—most recently just a few months ago in the National Defense Authorization Act (NDAA) for Fiscal Year 2015.

There is ample evidence, however, which demonstrates that rigorous operational testing actually reduces the overall time and money needed to get usable weapons into the field, and, even more importantly, saves the lives of the men and women fighting our wars.

After all, they need weapons that work, not weapons that need fixing.

Dr. J. Michael Gilmore, the current director, addressed operational testing’s detractors at a recent National Defense Industrial Association (NDIA) event: “When people claim that testers are driving billions of dollars of costs into programs, the facts just simply don’t support that claim.” Gilmore has backed this assertion with a 65-page briefing detailing more than a dozen recent programs—ranging from mine resistant vehicles to nuclear submarines—where independent operational testing identified serious combat suitability shortfalls early enough to allow for fixes before the weapons failed in combat.

One example from DOT&E’s briefing is the P-8A Poseidon, a newly developed maritime reconnaissance and anti-submarine aircraft based on Boeing’s 737-800 commercial airframe. A DOT&E review rejected the Navy’s proposed original operational test plan because it did not evaluate the P-8A’s two primary combat missions: anti-submarine detection and reconnaissance. Instead, the Navy simply wanted to test the aircraft’s engineering requirements; that is, its ability to fly a specific distance with a specific payload. DOT&E insisted on a test that exercised enemy submarine detection and attack as well as ocean reconnaissance under realistic combat conditions. The Navy’s operational testers agreed that such combat testing was essential.

When these tests were conducted as part of realistic fleet exercises, they revealed “important deficiencies” in surveillance and in the aircraft’s ability to hunt and destroy submarines, all due to problems with the P-8A’s sensor systems. Gilmore’s subsequent evaluation report concluded that the P-8A was “not effective for the intelligence, surveillance and reconnaissance mission and is not effective for wide area anti-submarine search.” As a result, the Navy is now working to address these serious problems and will do follow-on operational testing to ensure that this $35 billion program actually delivers combat-effective aircraft.

The Assault on DOT&E Independence

Naturally, the defense industry is opposed to realistic, independent operational evaluation and reporting because test failures can lead to program cancellation, reduced funding, and diminished profit. Companies with high-value contracts have become adept at spreading facilities and subcontracts out across as many states and congressional districts as possible in order to secure congressional allies, a phenomenon renowned defense analyst Franklin “Chuck” Spinney has termed “political engineering.” This outsourcing strategy weakens national security in many ways; one manifestation is the recurring effort by industry’s congressional allies to undermine the independence of DOT&E and the rigor of its testing.

Reflecting the misguided view that DOT&E—rather than procurement mismanagement coupled with shoddy contractor performance—is to blame for unprecedented overruns and schedule slippages, the House Armed Services Committee included language in this year’s NDAA intended to weaken the independence and combat realism of operational testing.

The bill, as passed by the House of Representatives, requires DOT&E to:

Consider the potential for increases in program cost estimates or delays in schedule estimates in the implementation of policies, procedures, and activities related to operational test and evaluation, and to take appropriate action to ensure that the conduct of operational test and evaluation activities do not unnecessarily impede program schedules or increase program costs.

Though not mandatory, this language opens the door for the acquisition bureaucracy to pressure DOT&E to pare back thorough operational testing, even though such operational testing consistently yields long-term savings and more combat-effective weapon systems.

The House Armed Services Committee’s NDAA also includes biased instructions to the Government Accountability Office (GAO) to evaluate DOT&E’s causing of “unnecessary” costs and delays in weapons procurement:

To help inform the committee’s understanding of how operational test and evaluation processes and activities may unnecessarily increase schedule and cost of major defense acquisition programs, the committee directs [GAO] to review operational test and evaluation processes and activities.

What is actually unnecessary is yet another GAO study on DOT&E’s effect on cost and schedule. GAO produced exactly such a study in 1997 and concluded that operational testing in fact saves money and time in fielding successful weapons. Nothing has happened since 1997 to change GAO’s fundamental conclusions.

After the House Armed Services Committee produced this alarming language, the Project On Government Oversight (POGO) sent a letter to their Senate counterparts in response to the outrageous claims that DOT&E unnecessarily causes cost increases and schedule delays. In the letter, Executive Director Danielle Brian pointed out that, “Clearly, these claims do not consider even larger—by far—offsetting long-term cost increases and longer production stretch outs associated with insufficiently tested, flawed hardware that has to be fixed at great expense and only after still more delay.”

Following POGO’s letter, the Senate Armed Services Committee included language in their version of the NDAA that at least recognizes the important role DOT&E performs in ensuring that weapon systems are “suitable and effective in combat.” Unfortunately, the Senate NDAA report also includes language requiring GAO to study the extent to which “major weapon systems have been required to conduct operational testing in excess of levels necessary to demonstrate compliance with program requirements.”

This language once again reflects an industry campaign to limit operational testing and evaluation to only the program’s contractual requirements (which are, after all, the engineering requirements set by the acquisition bureaucracy). This flies in the face of the longstanding and traditional military principle that the combat user needs to independently test all new weapons to determine whether they will work in combat, not whether they will meet engineering requirements set by the developer.

Both versions of the NDAA demonstrate a regrettable ignorance of how and why Congress created the independent DOT&E in the first place—and of the ensuing three decades of repeated industry-incited battles to undo the office’s major contributions to stronger defense.

Bipartisan Inception and Early Congressional Opposition

In 1981, a small group of reform-oriented staffers within the Pentagon and Congress, working with liberal Senator Gary Hart (D-CO) and conservative Representative William Whitehurst (R-VA), initiated the Congressional Military Reform Caucus, which soon grew to include more than 150 House and Senate members. One of the early discussions within the new caucus focused on the need for more combat-realistic weapons testing before funding production.

In 1983 with strong support from the Reform Caucus, GAO examined existing Pentagon weapons testing practices for newly introduced weapon systems and found that many of them were deployed “without having fully demonstrated their capabilities under representative combat conditions.” Furthermore, GAO found that Pentagon weapons testers were ineffective and may have had conflicts of interest with the defense contractors that developed the weapons under evaluation.

Soon after, the Deputy Inspector General of the Department of Defense testified before Congress that the Pentagon was developing and testing systems at the same time (known as concurrency), was not testing weapon systems in realistic combat situations, and was relying too heavily on defense contractors during testing and evaluation. The media’s coverage of testing inadequacies and associated weapons failures grew rapidly, significantly aided by detailed factual material unearthed by POGO’s predecessor, the Project on Military Procurement.

In 1983, a bipartisan group of the most active members of the Congressional Military Reform Caucus, led by Senators David Pryor (D-AR), William Roth (R-DE), Nancy Kassebaum (R-KS), and Charles Grassley (R-IA), introduced the Operational Testing and Evaluation Act to create an independent director overseeing the operational tests of all four services and reporting directly to the Secretary of Defense and Congress.

Winslow Wheeler, director of the Straus Military Reform Project at POGO, writes in Military Reform: An Uneven History and an Uncertain Future,

If [Pryor’s] bill were to become law and take hold, no longer would laboratory tests under cooperative conditions be substituted for combat realistic tests in dirt, mud, and confusion, and using regular soldiers as operators. No longer would the manufacturer be able to design or score the tests. No longer could the weapon system’s advocates write the reports on the tests of the weapons they sought to protect.

Despite the strong opposition of the chairman of the Senate Armed Services Committee, John Tower (R-TX), as well as other industry-friendly members working closely with senior Pentagon officials, Pryor’s bill secured the co-sponsorship of twenty-two Senators, both Democrat and Republican. Senator Pryor eventually offered his bill as an amendment to that year’s defense authorization act, and, after some last minute legislative maneuvering, Pryor’s amendment was approved overwhelmingly by a vote of 91 to 5. The underlying bill was signed into law on September 24, 1983.

Over the course of the next decade, DOT&E’s work improving operational testing won strong praise, including from former Secretary of Defense Dick Cheney—a very active former member of the military reform caucus—who remarked that DOT&E saved more lives during Operation Desert Storm than any other initiative.

Despite the bipartisan support that DOT&E enjoyed, industry’s congressional allies attempted to eliminate the office in the FY 1996 defense authorization bill. This move followed a report by a Department of Defense advisory panel of senior industry and government acquisition experts, which, unsurprisingly, found that DOT&E was unnecessarily impeding defense acquisitions.

After strenuous objection from Senators Pryor and Roth, as well as the Deputy Inspector General of the Department of Defense, the proposal to eliminate DOT&E was eventually dropped. DOT&E and its supporters would live on to fight another day.

Current Efforts

The alleged hardship of living within a sequestered budget—albeit one that exceeds Vietnam War and Reagan-era funding peaks—provides a convenient excuse for new attempts to cut operational testing and evaluation budgets. That this is an excuse and the opposite of fiscal prudence is obvious: in times of budgetary constraint, the worst activity to cut is the one that provides the most important evidence for deciding where acquisition money is ill-spent. As Gilmore noted at the recent NDIA conference, “before you commit to production…you should want to understand whether what you’re about to start producing is actually useful to the men and women who have to use it in combat.”

It is also fiscally imprudent—not to mention dangerous—to cut the one activity that will discover design flaws early enough in the process to avoid not only astronomically expensive but also possibly fatal mistakes.

Now, both the House and Senate committee reports accompanying their versions of the FY 2015 NDAA—using biased but somewhat differing language—guide the GAO review of DOT&E processes and activities toward a hostile point of view. The committee reports’ negative instructions to GAO are, quite unfortunately, unlikely to be undone. However, the two NDAA bills themselves still have to go to conference. That final conference report should include a separate and much more constructive mandate for a different GAO review that examines ways to improve and strengthen DOT&E’s contribution to providing more effective weapons for our troops.

As POGO Executive Director Danielle Brian recommended in her letter this spring, rather than undermining the ability of DOT&E to conduct realistic and thorough testing and evaluation, Congress should instead enhance DOT&E’s mandate with expanded resources and thoroughly protected independence. As a first step to accomplishing that, the conference version of the final NDAA should require that GAO review specific funding and organizational steps needed to enhance realistic and independent operational testing.

Beyond that, if Congress truly is concerned about the real causes of today’s hundreds of billions of dollars of cost overruns and unending schedule delays, then it should require that operational testing and evaluation always take place before production begins.

Nothing else will eliminate the ever-worsening concurrency malpractice that has resulted in such disastrous recent programs as the F-35 Joint Strike Fighter, Littoral Combat Ship, ballistic missile defense, Joint Tactical Radio System, and Ford-class carrier.

Image from the U.S. Navy.