Bad Watchdog Season 2 launches June 20.

Championing Responsible National Security Policy

Will We Ever Fly Before We Buy? F-22 Doesn't Meet Basic Testing Criteria

Why does this issue matter?

The F-22 is a textbook example of the Pentagon's frequent attempts to manufacture and deploy weapons before it is known if they really work. This "buy before you fly" mentality too often results in the fielding of weapons that are not thoroughly tested -- placing inadequate weapons and weapons systems in the hands of U.S. fighting men and women and increasing costs to the taxpayer. The Congress and the Secretary of Defense should demand that weapons meet technical requirements and are proven to be effective in the hands of the warfighters who will actually use them.


This report, written on December 20, 2000, indicated that the Air Force had not yet completed five of 11 Congressionally-mandated testing "exit criteria." Since then, Air Force officials have announced the completion of all the testing requirements listed in Appendix A, "Defense Acquisition Board Exit Criteria to Release F-22 Spending."

The final exit criteria requirement, initiate radar cross section testing, was completed on February 5, 2001.

To date, Pentagon officials have not scheduled a meeting of the Defense Acquisition Board to approve low-rate initial production of the F-22. However, it is believed that the meeting will occur sometime before Congressionally-appropriated bridge funding runs out on March 31, 2001.

The Project On Government Oversight believes that the Pentagon should adhere to a higher standard and not begin production of the F-22 until the Air Force has completed all operational testing of the aircraft.

Executive Summary

In the coming days, the Pentagon will likely recommend releasing $2.1 billion to begin production of the F-22 fighter even though the aircraft has not met as many as five out of 11 testing criteria required to be met before funding is released. (Appendix A)

The Defense Acquisition Board (DAB) (Appendix B), composed of the Pentagon's senior acquisition officials as well as the top personnel running the F-22 program, is expected to give the green light to build 10 F-22's on January 3, 2001. This first wave of the production process, known as low-rate initial production, is, despite Congress' clear directive in the defense budget bills of 2000 and 2001 that F-22 production funding only be released if all 11 of DAB's testing criteria are met.

The DAB should not only adhere to these very minimal testing criteria before going into production, it should set a higher standard requiring that all operational testing be completed, particularly of the avionics (the eyes, ears and brain of the plane) prior to costly buys of new aircraft like the F-22. No greater lesson than this was learned from the "buy before fly" B-1 and B-2 bomber fiascos.

"Incomplete testing is fatal and extremely wasteful. B-1 avionics . . . still do not function in the aircraft after two decades, despite large transfusions of funds. Such refined identification capability has never been achieved though frequently promised," according to a report earlier this year by legendary aircraft designer Colonel Everest Riccioni (ret.).

The DAB is chaired by Jacques S. Gansler, Under Secretary of Defense for Acquisition, Technology and Logistics, who will be leaving this position two days after overseeing this critical decision. It is significant that this decision is being squeezed under the wire on the eve of a major military review by the incoming Bush administration. Aides to President-elect George W. Bush have said they want an immediate review of tactical aircraft programs to determine whether or not the Pentagon is buying too many fighter jets at the expense of needed transport and refueling aircraft.

After all, it was Vice President-elect Richard Cheney who in 1991 as Defense Secretary boldly terminated development of the Navy's $57 billion A-12 attack plane, the largest weapons program ever terminated by the Pentagon. Cheney also attempted, without success, to nix the V-22 Osprey death trap during his tenure as Defense Secretary.

The Pentagon's history of purchasing incomplete planes like the F-22 has not only cost the American taxpayers billions of dollars, it has cost tens and possibly hundreds of military personnel's lives. In recent weeks, despite testing deficiencies and several fatal crashes, the Pentagon pushed to secure funding to begin full scale production of the V-22. It took the December 12th V-22 crash killing four to postpone this decision.

Testing Not Complete

As of August, the F-22 has flown less than 19 percent of the total hours of scheduled testing, and at the writing of this report, five of the 11 Congressionally-mandated tests have not been completed. The requirements not yet met include testing of the F-22's stealth (radar cross section testing) and of the sophisticated avionics (first flight of test aircraft 4005 and 4006). The advanced integrated 3.0 avionics software, thought to be the F-22's toughest development challenge, is required to be flown in two test aircraft although the criteria do not require the avionics to work.

Another requirement that has not been completed is a laboratory static load test of the aircraft's airframe (complete static structural testing). This test will not likely be finished in 2000 because an aluminum test rig broke during a recent test. Nor will the Air Force likely meet its required goal of completing 40 percent of fatigue testing aimed at demonstrating how an aircraft performs as it ages (fatigue life testing).

The Air Force says it expects to complete all but two of the 11 criteria by year's end. However, the total completed may be even fewer as bad weather has slowed the ferrying of one of the aircraft from the assembly plant in Georgia to the site of F-22 testing at Edwards AFB in California, according to a spokesperson.

Earlier this year, Brigadier General William Jabour, the director of the F-22 System Program Office at Wright-Patterson AFB, said structural testing won't be completed until the spring of 2001. 1

In Congressional testimony this March, Philip E. Coyle, the Department of Defense's chief testing evaluator, expressed concerns that F-22 testing is proceeding much more slowly than previous programs. (Appendix C) The result: Cost overruns and and an overall reduction in hours of testing.

As a third and final step toward releasing F-22 production funding, Coyle must by law report to Congressional committees on the adequacy of testing to date. He must also measure and predict performance of F-22 avionics systems, stealth characteristics and weapons delivery systems.

"Over the past three years we have lost 49 flight test months which could have been available for (F-22) testing," Coyle said in his testimony. "...Basically, not enough of the test program has been completed to know whether or not significant development problems remain to be corrected."

Last year, pressure from the Pentagon to prematurely begin low-rate initial production of the F-22 drew attempts in the House to force a "pause" in procurement funding. In October 1999, a proposal to eliminate $1.8 billion in F-22 procurement funding passed the House of Representatives, effectively delaying the F-22 procurement for a year.

"We now have three fighter aircraft lines in development which, if all three are completed, will cost more than $340 billion, and if we eliminate one of those lines, we might save a third of that," the House bill's author, Rep. Jerry Lewis (R-CA), said in July 1999. "And we can still end up with the finest tactical fighting force in the world - we are committed to that." (Appendix D)

Learning From Mistakes: B-1 and B-2 Bombers

The DOD has a long history of pouring billions of dollars into the production of new aircraft before ensuring they are safe and ready for combat. It should learn from its past mistakes so the F-22 doesn't become the latest boondoggle, at best - or tragic death trap at worst. The B-1 and B-2 bombers have been plagued with serious and lingering avionics problems that should have been solved before going into production.

These fiascos have drawn the ongoing criticism of the GAO and others who have panned the long-standing DOD practice called "concurrency" - an overlap of testing and production that can result in the purchase of large numbers of new aircraft before extensive testing determines if they work.

In the B-1's case, more than half of the fleet was delivered to the Strategic Air Command before avionics testing was substantially underway. After delivery, the Air Force discovered there was a serious structural avionics design problem. (Appendix E)

Similarly, the Air Force began production of the B-2 in 1986 despite numerous problems that included the aircraft's avionics. It wasn't until 1997 that an initial operational test and evaluation was completed, and as late as in 1999 there were still avionics problems. Originally projected to cost $438 million per aircraft, by 1998 the cost of each B-2 increased more than fourfold to $2.1 billion.

The avionics system is supposed to provide the crew information on enemy threats. But a 1998 GAO study concluded that the B-2A's defensive avionics system provided "inaccurate and cluttered" information to the crew. (Appendix F)

"Historically, defensive avionics have experienced significant problems during development," said the study. "The B-1B bomber had serious deficiencies with its defensive avionics and the Air Force is still working to provide an effective defensive capability for the B-1B." Also in the study: "Air Force officials said the cost of making the defensive system meet originally planned capability is unaffordable at this time."

The B-1 and B-2 bomber problems are indicative of a broader systemic ill. In fact, the GAO in a 1997 report called the DOD's acquisition strategies "high risk."

"DOD continues its practice of beginning production of a weapon system before development, testing, and evaluation are complete," the study said. "When this strategy is used, critical decisions are made without adequate information about a weapon's demonstrated operational effectiveness, reliability, logistics supportability, and readiness for production." (Appendix G)

The 1997 report followed an earlier stinging criticism in a 1995 GAO report. The 1995 report concluded that several aircraft systems, including the T-45 trainer aircraft, B-1B bomber and C-17 cargo aircraft, entered low-rate initial production before completing critical operational testing intended to ensure that a plane met rigorous combat standards. (Appendix H)

"This resulted in the purchase of systems requiring significant and sometimes costly modifications to achieve satisfactory performance, acceptance of less capable systems than planned, and in some cases deployment of substandard systems to combat forces [emphasis added]," the report said.

Learning From Mistakes II: The V-22 Osprey

The V-22 offers another example of the Pentagon's egregious acquisition practices. Prior to the latest in a series of fatal accidents that have killed 30 since 1991, the Pentagon was aiming for a decision this month to begin full production. In all, the Marines, Navy and Air Force want to build a total of 458 V-22s at a cost of $38.1 billion. (Appendix I)

The most recent V-22 crash, a December 12, 2000 North Carolina accident that killed four, resulted in a postponement of a production decision and an investigation into what went wrong.

Although the past accidents have been attributed to pilot error, the DOD's Operational Testing and Evaluation Office (OT&E) as well as its Inspector General have cautioned that additional testing is needed to verify the correction of numerous V-22 deficiencies.

"The MV-22, as tested, is not operationally suitable, primarily because of reliability, maintainability, availability, human factors and interoperability issues," the OT&E office wrote.

The report claims that one of the V-22's nagging problems has been an aerodynamic phenomenon known as "Vortex Ring State," which causes the aircraft to roll sharply to one side, followed by a rapid dropping of its nose and a steep dive. (Appendix J)

Likewise, the Department of Defense Inspector General report said that the V-22 will not successfully demonstrate test and evaluation of key operational requirements prior to full-rate production. The agency cited 22 major deficiencies with the aircraft, ranging from a low reliability of its weapon system to the lack of a ground collision avoidance and warning system.

"Failure to test and correct the deficiencies could adversely affect the survivability, logistics support, and readiness of the aircraft and its ability to fully perform its intended mission," the Inspector General report said. (Appendix I)

Why The Rush?

The GAO has argued that the rush to put the F-22 in production is ill-founded.

"Since the F-22 program entered full-scale development in 1991, the severity of the projected military threat in terms of quantities and capabilities has declined," a 1994 GAO report said. "Instead of confronting thousands of modern Soviet fighters, U.S. air forces are expected to confront potential adversary air forces that include few fighters that have the capability to challenge the F-15--the U.S. front line fighter."

"Our analysis shows that the F-15 exceeds the most advanced threat system expected to exist. We assumed no improvements will be made to the F-15 but the capability of the 'most advanced threat' assumes certain modifications. Further, our analysis indicates that the current inventory of F-15s can be economically maintained in a structurally sound condition until 2015 or later." (Appendix K)

Meanwhile, the cost of replacing the F-15 with the F-22 has escalated to a whopping $180 million-plus per aircraft. Although original plans called for an order of 750 of the tactical fighters, the Air Force now says it only plans to purchase 339 F-22s, currently expected to go into service beginning in 2005. However, recent estimates by the GOA Office have reduced to 254 the number of aircraft that the $62 billion contract will fund. (Appendix L)

Rep. John Murtha (D-PA), the ranking member of the House Appropriations Committee's defense panel, believes that number could even be as low as 150.2

F-22 critics such as legendary aircraft designer and retired Air Force Col. Riccioni contend that the excessive price tag for the fighter will ultimately weaken U.S. air power.

The Air Force has always depended on superior numbers to win air battles, the former fighter pilot said. Col. Riccioni believes that in the end the Air Force will only be able to afford as few as 100 F-22s.

"Numbers guarantee victory," Riccioni said. "Numbers develop intensity and allow multiple attacks."3


  • Production of the F-22 should not begin until the U.S. Air Force has completed all operational testing of the aircraft.
  • Congress should demand the standards they set be upheld. If not met, the Pentagon should be held accountable and funds for low-rate initial production funding of the F-22 withheld.
  • The Department of Defense should end its longstanding practice of "concurrency." Congress should ensure that the Pentagon follows guidelines to ensure that the history of "buy before fly" problems with such aircraft as the B-1, B-2 and V-22 are not repeated.


Appendix A:

"Steps Mandated by Congress before F-22 Funding is Released," Sec 8124 of the 2001 Defense Appropriations Act resulting in the creation of Defense Acquisition Board Exit Criteria to Release F-22 Funding; Annotated by Project On Government Oversight; December 20, 2000.

Steps Mandated by Congress before F-22 Funding is Released

Language from Defense Appropriations Act 2001 (H.R. 4576)

SEC. 8124. None of the funds made available in this Act or the Department of Defense Appropriations Act, 2000 (Public Law 106-79) may be used to award a full funding contract for low-rate initial production for the F-22 aircraft program until--

(1) the first flight of an F-22 aircraft incorporating Block 3.0 software has been conducted;

(2) the Secretary of Defense certifies to the congressional defense committees that all Defense Acquisition Board exit criteria [listed below] for the award of low-rate initial production of the aircraft have been met; and

(3) upon completion of the requirements under (1) and (2) above, the Director of Operational Test and Evaluation submits to the congressional defense committees a report assessing the adequacy of testing to date to measure and predict performance of F-22 avionics systems, stealth characteristics, and weapons delivery systems.

Defense Acquisition Board Exit Criteria to Release F-22 Funding

Status as of December 20, 2000


Yes ----1. Complete first portion of engine Initial Service Release Qualification Test.

Yes ----2. Complete air vehicle Final Production Readiness Review.

No-----3. Complete first flight on Engineering and Manufacturing Development (EMD) Aircraft 4003, 4004, 4005, and 4006.

Yes ----4. Complete EMD Aircraft 4008 fuselage, wing, and empennage mate.

No-----5. Complete static structural testing.

No-----6. Initiate fatigue life testing with the goal of completing 40% of first fatigue life.

Yes ----7. Complete Critical Design Review for avionics Block 3.1 software.

No-----8. Complete avionics Block 3.0 first flight & initiate testing of Block 3.0 unique functionality.

No-----9. Initiate radar cross section flight testing [stealth].

Yes ---10. Initiate high angle-of-attack testing with weapons bay doors open.

Yes ---11. Initiate separation testing of AIM-9 and AIM-120 missiles.

Appendix B: Composition of Defense Acquisition Board, Interim Regulation (DoD 5000.2-R) -- Mandatory Procedures for Major Defense Acquisition Programs (MDAPs) and Major Automated Information System (MAIS) Acquisition Programs, October 23, 2000.

Appendix C: Statement by The Honorable Philip E. Coyle, Director, Operational Test & Evaluation, Before The Senate Armed Services Committee AirLand Forces Subcommittee, Tactical Aviation; March 22, 2000.

Appendix D: News Release, Rep. Jerry Lewis on FY2000 Defense Appropriations; July 22, 1999.

Appendix E: Testimony of Nancy Kingsbury, Director Office of Air Force Issues for National Security and International Affairs Division, U.S. General Accounting Office, Hearing of the House Subcommittee on Legislation and National Security of the Government Operations Committee; March 6, 1991.

Appendix F: "B-2 Bomber: Additional Costs to Correct Deficiencies and Make Improvements," General Accounting Office; June 6, 1998.

Appendix G: "DOD High-Risk Areas: Eliminating Underlying Causes Will Avoid Billions of Dollars in Waste," Testimony, General Accounting Office; May 1, 1997.

Appendix H: "Tactical Aircraft: Concurrency in Development and Production of F-22 Aircraft Should Be Reduced," General Accounting Office Report; April 19, 1995.

Appendix I: "Appendix C. Waived Operational Requirements for the V-22," Audit Report, V-22 Osprey Joint Advanced Vertical Aircraft, Office of the Inspector General, Department of Defense; August 15, 2000.

Appendix J: "Combined Operational Test & Evaluation and Live Fire Test & Evaluation Report on the V-22 Osprey," Director, Operational Test & Evaluation Department of Defense; November 17, 2000.

Appendix K: "Tactical Aircraft: F-15 Replacement Is Premature as Currently Planned," General Accounting Office Report; March 25, 1994.

Appendix L: "Defense Acquisitions: Recent F-22 Production Cost Estimates Exceeded Congressional Limitation," General Accounting Office; August 15, 2000.


1. Defense Daily, "Jabour Advises Structural Test Waiver Not Needed for F-22 Before DAB Next Month," November 29, 2000.

2. Defense Daily, Murtha: Pentagon Can Afford Only 150 F-22s, Sept. 22, 2000.

3. Project On Government Oversight F-22 Press Briefing, Everest E. Riccioni, Col. USAF, Retired, June 8, 2000.