Big Dreams Still Need Oversight: Missile Defense Testing and Accountability are Being Circumvented

Introduction

On January 2, 2002, Defense Secretary Donald H. Rumsfeld issued a "Program Direction" memorandum ordering a new, far-reaching new effort to hasten development of the nation's multi-billion-dollar missile defense program. Proclaiming missile defense a national priority, Rumsfeld reorganized and renamed the military organization overseeing the program. Now, a new department called the Missile Defense Agency (MDA) will be responsible for the multi-tiered program, coordinating all aspects of missile defense, ranging from shorter-range advanced Patriot missiles and airborne lasers to intercontinental ballistic missiles intended to knock enemy missiles out of the sky.

While all the implications of the memo are not yet fully clear, the directive appears to give the Missile Defense Agency unprecedented unilateral power. It also exempts the agency director and defense contractors from longstanding acquisition statutes and regulations which ensure honest and cost-effective acquisition practices as well as objective research, development, and testing of Pentagon weapons systems.

Contrary to current weapons systems procurement practices and standards, the latest prevailing Department of Defense attitude toward missile defense is that any missile defense system, no matter what how limited its capabilities, is better than no missile defense system at all.

These program changes raise serious questions about whether the normal checks and balances that ensure financial oversight and sound weapons testing practices will be discarded. Here are a few aspects of the Rumsfeld directive that are troublesome:

• Under the new guidelines, the Missile Defense Agency will be exempt from regulations that require military commanders to set technical requirements for a new weapons system that must be met before it can be operationally tested and purchased by the Pentagon. These requirements, called Operations Requirements Documents, or ORDS, have been eliminated in favor of a less structured "capabilities-based" acquisition strategy.
• The Pentagon's chief independent tester, the Director of Operational Test and Evaluation, has not been given complete access to the Missile Defense Agency testing documents while the weapons system is being developed. Since the Rumsfeld memo was issued, DOT&E has been forced to negotiate for missile defense developmental testing data on a piecemeal basis, rather than being given customary access to any and all testing data. This new way of doing business - coupled with the Bush Administration's rush to deploy a missile system by 2004 - will greatly increase the chances of deploying a flawed system before DOT&E completes rigorous operational testing.
• The Rumsfeld memo authorizes the director of the Missile Defense Agency to negotiate so-called "other transactions" contracts. The Department of Defense Inspector General has noted that "other transactions" agreements exempt defense contractors from complying with many current acquisition statutes or regulations that ensure financial transparency, including the Federal Acquisition Regulation and its Supplements and Cost Accounting Standards.1

If Congress allows the sweeping changes ordered unilaterally by the Rumsfeld memo, the Missile Defense Agency will walk away with a blank check without going through normal channels that ensure the taxpayers buy the best possible weapons at the lowest possible price.

The Critical Need For Independent Operational Testing

The typical weapons system is exposed to two types of testing as it winds its way through the Pentagon's acquisition system. The first type, called developmental testing, is performed by the defense contractor and military service branch that will use the weapon.

The aim of developmental testing is to ensure that a weapon system is technologically sound. Developmental testing is conducted to assist in the engineering design and development of a system and to verify that developmental performance specifications have been met.

The other form of testing, called operational testing, is undertaken first by the services, but then handed off to the Director of Operational Test and Evaluation for an independent test, conducted under the most realistic battle conditions possible. DOT&E then determines if the weapon system is "suitable" and "effective" - that is, ready for full-rate production, and deployment by the military. By law, DOT&E is required to operationally test a weapon system before it can progress beyond low-rate production.

The DOT&E director's job is not only to conduct operational testing and an annual assessment of major weapons systems, but also to review the military services' test and evaluation master plans to ensure that operational test concerns are incorporated early in the life of a weapons system.

The Rumsfeld memo threatens to erode the power and independence of DOT&E which is the eyes and ears of the Secretary of Defense and Congress. So far, it is not yet fully clear just how much involvement DOT&E will have in the restructured Missile Defense Agency's testing blueprint, but the memo does depart from tradition, specifically granting all pre-production testing authority to the agency's director. Current DOT&E current chief, Thomas Christie, has admitted in Congressional testimony that he is not being given unfettered access to missile defense testing documents and data. Although Christie has a legal right to all testing records and data in the Department of Defense, he is said to be negotiating with the Missile Defense Agency to obtain the data he needs to evaluate the program.2

Christie recently told Congress he believed that he would ultimately be able to carry out his oversight responsibilities, but that his missile defense testing role was still "evolving" after a period during which his office "backed away" from some of its involvement while missile programs were being restructured late last year.3

At the March 13, 2002, hearing before the Senate Strategic Forces Subcommittee, Christie stated, "That is certainly different than the large programs such as F-22 or the other standard major defense acquisition programs, where we have statutory responsibility for deciding or evaluating the effectiveness and suitability of a system and the adequacy of the operational testing that took place before the system is, in fact, produced and deployed. So it is different. And we're evolving. We have evolved in this process. We haven't been there before."4

Although the Bush Administration says it hopes to deploy a rudimentary missile defense system in Alaska by 2004, Christie said that portions of the system are not sophisticated enough for his office to even plan for operational testing yet.

Since the mid-1980s, the DOT&E has been tenaciously fighting to ensure that the best weapons systems end up in the hands of U.S. fighting men and women. Along the way, there have been countless efforts by weapons program managers to cut back testing on their programs, but DOT&E has persisted in its drive to ensure that the weapons produced are useful in combat. In many cases, these efforts have yielded very visible successes, including improvements in the C-17 cargo jet, the M-1 tank, and the Javelin Missile.5

Early on, DOT&E intervention proved very helpful to the C-17A, the first major acquisition program that relied on early operational assessment as a decision-making tool. Although in 1988 the Air Force supported the low-rate initial production of 40 C-17s, the Secretary of Defense lowered the number to 10 because operational testing had not begun. The subsequent operational testing provided timely insight into potential problems with the aircraft that may have been missed during developmental testing.6

In later testing of the C-17, DOT&E pressed for further evaluation of one of the more valuable uses of the cargo aircraft, the transportation and deployment of large numbers of paratroopers into a combat zone. The Air Force approved full-rate production of the C-17 without even testing the aircraft for this function. DOT&E demanded operational testing and discovered that the air turbulence created in the wake of the aircraft, flying in close formation, caused the parachutes on soldiers dropping from the aircraft to oscillate, partially deflate, or collapse. DOT&E testing continues to determine if the Army standards of dropping a brigade and its equipment in 30 minutes or less can be achieved.

In operational testing DOT&E discovered that the upgraded M-1A2 tank was unreliable and unsafe due to unprompted main gun and turret movements, hot surfaces that caused contact burns, and unintended firing of the .50 caliber machine gun. After a series of follow-up tests, DOT&E demanded that the Army make design changes. Subsequent testing showed that the safety issues were corrected and that the tank is now operationally suitable.

The reliability of another weapons system, the Javelin Missile, also was greatly aided by DOT&E intervention. DOT&E insisted that the system undergo additional operational testing prior to a full-rate production decision in 1997 because over 50 design changes had been made to the system since initial operational test and evaluation in 1993. This subsequent operational testing ultimately led to a missile assembly design change that eliminated missile launch failure problems that had occurred when tested under realistic combat conditions testing.

Despite such successes, the recent events at the Missile Defense Agency are the latest threat to the office's authority. When Congress first authorized the DOT&E in 1983, the goal was to provide an impartial and independent review of the adequacy of testing performed by the military services and interpret the success or failures of weapons systems before they enter the production stage.

In the case of missile defense, rigorous independent performance testing is perhaps even more critical because the program has experienced numerous problems and faces tremendous technological obstacles.

Although DOT&E is not responsible for development or operational testing by the services, it does routinely monitor the military's testing. The hope is that with DOT&E involvement in the early development and testing of a weapons system, the weapon will be better prepared for the final exam administered by DOT&E when it is nearing full blown production and subsequent deployment in the field.

Despite more than 17 years of hard work by DOT&E, and its successful record, there is mounting evidence that the military branches routinely ignore its recommendations or do what they can to circumvent early testing oversight of weapons systems, according to a December 2000 study by the Defense Department's Defense Science Board (DSB).7

The DSB's report stated that "there is growing evidence that the acquisition system is not meeting expectations as far as delivering high quality, reliable and effective equipment to our military forces. In those cases where the testing is adequate, we fail to take the corrective actions needed based on the results of that testing. In many cases, we allow our acquisition programs to proceed to their next phases...when the test results we have gathered clearly indicate the systems are not ready."8

The report went on to conclude, "The DSB Task Force has come to the same conclusion as have many of the congressional responses in the past. The value of testing and evaluation is finding flaws or weaknesses as early as possible during development at the lowest reasonable cost."9

Defense Procurement Before DOT&E

Cutting DOT&E out of the early testing picture would turn the clock back to a weapons procurement era that produced many overpriced and under-tested systems. The following are reminders of some of the weapons failures before the Pentagon's DOT&E became a force inside the acquisition system.

The Sergeant York Gun: The Little Gun That Couldn't Shoot Straight

The Sergeant York Gun is illustrative as a historical case of what can happen when a weapon is not subjected to close development oversight and rigorous independent operational, or battle condition, testing. Named after World War I hero, Sergeant Alvin York, the Army's division air defense gun - or DIVAD - began development in 1978, years before Congress ever envisioned an independent Pentagon testing office. Yet, the DIVAD procurement bears a striking resemblance to today's Missile Defense Agency's fast-track procurement blueprint. In the late 1970s the Army also was racing to deploy the DIVAD very quickly and had adopted a "hands-off" acquisition strategy with minimum oversight.

The result instead was a rush to failure.

The Sergeant York Gun was conceived to fill an air defense void on the battlefront. It was intended to locate and shoot down armed helicopters and fixed-winged low-flying aircraft, as well as engage lightly armored vehicles, trucks, and personnel. Unfortunately, by 1982, it was becoming clear that the Sergeant York's computerized gun and radar system had at least two fundamental problems: It could not consistently locate, or hit, its targets.

It was then, during a preliminary demonstration of the prototype, that the Army learned that the radar fire control system failed to operate reliably, the graphic display unit failed intermittently, the hydraulics system leaked, and the armament feed system's performance was unsatisfactory. Yet, the Army continued to defend the merits of the DIVAD, permitting the contractor to perform reliability tests with virtually no Army oversight.10

In February 1982, the General Accounting Office told Congress, "It is not possible now to make a reliable assessment of the Division Air Defense gun's potential in combat. Little is known about how well DIVAD meets the Army's requirements for maintainability, logistics supportability, and ease of operation by the troops."11

But like a snowball rolling down a hill, even the discouraging performance of the Sergeant York Gun couldn't stop the Army from exercising two of three production options in 1982 and 1983 for orders totaling 146 units. It wasn't until February 1984 that the Army procurement contract office notified the contractor, Ford Aerospace & Communications Corp., that the performance on DIVAD was "totally unacceptable." The following year, the problems had grown too serious to ignore. In the summer of 1985, the Sgt. York Gun was officially declared ineffective and cancelled by then Secretary of Defense Caspar Weinberger.12

The General Accounting Office partly faults the Army's "hands-off" acquisition strategy for the failure of the DIVAD. It also believes the system was not subjected to adequate operational testing which in turn prevented accurate and complete information from getting to decision-makers.

"An example of the adverse effect of a long-standing concern with OT&E is illustrated by the Army's Sergeant York weapon program, namely that OT&E results were not available before the decision to begin limited production," the GAO concluded in a 1986 report. "The recent cancellation of the program emphasizes the adverse effect of a long-standing concern with OT&E - over a billion dollars was spent on an unproven system that ultimately had to be terminated."13

The Story Of The Unsinkable Bradley Fighting Vehicle

Like the Sergeant York Gun, the Bradley Fighting Vehicle was a classic example of the Pentagon's "buy now, fix later" philosophy that was so prevalent during the 1960s, 1970s, and early 1980s. In fact, when the decision was made in December 1979 to begin full-rate production of the Bradley Fighting Vehicle, limited test results had shown that the vehicle's armor couldn't even protect its occupants from hostile fire.14 Full vulnerability testing did not begin until 1980, a year after the go-ahead to buy the system was made.15

"This information [on the weapon's vulnerabilities] was not reported to key decisionmakers," the General Accounting Office said in a 1986 report. "Even though the system had been deployed, the vehicle's vulnerability is still a major concern as demonstrated by test results."16

Once again, a lack of independent testing allowed an unsafe weapon to be placed in the hands of the nation's fighting men and women.

The Bradley, named after the famous World War II general Omar Bradley who led the D-Day assault on Normandy, is an armored carrier that transports cavalry units and infantry units to and from the battlefield and acts as a scout vehicle for reconnaissance and security missions. Ironically, the Bradley was intended to replace the M-113 whose armor wasn't thick enough to protect a squad of troops from anything larger than small arms fire. For that reason, many soldiers refused to ride inside the M-113, instead choosing to ride to battle atop the vehicle.17

But the Bradley, first deployed in 1983, wasn't put though any live-fire testing to determine if it would be better able to protect its occupants than the M-113. Live-fire testing wasn't done until 1985 when it was discovered that the Bradley was highly vulnerable to anti-armor weapons.

It also was having performance problems with its "swim capability" (some Bradleys were sinking while attempting to transport troops over bodies of water), transmission, electrical systems, and integrated sight unit.18 From 1980 to 1987, a total of 11 Bradleys sunk or swamped during swimming training operations. In 1987, after a Bradley sunk at Fort Benning, Georgia, the Army suspended Bradley training swims worldwide until problems could be corrected later that year.

"The Army has been testing certain modifications designed to increase the vehicle's survivability," General Accounting Office Associate Director Mark E. Gebicke told a House Subcommittee in 1987. "As a result of these tests, the Army has decided to modify the approximately 3,200 Bradleys still to be produced and to retrofit many of the vehicles already produced with certain survivability enhancements."19

Since it was live-fire tested in the late 1980s, the Bradley's reliability has improved - but only after it has undergone a number of fixes and upgrades totaling billions of additional dollars. Yet, as late as 1992, nearly a decade after the Bradley was first deployed, studies by the U.S. Army Ballistics Research Laboratory had still not drawn any firm conclusions on the vehicle's survivability.20

The B-1 Bomber: Victim Of A "Buy Before You Fly" Philosophy

Originally conceived as a long-range, nuclear bomber in the 1960s during the height of the Cold War, the first of the B-1B Bombers was delivered to the Air Force in 1985. By the time the 100th - and last - copy of the supersonic aircraft was delivered in 1988, testing had revealed a number of critical shortcomings.

Once again, the Pentagon's troublesome habit of failing to operationally test its new aircraft until after deciding to begin production had taken a serious toll. In fact, initial operational testing did not even begin until three years after the aircraft's production decision was announced by then President Ronald Reagan. Rushing to deploy the B-1B by 1986, the Air Force decided to develop and produce the B-1B concurrently, a practice that the General Accounting Office has repeatedly criticized as too risky. With a $20 billion price tag, not only was the program costly, but it had a "compressed" development and production schedule.

It wasn't that the dismal test results were any secret. As early as 1983, the General Accounting Office said that flight tests on the earlier B-1A model between 1974 and 1981 had shown that the aircraft's defensive avionics countermeasure system never had time to mature to a level needed for operational testing.21

By the time the B-1B was fully deployed, testing had revealed that the aircraft's sophisticated avionics systems - its threat-warning defensive system and radar-jamming offensive system - were jamming each other. The result was that the pilot had to choose between protecting himself or carrying out his mission.

Three years later, in 1991, when the U.S. launched the air war against Iraq, the entire B-1B fleet was grounded due to catastrophic engine blade failures. Munitions limitations, inadequate crew training, and electronic warfare deficiencies also played a role in the B-1B not making an appearance during the Gulf War. That same year, it was publicly revealed that although the B-1B's mission included flying at high altitudes, its de-icing system didn't work, an essential component at high altitudes, did not work.

Since the Gulf War, upgrades to the B-1B have cost taxpayers billions of added dollars and some of the upgrade testing will continue for years to come. Upgrades have recently been estimated to total more than $11 billion.

The biggest factor in the B-1B program's problems resulted from its concurrent development and production, according to the GAO. "Air Force procurement regulations dictate generally that development, production, test, and deployment of a major weapon system be conducted sequentially," the report said. "Many of the problems being experienced today provide lessons about the risks concurrency poses for complex, high technology weapon system procurements."22

Nancy Kingsbury of the GAO, testifying at a Congressional hearing in 1991, said that the Air Force's acquisition strategy resulted in a situation where more than half of the B-1B fleet was delivered for deployment before testing had even begun on the aircraft's defensive avionics.23

"So you had a full-blown system out there in operation, accepted by the user, with a system with a major design flaw," Kingsbury said. "If you had had a different acquisition strategy, you might have discovered that somewhat sooner. You might have been able to recognize you couldn't fix it with that design architecture and had the opportunity and the funds available to go do something different."

The C-5: A Big Cargo Aircraft With A Big Price Tag

Investigative reporter Jack Anderson once called the C-5 cargo jet the "notorious granddaddy of Pentagon overruns."24

The C-5, first conceived in 1961, was intended to supplement the Lockheed C-141 Starlifter and to replace the Douglas C-133. Specifically, the military wanted a plane that could carry a payload of 100,000 to 200,000 pounds over intercontinental distances and one that could operate from semi-prepared runways. But events in Vietnam caused the C-5 to be rushed into production and operation before its fatal flaws were detected.25

The first "operational" C-5 took flight on June 6th, 1970. As the plane landed at Charleston Air Force base, a tire on one of the main landing-gear trucks blew out, and a wheel from another truck fell off and bounced down the runway.

In the years prior to this embarrassing incident, the C-5 had endured the largest cost overrun in history. One Department of Defense official, A. E. Fitzgerald, infuriated the Pentagon when he came forward and told Congress in 1968 that the 115 C-5s would cost $2 billion more than anticipated, due largely to a wing defect that was discovered only after production was long underway.

Even after the costly wing repairs were made to each of the 77 planes already produced (the wings were fully replaced in the 1980s), the C-5 was plagued by further problems with the landing gear, and the delays and problems continued.26

Originally, the Department of Defense planned to purchase the planes in three production stages, in lots of 57, 58, and 85 aircraft respectively. But when problems with the wing developed - and were finally noticed - 40 planes had already been built, and wing parts had already been manufactured for an additional 20 of the aircraft.

Lockheed agreed to fix the problem, but didn't follow through on the agreement, leaving the military with 77 defective planes. Although the Air Force wanted another 115 copies of the aircraft, President Richard Nixon reduced the number procured to 81 - and also gave Lockheed a $250 million loan guarantee so the defense contractor could avoid bankruptcy.27

Like other weapons systems nightmares, the C-5 program left the contractor in charge of research, development, testing, evaluation, and production with virtually no oversight.28 In the end, the gigantic cargo aircraft joined the Pentagon's procurement hall of shame, becoming another victim of a misguided acquisition strategy, this one known as the "Total Package Procurement" concept.

A Move Away From Financial Oversight

The Rumsfeld memo also opens the door for a broader use of special contractual agreements called "other transactions." These types of contracts waive many of the financial oversight requirements of typical contracts for goods or services with the aim of attracting so-called "nontraditional" defense contractors. The theory is that some of these firms have promising research and technologies to offer, but shy away from the complex requirements of more highly-structured Pentagon contracts.

While such "other transactions" offer the government and defense contractors flexibility, they are nonetheless ripe for abuse and often misused. "Other transactions" allow contractors to avoid taxpayer protections and transparency requirements in the Federal Acquisition Regulation (FAR) and Cost Accounting Standards (CAS). FAR essentially requires contractors to provide the government with cost or pricing data supporting their prices and to certify that the data are accurate, current, and complete. CAS ensures that accountants do not use creative schemes that allow contractors to overcharge the government. These important protections give the federal government the information it needs to ensure fair and reasonable contract prices.

An "other transactions" contract can even exempt a defense contractor from undergoing government audits or providing the federal contracting agency and government auditors with access to the contractor's pertinent records.

Unfortunately, two "other transactions" have already been awarded by the Missile Defense Agency since the Rumsfeld memo was penned. Both went to traditional, large contractors. The Boeing Co. was awarded a sole-source letter "other transaction agreement" not-to-exceed $23.9 million for system engineering and integration work and Lockheed Martin Mission Systems was awarded a sole-source letter "other transaction agreement" not-to-exceed $23 million for the development and integration of battle management, command and control, and communications capabilities.29

These two contracts do not represent the intent of "other transactions" agreements, according to the guidelines outlined by the Department of Defense Inspector General. The Inspector General has stated that "other transactions," first approved by Congress in 1989 and broadened in 1993, are intended to increase the technology and industrial base available to DOD and to foster new relations and practices within the technology and industrial base that supports national security.30

Inspector General audits have noted that although such transactions are intended to bring new defense contractors into the fold, the process has generally failed to meet this objective. The Inspector General found that, from 1994-2001, traditional defense contractors have received more than 94 percent of the $5.7 billion in funds paid for 209 prototype "other transactions" agreements.31 Lacking this purpose, "other transactions" are of no benefit to the government and do nothing more than remove financial oversight from otherwise typical contracts.

A 2000 General Accounting Office study agreed that most of the other transactions business was going to traditional defense contractors. Among the 10 largest of these agreements, nine went to traditional contractors for projects ranging from the Air Force's Global Hawk unmanned aerial vehicle program to the Navy's DD-21 Land Attack Destroyer.32

In response to this use of "other transactions", the Inspector General stated, "We find this trend disturbing, as other transactions do not provide the government a number of significant protections, ensure the prudent expenditure of taxpayer dollars, or prevent fraud. The traditional protections for the public trust do not exist, for the most part, for other transactions."33

A recent Inspector General audit concluded that "DOD officials were not always aware of the actual cost to the Federal Government for other transactions," and that, "DOD reports to Congress did not fully disclose the actual costs to the Federal Government for other transactions."34

The GAO report said that the typical other transactions agreements it studied regularly relied on means other than certified cost or pricing data to establish fair and reasonable prices, limited the governments audit rights, paid contractors based on accomplishment of agreed upon technical milestones rather than on an incurred cost basis, and did not provide the Department of Defense a right to terminate an agreement for default by the contractor.

The "Any Weapon Is Better Than No Weapon" Acquisition Strategy

At the heart of the Rumsfeld memo is a return to the fast-track procurement strategies of the 1960s, 1970s, and early 1980s that often resulted in the production of ineffective and overpriced weapons.

But the lessons of the earlier era have led to a new way of doing business. The normal path to successful weapons systems development and acquisition is now highly structured and sequential. Typically, when a weapons system is conceived, the military must first identify a need for it and set detailed requirements, or technical standards. These requirements - spelled out in what the military calls an "Operational Requirements Document" (ORD) - are almost always set by the Joint Requirements Oversight Council (JROC), a group of experienced military leaders representing all the services and chaired by the Vice Chairman of the Joint Chiefs of Staff.

The JROC, independent of the service or agency developing a particular weapons system, draws up the ORD based on the needs of the military, rather than on the needs or desires of the contractors. These documents then become a testing yardstick as a weapons system progresses through the research and development process and eventually goes into full-rate production.

More specifically, the JROC is required by law to evaluate the cost, schedule, and performance criteria of acquisition programs.35 Therefore, one of the JROC's important duties is to review and approve a specific weapons system's ORD that establishes the specific performance goals a weapon must reach before it makes it to the battlefield.

But under the new guidelines, the Missile Defense Agency will no longer be required to start the development process with specific military requirements generated by the users and formalized in the customary ORD. Instead, the agency will use a "capabilities-based" acquisition process for the nation's missile defense, guided not by military leaders, but by a political body known as the Senior Executive Council, chaired by the Secretary of Defense and including the civilian leaders of the three military branches.

The Bottom Line: The Missile Defense Agency is essentially committed to a subjective "anything is better than nothing" philosophy. Here's how DOD Acquisitions Chief Pete Aldridge recently explained the agency's new "capabilities-based" approach:

"It's different from an ORD, which will say to you, 'You will not deploy a system until it achieves a .95 kill probability against 10 rockets or 10 ballistic missiles within five minutes,' If General Kadish could say, 'I can shoot down eight of these missiles, it's going to take me six minutes, with a kill probability of .9,' it may be sufficient that we say that's a capability we believe we need, it's adequate for or needs to protect our country at this time against the threat."36

Ironically, this new capabilities-based, streamlined process, comes at a time when Pentagon tester Thomas Christie is making a push to eliminate the Army, Navy, and Air Force's ability to "waive" tests and operational requirements without first gaining DOT&E review and approval.37 Christie recently sent a memo to all the services instructing them to "cease the unilateral waiving of the operational requirements."

This waiving of requirements was a common practice during the acquisition of the troubled V-22 Osprey tiltrotor aircraft. During initial operational testing, the Navy identified and waived a total of more than 20 major deficiencies. For example, the aircraft was not cleared for icing operations or air combat maneuvering, it carried no defensive weapons, had an inadequate cargo handling system and airdrop capability, and was spending too much time in the hanger for maintenance and repairs.

The Navy still recommended that the aircraft progress to full-rate production. Although there was no evidence that these deficiencies contributed to two fatal V-22 accidents before the aircraft was temporarily grounded in 2000, these problems should have been a sign that the aircraft was not ready for production.

"The process of handling waivers seriously undermines the training and evaluation process - and may have already had negative impact on weapon systems," the 2000 Task Force Report of the Defense Science Board concluded.38

While he has not taken a public position on this new "capabilities based" approach, DOT&E Chief Christie, said recently that the new system will signal a change for the testing community.

"One of the features of this approach, up to this point, there are no hard and fast requirements, threat-based or otherwise, against which to measure the operational effectiveness or suitability of the system," Christie said in a recent speech.39

Despite his optimism for the future of missile defense, even Missile Defense Agency Director Kadish hinted recently that the missile defense program is still technologically immature and that its capabilities-based philosophy could raise some tough questions along the way. "Right now we have zero probability of intercepting a ballistic missile, and some judgment will have to be made as to whether anything greater than zero is useful," he said in a recent interview.40

In his speech, Christie focused on three weapon systems that have encountered delays and cost overruns - the Army's Comanche helicopter, the Air Force's F-22 tactical fighter, and the Marine's V-22 Osprey.

Christie attributed the problems of these three programs, and others, to a "lack of appreciation of the technical challenges faced by programs at their outsets."

"Clearly, we have failed time, and again, to do our homework early-on or to make the up-front investments required for an informed understanding of the technical and cost risks inherent in a program before we launched off into full-scale development and procurement," he said.

"Too often, over the past few years, we've rushed into operational testing when the results of development test and evaluation have clearly shown us that we were not ready and that our chances of success were minimal. In essence, we have been 'rushing to failure.'"41

Recommendations

The Director, Operational Test and Evaluation should not be required to negotiate the nature of information it is provided by missile defense program managers. The office should have unfettered access, and be an active participant in early testing of missile defense systems.

Congress must require the Pentagon to successfully complete DOT&E's operational testing before deploying a national missile defense system, consistent with the provisions of section 2399 of title 10, United States Code.

The Director of the Missile Defense Agency should be required to follow the intent of the law permitting "other transactions." The Director should only be permitted to negotiate such agreements with nontraditional defense contractors that would otherwise not offer their expertise.

Endnotes

1. Cost Charged to Other Transactions, Office of the Inspector General, Department of Defense, Report No. D-2000-065, December 27, 1999.

2. On June 12, 2002, Democratic Congressman John Tierney asked the General Accounting Office to conduct an expedited investigation into potential violations of federal law by officials at the Missile Defense Agency. Tierney said he made the request because he was concerned that the Missile Defense Agency was withholding testing data that DOT&E should be provided by law.

3. "Missile Defense Acquisition Policy," Testimony of Thomas Christie, Director, Operational Test and Evaluation, before the Strategic Forces Subcommittee of the Senate Armed Services Committee, March 13, 2002.

4. Ibid.

5. Test and Evaluation: Impact of DOD's Office of the Director of Operational Test and Evaluation, General Accounting Office, GAO/NSIAD-98-22, October 1997.

6. Test and Evaluation: The Director, Operational Test and Evaluation's Role in Test Resources, General Accounting Office, GAO/NSIAD-90-128, August 1990.

7. Report of the Defense Science Board Task Force on Test and Evaluation Capabilities, Office of the Under Secretary of Defense for Acquisition and Technology, December 2000.

8. Ibid.

9. Ibid.

10. The Army Should Confirm Sergeant York Air Defense Gun's Reliability and Maintainability Before Exercising Next Production Option, General Accounting Office, GAO/MASAD-83-8, January 27, 1983, p. 1

11. Tests and Evaluations Still in Progress Should Indicate Division Air Gun's Potential Effectiveness, Unclassified Digest, General Accounting Office, C-MASAD-82-7, February 26, 1982.

12. Weapon Performance: Operational Test and Evaluation Can Contribute More to Decisionmaking, General Accounting Office, GAO/NSIAD-87-57, December 1986.

13. Ibid.

14. Ibid.

15. The Pentagon Wars: Reformers Challenge the Old Guard, James G. Burton, Naval Institute Press, 1993, p. 135.

16. Weapon Performance: Operational Test and Evaluation Can Contribute More to Decisionmaking, General Accounting Office, GAO/NSIAD-87-57, December 1986, p. 27.

17. The Pentagon Wars: Reformers Challenge the Old Guard, James G. Burton, Naval Institute Press, 1993, p. 133.

18. Army Modifications to Improve the Bradley Fighting Vehicle's Survivability, Reliability, and Performance", Testimony of Mark E. Gebicke, General Accounting Office, before the Subcommittee on Procurement and Military Nuclear Systems, House Committee on Armed Services, December 17, 1987.

19. Ibid.

20. "Operation Desert Storm: Early Performance Assessment of Bradley and Abrams," General Accounting Office, GAO/NSIAD-92-94, January 1992.

21. The B-1 Bomber Program - A New Start, General Accounting Office, GAO/MASAD-83-21, April 13, 1983.

22. "The B-1B Aircraft Program," Testimony, Frank C. Conahan, Assistant Controller General, before the Subcommittee on Research and Development and Subcommittee on Procurement and Military Nuclear Systems, House Committee on Armed Services, February 25, 1987.

23. "The B-1 Bomber Program," Testimony, Nancy Kingsbury, Director, Office of Air Force Issues for National Security and International Affairs Division, before the Subcommittee on Legislation and National Security, House Government Operations Committee, March 6, 1991.

24. "Air Force Hid True Cost of Cargo Plane," Washington Post, September 28, 1982.

25. www.http://www.globalsecurity.org/military/systems/aircraft/c-5-history.htm, June 30, 2002.

26. "Foes Warn of Cargo Jet Cost Overrun," Chicago Tribune, January 25, 1982.

27. www.http://www.globalsecurity.org/military/systems/aircraft/c-5-history.htm, June 20, 2002.

28. DOD Defense Acquisition History, Project Newsletter, Spring 2001, Volume 1, Issue 2.

29. DOD Defense Link: Contracts from the U.S. Dod, February 19, 2002.

30. Cost Charged to Other Transactions, Office of the Inspector General, Department of Defense, Report No. D-2000-065, December 27, 1999.

31. "DOD Inspector General Comments on the Service Acquisition Reform Act (HR3832)," letter to Congressman Tom Davis from DOD Deputy Inspector General Robert J. Lieberman, March 12, 2002.

32. Acquisition Reform: DOD's Guidance on Using Section 845 Agreements Could be Improved, General Accounting Office, GAO/NSIAD-00-33, April 2000.

33. "DOD Inspector General Comments on the Service Acquisition Reform Act (HR3832)," letter to Congressman Tom Davis from DOD Deputy Inspector General Robert J. Lieberman, March 12, 2002.

34. Cost Charged to Other Transactions, Office of the Inspector General, Department of Defense, Report No. D-2000-065, December 27, 1999.

35. Chairman of the Joint Chiefs of Staff Instruction 3170, "Requirements Generation System," August 10, 1999.

36. E.C. "Pete" Aldridge, Under Secretary of Defense for Acquisition, Technology and Logistics, Testimony before the Strategic Forces Subcommittee, Senate Armed Services Committee, March 13, 2002.

37. 2001 Annual Report, Director, Operational Test and Evaluation, Department of Defense, February 2002, p.ii.

38. Report of the Defense Science Board Task Force on Test and Evaluation Capabilities, Office of the Under Secretary of Defense for Acquisition and Technology, December 2000.

39. "Test and Evaluation in the 'New World'," Thomas Christie, Director, Operational Test and Evaluation, speech in Savannah, Georgia, to the National Defense Industrial Association, February 26, 2002.

40. Defense Week, March 18, 2002.

41. "Test and Evaluation in the 'New World'," Thomas Christie, Director, Operational Test and Evaluation, speech in Savannah, Georgia, to the National Defense Industrial Association, February 26, 2002.