Three Reports on the F-35: One of Them Informative

A few days ago, Breaking Defense reported an $11.5 billion cost decrease in the F-35. The outlet proclaimed "This is no program estimate that critics might savage. This comes from the Government Accountability Office's definitive annual Assessment of Selected Weapons Report [sic.]." Breaking Defense also quoted GAO's report (titled "Defense Acquisitions: Assessment of Selected Weapon Programs") that the newly discovered reduction in costs was "due solely to efficiencies found within the program as no decrease in quantities was reported." (See pages 11-12 of the GAO report.)

It all fit in nicely with the ongoing advocate narrative on the F-35: costs are coming down; yes, software and a few other things are a problem, but despite a few bumps in the road, the program is headed in the right direction, especially on cost.

The news of cost reductions, validated by GAO, was surely welcome in the face of GAO's other report, detailing some of those software issues. They threaten to force a delay in the Marine Corps' plan to declare the F-35B "initially operationally capable" in 2015 and the Air Force's plan to do the same in 2016, or to require them to declare IOC with even less capability than had been originally-and very modestly--planned.

There's just one thing wrong with the new cost reduction report, reported in the opening sentence of Breaking Defense's report as a "major win" that was "just scored" by the F-35 program. As GAO explained in detail in the 10 page explanation of its methodology (on pages 149 to 159), its analysis compared the DOD Selected Acquisition Reports (SARs) that came out in early 2013 (dated December 2012) to the previous SAR that was issued in 2012 (dated December 2011). Or as GAO put it "We compared the programs that issued SARs in December 2012 with the list of programs that had issued SARs in December 2011."

In that more than year old SAR (dated December 2012), DOD reported that it had declared total F-35 costs to be lower than estimated in the previous SAR (dated December 2011): costs had come down by $10.8 billion in 2012 dollars, which translates to $11.5 billion in GAO's conversion to 2014 dollars.

And that's your savings, reported by Breaking Defense as something new.

Note also, this is not "GAO's F-35 Estimate" (as proclaimed in Breaking Defense's headline: "GAO's F-35 Estimate Plunges $11.5 Billion; EELV Costs Soar $28.1 Billion"), it is GAO's regurgitation of what DOD reported in its past F-35 SARs, comparing the 2012 numbers to the 2011 numbers.

Moreover, take a look at the reasons for the lesser costs in the newer (2012) SAR, described by GAO as "due solely to efficiencies found within the program." Those "efficiencies" are variously listed on pages 77 - 81 of the December 2012 SAR; they include subcontractor re-estimates of costs, revised (more favorable) inflation estimates, redefinition of customer (DOD) requirements, and realignment of where to charge costs. Rejiggering inflation numbers, lesser hardware requirements, and unverified subcontractor numbers are what GAO characterizes as "efficiencies." Let's be polite and describe GAO's characterization as incomplete.

Note also that what we are getting is subcontractor and other numbers reported by DOD (with or without independent verification), re-reported by GAO (apparently without any verification or analysis, none of which appeared in the report). And finally, all of it misinterpreted to us by Breaking Defense.

Meanwhile there is more informative reporting on the F-35, mostly uncovered by the media, except for the revealing work of Jason Sherman at InsideDefense.com. (Behind a paywall, the article is available on request.) Sherman summarized most of a new annual report from DOD's Deputy Assistant Secretary of Defense (DASD) for Systems Engineering, Stephen P. Welby. His March 2014 "Systems Engineering Annual Report" confirms some of the bad news we have already heard from other DOD testing reports, and the report implies some important contradictions to GAO's prognosis of declining F-35 costs and that there is anything approaching new "efficiencies" that can be understood to justify an estimate of lower cost being in hand. The bad news comes in four categories: Performance, Reliability, Software and Manufacturing, as follows:

• The F-35 is hiccupping on one of its Key Performance Parameters (KPPs). It happens to be one that many ignore but which is integral to the aircraft's ability to engage in combat. The issue is the F-35's "sortie generation rate," or how often it can fly in either combat or-very important-training. In addition, there are problems in other KPPs where DOD has already relaxed the standard, and there are numerous "non-KPP" thresholds where the F-35 is having problems. DASD Welby puts it as follows:

o "Performance: The program is on track to meet seven of the eight KPPs. An issue with incorrect analysis/assumptions is hampering the attainment of the sortie generation rate (SGR) KPP. The program office is examining the sensitivity of the SGR KPP to establish more operationally realistic ground rules and assumptions. As a result, the program plans to reassess SGR. Although on track, the combat radius, STOVL performance, and CV recovery KPPs have limited margins. During a requirements review this year, the program determined of 62 non-KPP ORD thresholds, 16 are not achievable by the end of SDD based on the current plan, and eight others are at risk of not achieving the threshold. The program identified corrective actions or has way-ahead recommendations."

• In addition and as we already know from previous test reporting, the F-35 is not meeting its interim standards for reliability. Note that as DASD Welby states, this can mean more, not less, cost. As Welby puts it -

o "Reliability: Reliability data are below growth curves for all variants, and the program could face a risk to meeting reliability requirements without dedicated funding for a reliability growth program. Similarly, since O&S costs are based on meeting the required reliability at maturity, there are increasing risks to O&S cost and future aircraft availability. The program does not plan to complete prognostics portion of the Prognostics Health Management (PHM) requirements within SDD."

• We know from a different GAO report, which goes more seriously into the issue at hand than the GAO report discussed above, that the F-35 is having serious problems with its all-important software. Note that Welby projects an actual delay in the Marines' and Air Force's IOC plans, while GAO only asserted the possibility of it (which F-35 program manager Lt. General Bogdan tried to minimize to as little as 30 days). Welby says -

o "Software: Software delivery for the remainder of Blocks 2/3 is a challenge because of the size and complexity (~28.9 million software lines of code (SLOC), with ~2 million SLOC remaining). DASD(SE) forecasts a schedule delay for Block 2 and a delay for Block 3. As a result, the program improved software processes but also shifted resources to Block 2 at the expense of Block 3."

• DASD Welby's findings on manufacturing "efficiencies" are hardly grist for a declaration that cost reductions are in hand. While he cites "progress," his caveats are so serious as to question whether the direction of cost estimates should be up, rather than down-at least until there is a proven, empirical basis for declaring any savings whatsoever. Nonetheless, both DOD and GAO find the basis for a rosy future. More wary, Welby states -

o "Manufacturing: There was steady manufacturing progress in FY 2013, but quality, scrap/rework/repair, on-time part delivery, supplier execution, and reduced funding for future affordability initiatives are issues that may have an impact on costs for LRIP ramp-rate increases and FRP. In addition, there are production risks including part-interchangeability variation and fix schedule, outer-mold-line control, and maturing international capabilities. DASD(SE) participated in two supplier reviews and the annual prime contractor PRR. There was improvement from the previous year, but there are risks remaining for all eight manufacturing areas assessed. Mitigation plans are in place or in development for all production issues, risks, and PRR findings."

Sometimes, reports serve only the purpose of informing us what the advocates happen to be pushing and who or what they have enlisted in their efforts. Having worked at GAO for nine years and witnessing all too close the mentality of some senior management there, it does not surprise me that GAO's-shall we say--incomplete descriptions of DOD's numbers have the effect of enabling the advocates and their mouthpieces.

We should thank InsideDefense.com for bringing the Systems Engineering Annual Report to our attention. Needless to say, we can ignore the rest.