The Pentagon’s contracting gurus repeatedly made massive, preventable mistakes while managing contracts for a critical software project of their own, violating federal budget law along the way, according to scathing internal reports and other records obtained by the Project On Government Oversight. Ironically, the purpose of the mismanaged system is to help manage the rest of the Pentagon’s contracts.
The Pentagon relies on the obscure Defense Contract Management Agency (DCMA) to negotiate and administer $5 trillion in contracts across the Defense Department as an average of $455 million in taxpayer dollars are paid out each day to contractors. How well that agency does its job directly affects both how wisely taxpayer dollars are spent and whether our troops get what they need when they need it. Yet the cascading series of major, systemic failures within the agency, which has nearly 12,000 employees and an annual budget of roughly $1.5 billion, led an independent investigator from the Army to go so far as to recommend that it be prohibited from awarding its own contracts in the future.
Documents obtained by POGO—including a September 2017 preliminary investigative report, an April 2018 internal memo, and a draft of the final report from this summer—show that over the course of several years, the agency mismanaged contracts for the software project. The agency spent far more than it was authorized to spend, violated numerous policies and regulations, and received insufficient oversight from the Pentagon. When these documents are viewed alongside Defense Department watchdog findings from recent years, there appears to be an agency-wide pattern of failing to adequately support the Pentagon’s procurement efforts and sufficiently protect the taxpayer—the agency’s very mission.
At various points, the internal documents POGO obtained state that some of the agency’s foul-ups on the project indicate “a serious systemic problem within the organization” and raise “serious concern over contracting practices and processes within DCMA.” The draft final report states that many senior leaders set the agency’s procurement staff “and DCMA up for failure.”
After POGO reached out to the agency for comment, the agency launched a leak investigation into who provided POGO the information.
The project, called the Integrated Workload Management System, is a software system created to help review, implement, and track defense contracts. By the time the agency abruptly stopped all development work on the system in May 2017, it had awarded over $46.6 million in contracts related to the project over the course of three fiscal years (an agency spokesperson says the agency kept contract costs down to $44.9 million). At least $17 million of this was unauthorized—a violation of a budget law called the Antideficiency Act. Penalties for such a violation can include a prison sentence, though that is rare; however, the investigation found evidence that one employee had “willful intent to violate” the law. Those numbers also don’t include any in-house costs like government labor, which would likely add millions to the overall cost of the project. Because the agency never tracked or reported those numbers, the true cost of the project may never be known.
While the agency declared the software “operational” in September 2017 and is now using it, the software was not even halfway done when development was halted, according to the draft final report. This means the software does not provide the full capabilities it was supposed to provide, even though the agency spent more than what the project was estimated to cost.
The investigation was commissioned by the agency’s then-director, Lieutenant General Wendy Masiello, shortly before she retired from the government in May 2017. (In August 2017, she joined the board of defense contractor KBR.) She inherited the project when she became the agency’s director in May 2014. The investigation identifies nine responsible individuals, six of whom hold or held senior leadership positions: the former director of information technology, comptroller, director of contracting, former chief general counsel, and both Deputy Director Marie Greening and a former agency director, Charlie Williams. Several no longer work at the agency; to date, none of the nine have been formally disciplined. When reached for comment, an agency spokesperson stated that as the investigation was ongoing, “It would be premature to take any disciplinary action until the Agency has all the necessary information.” It is clear, however, that while these nine may bear the brunt of the blame for the mismanagement of the software project, the systemic nature of the problems points to larger agency failings.
POGO contacted or attempted to contact these current and former senior leaders and other individuals involved. Most did not respond to the requests for comment or referred POGO to an agency spokesperson. The agency’s former director of information technology, Jacob Haynes, who is criticized heavily in the investigative reports, told POGO that he attributed the agency’s failures to attempting to move too quickly through a new and complex process. He said he relied on the judgement of his subordinates, the legal department, and procurement staff for the decisions he made. He said he retired last year and was not pushed out of the agency.
“No one wanted to do anything wrong,” Haynes told POGO, explaining that he and his coworkers tried to do the best they could to make the project a success. “It was the most visible program at the time” inside the agency.
This story offers a window into the deeply troubled world of government IT contracting—and the troubled state of contract management writ large in the Pentagon’s contract-management agency. As Congress’s investigative arm wrote last year, “federal IT investments too frequently fail or incur cost overruns and schedule slippages while contributing little to mission-related outcomes.” The DCMA project is a poster child for such failures.
A “Conflict of Interest”
Like agencies across the federal government, the Defense Contract Management Agency sought to replace an old data system with something newer and more capable purchased from the private sector. For years, the agency used its Electronic Document Workflow system to help manage major defense acquisition programs. A budget document describes it as a “combination of custom written software and Commercial Off-the Shelf” technology. In 2011, someone sent a “SAVE Award” proposal to the White House in 2011 suggesting the government “Replace DCMA’s cumbersome and bug-filled Electronic Document Workflow (EDW) system” (these awards went to federal employees who proposed “cost-cutting ideas”). The agency sought a “commercial off-the-shelf” software solution it eventually dubbed the Integrated Workload Management System.
Senior leaders across multiple administrations have called for increased use of commercial off-the-shelf solutions to save the government the headache of developing new custom technologies, a process that can grow unwieldy, cost far more than anticipated, and take years. In case after case, though, projects sold as commercial off-the-shelf solutions turn out not to be. Instead, many resemble traditional development projects, with their accompanying problems. As it turned out, the Integrated Workload Management System became such an example—although that was far from the only reason the project was troubled.
In 2011, the agency got the ball rolling with a contract paying a company called Apprio to conduct a study on what a “Model Contract Management Office” would look like and to “identify relevant technology” it would use. The next year, Apprio’s contract was expanded to envision a replacement for the Electronic Document Workflow system.
Things started going south for the software project in mid-2013, when Apprio won a contract worth more than $3 million to begin testing and developing software as the replacement system. But because the agency had previously hired Apprio to help lay out the requirements for this work, under federal acquisition regulations the company was ineligible for contracts implementing its prior recommendations. Regulations specifically prohibit this so that a contractor is not tempted to misrepresent the needs of the agency in order to sell more of its own products or services. The preliminary investigative report found this was an “organizational conflict of interest violation” that raised “serious concern over contracting practices and processes within DCMA.”
Because the contract involved spending more than $1 million on a system the Defense Department uses to manage its resources, then-Director of Information Technology Haynes needed approval from the Pentagon’s deputy chief management official. The official rejected Haynes’ initial request, citing missing documentation and other flaws, and asked him to resubmit it. Despite not having the Pentagon’s approval, Haynes “knowingly had” the contract awarded to Apprio, according to the preliminary investigative report. But during testing over the next several months, Apprio’s software “failed to properly integrate” with the agency’s other systems. (The company continued working on other aspects of the project in a more limited capacity and was awarded other contracts in following years.)
The agency turned to another company, Discover Technologies, to develop a software solution. Even before this contract was awarded, the process for doing so was beset with flaws. According to the preliminary report, the documents Haynes provided to the Services Acquisition Review Board to get approval for the contract appeared “deliberately misleading.” The documents, submitted just two days before the end of the fiscal year, listed the upcoming year’s projected expenses as $960,000, leaving out estimates that the contract would likely cost up to $40.5 million over its lifetime. Had board personnel known that, the preliminary report says, they probably would have asked whether the Pentagon had approved the necessary funds—which the Pentagon had not done. Instead, the board received and approved the contract on September 28, 2013, and reapproved it each year through 2017.
Additionally, the board failed to conduct a formal review of the Discover Technologies contract before approving it, and the preliminary report says that board personnel consistently “did not understand what was being communicated” by Haynes and others in the IT department, “and therefore either stopped asking questions or failed to ask” any at all. This tendency extended beyond the Discover Technologies contract. “In the absence of any questions or objections,” the preliminary report continues, “approvals were rubber stamps,” with the board approving 95 percent of IT submissions on first attempt. (The agency restructured and renamed the board in June 2018, an agency spokesperson told POGO.)
If it had done a thorough review of the Discover Technologies contract before approving it, the review board might have caught numerous problems. For example, the agency used the wrong type of contract: a “Blanket Purchase Agreement,” which is intended for recurring needs for straightforward supplies or services, not the type of complex software modifications Discover Technologies would be performing. The agency mischaracterized the software as a “commercial off-the-shelf” product, which requires less oversight, despite the fact that those involved knew it required a significant amount of coding and development work. The contract’s requirements were also vague, with some overlapping with work already assigned to Apprio. In many cases, task orders within the contract were missing key components, such as statements of work and standards by which to measure performance, and the document detailing the work requirements didn’t include a list of specific objectives.
On the last day of the fiscal year, two days after the review board approved the Discover Technologies contract, the Pentagon approved spending about $6.3 million on the software project over the next year. Hours later, the agency awarded the contract. The last-minute approval, according to the preliminary report, was “contrary to regulatory and statutory intent.” The draft final report explains there was “pressure in contracting to get contracts awarded at year end,” leaving insufficient time to plan and properly craft the project’s requirements.
The independent investigation found that some of the project’s most significant problems stemmed from the structure and administration of the contract. The lack of clear objectives in the contract directly impacted performance, and the “lack of planning and strategic vision” led to money being wasted through duplicative work and cost overruns, according to the preliminary report, which notes other IT contracts had similar problems.
For the next three years, Haynes consistently spent more—sometimes far more—than he had approval for. The agency also used the wrong type of funding for the project. Haynes said he relied on legal guidance, which he described as “gospel” despite having not read it, according to the preliminary report. The investigation determined that the legal guidance was based on “two important false premises” that led the agency’s legal department to give bad advice justifying the use of the wrong type of funds.
When the independent investigator questioned those involved with the matter, “several witnesses were deliberately vague in responses”; the investigator also found “recordkeeping is a systemic and significant problem in certain areas” of the agency, including IT and contracting. The report notes that when interviewed, the primary contracting official “[could not] recall discussions from any meetings he attended.” Haynes, according to the draft final report, was “evasive” in interviews.
In a phone conversation with POGO, Haynes said he was forthcoming with the investigator.
As for why there were multiyear financial violations, the preliminary report attributes this in part to the fact that there were “essentially two budget offices with a separate one in IT,” and “there was no oversight of the IT budget personnel by the [agency] budget office.” This meant the agency’s own budget office didn’t know the agency was spending more money than it was authorized to. In a similar vein, Pentagon budget officials also didn’t adequately keep track of the numbers or talk to each other, so one office kept giving away more money than the other had authorized. Given this lack of communication, the draft final report indicates, the Pentagon wouldn’t know if other defense agencies were also spending more than permitted.
A recent lawsuit brought against the agency by a former Apprio consultant alleges that some within the agency knew the project “was at a substantial risk for failure” as early as May 2016. What is certain is that a major turning point in the software project came in November 2016 when contracting officials told Discover Technologies to stop new development in order to free up funds for the agency’s labor budget—in spite of the fact that the project was seen as central to the agency’s “mission execution.” A press release from that month boasts of the system’s rapid development with “minimal resources.” However, over the ensuing months, agency director Lieutenant General Wendy Masiello “became concerned with the spending...and what progress was actually being made,” according to the draft final report. The report describes Haynes as “stonewalling” and “not forthright” with the director. She finally ordered on March 8, 2017, that all development work stop. A few weeks later, she also chartered a “Turn Around Team” with the goal of resolving critical problems in the IT department’s financial management and contracting process.
The Turn Around Team reported some of its findings in May 2017, informing Masiello that the project had far exceeded its funding, used the wrong type of funds, and was far behind schedule. Masiello subsequently requested both an Inspector General audit and an independent investigation into the suspected Antideficiency Act violations. By then, the agency had spent around $45 million on the project, not including government labor costs. Two weeks after the Turn Around Team’s briefing, Vice Admiral David Lewis replaced Masiello. (“She retired at the end of her normal cycle for her rank and assignment,” an agency spokesperson told POGO.)
On April 25, 2018, the Pentagon’s Inspector General released the report that Masiello had requested. The watchdog examined a sample of the agency IT department’s contracts and reported that the vast majority were severely mismanaged: out of 14 contracts, the agency improperly administered 13. The report notes that Lewis agreed with the findings and has begun implementing its recommendations.
Previous Defense Department Inspector General reports have found fault with the agency’s contract management, beyond the IT department.
A December 2017 audit found the agency was sometimes failing to investigate why contractors provided defective parts or services for the Evolved Expendable Launch Vehicle—a multibillion-dollar Air Force rocket program. A February 2017 review found that agency “contracting officers may have inappropriately reimbursed [Department of Defense] contractors for millions of dollars in unallowable costs.” An October 2015 report says that, in 8 out of 21 cases examined, the agency didn’t withhold payments from contractors even though auditors had found reasons not to rely on the contractors’ “business systems,” which are used to track expenses and justify charges billed to taxpayers. The Inspector General says in these cases the government was “not protected” from potentially excessive contractor billing. A March 2015 report found the agency was violating its own policy by failing to escalate identified problems that went unaddressed in the troubled F-35 fighter jet program, which meant, according to the Inspector General, that F-35 program leaders “may not be aware of problems until they adversely impact the cost, schedule, and performance of the program.” An October 2014 report found “DCMA cannot demonstrate” that its negotiations in “at least $70 billion in Government sales result[ed] in fair and reasonable contract prices.”
DCMA’s shortcomings are not new, but it doesn’t appear that this under-the-radar, yet hugely important, agency has made big enough strides over the last decade. In 2009, the congressionally created Commission on Wartime Contracting wrote, “DCMA is not aggressive in motivating contractors to improve business systems,” which it described as “the first line of defense against waste, fraud, and abuse.” The agency’s “conservative approach results in little, if any, motivation for contractors to improve their business systems, and ultimately has a direct impact on the warfighting mission.”
In 2011, the Government Accountability Office attributed many of the agency’s struggles to “a seriously eroded workforce” stretched thin by the surging number and value of contracts in prior years due in part to the wars in Iraq and Afghanistan; a decentralization initiative that led to inconsistent guidance and practices across the agency that one official called a “free for all”; and the challenges posed by defense contractors’ increasing reliance on subcontractors. The agency said at the time it was taking steps to tackle these issues. This is the broadest review of the Defense Contract Management Agency by an independent government watchdog office in recent years.
While the agency has in many cases not been zealous enough in protecting taxpayer interests, the shortcomings are not across the board, and parts of the agency have at times seemed more willing to hold contractors accountable. The agency’s hard-hitting findings that Lockheed Martin had used a significantly flawed business system for the F-35 program, and the Pentagon’s subsequent willingness in 2012 and 2013 to withhold some payments from the company, is one example. Last year, it did the same with helicopter manufacturer and major defense contractor Sikorsky.
But the extent to which the agency was failing to properly manage contracts for even its own internal systems is coming into focus this year with the April Inspector General report and the documents POGO obtained. An April 2018 memo from the head of the agency’s compliance division describes additional violations of financial laws and rules in the IT department outside the mismanaged software project, including over $37 million in suspected separate Antideficiency Act violations—on top of the confirmed $17 million in violations found by the independent investigation. Although the Turn Around Team discovered these additional violations in December 2017, it took months for the agency to open another investigation. The memo acknowledges that the final sum will likely be significantly higher, as several parts of the department had yet to be reviewed.
When reached for comment, a spokesperson emphasized that the agency “self-reported” these issues and sought outside help to resolve them.
Because the violations did occur, and went unaddressed for years, an independent investigator concluded the agency’s problems were so systemic and serious that they—the Pentagon’s contract managers, who handle about $5 trillion in contracts—couldn’t be trusted to issue or administer contracts of their own.
After POGO requested comment from the agency’s spokesperson on these issues, an investigator from the agency’s internal audit office called asking how the information about the mismanagement of the software project was obtained. POGO refused to discuss the matter.
This same investigator appears in a DCMA video announcing a hotline for whistleblowers to make disclosures “in order to make a difference.” But by trying to root out employees who disclose mismanagement and violations of law and policy, this watchdog may end up scaring off whistleblowers when the agency is facing real troubles and needs them more than ever. The draft final report itself, after all, recommends “that DCMA develop and foster an internal process and culture that allows any employee to raise a funding concern to senior leadership.”