DoD Test and Evaluation Process Adds 2+ Years to Every Software Release

defense0 views
Before any software system can be deployed to warfighters, it must pass through operational test and evaluation (OT&E) conducted by the Director of Operational Test and Evaluation (DOT&E), an independent office that reports directly to Congress. OT&E was designed for hardware systems — testing whether a missile hits its target or an aircraft meets performance specifications. Applied to software, OT&E imposes a testing paradigm that assumes software is a finished product to be validated, rather than a continuously evolving system. A single OT&E cycle takes 12-24 months and must be repeated for each major software release. The consequence is that warfighters receive software updates years after they are developed. A bug fix written in January 2024 might not reach the fleet until 2026 because it must be bundled into a major release, scheduled for OT&E, tested over months, findings adjudicated, and then approved for deployment. In the commercial world, software companies deploy multiple times per day. In the DoD, deployments happen once or twice per year. The gap between commercial and military software delivery speed is not narrowing — it is widening, because commercial practices are accelerating while DoD processes remain static. OT&E persists in its current form because DOT&E has strong institutional incentives to maintain its role. It is a congressionally mandated office that justifies its existence by finding problems in weapons systems. If DOT&E endorsed continuous delivery with automated testing, it would be endorsing a model that reduces the need for its own independent evaluation. Congressional defense committees value DOT&E as an independent check on service acquisition programs and are reluctant to reduce its authority. The Goldwater-Nichols Act of 1986, which established DOT&E's independence, would need to be amended — a heavy legislative lift. The DoD's own DevSecOps initiatives (Platform One, Black Pearl, Party Bus) have demonstrated that continuous integration and automated testing can deliver software rapidly while maintaining security. But these platforms operate in a gray zone — they have not fully replaced OT&E, and programs that use them still face pressure to undergo traditional testing for major milestones. The cultural shift from 'test at the end' to 'test continuously' requires not just new tools but a fundamental change in how Congress oversees software acquisition.

Evidence

DOT&E's FY2023 Annual Report documented OT&E timelines averaging 18 months for software-intensive programs (https://www.dote.osd.mil/). The Defense Innovation Board's 2019 software report recommended replacing traditional OT&E with continuous testing but DOT&E objected. Platform One (https://p1.dso.mil/) demonstrated continuous delivery for Air Force programs. The Kessel Run program deployed software to KC-46 tanker operations on 2-week cycles, bypassing traditional OT&E through a tailored test approach approved by the Air Force Operational Test and Evaluation Center.

Comments