Click to Enlarge: Number of outages, performance degradations, and incomplete functionality incidents affecting the
EHR system by fiscal quarter.
Note: These totals reflect only major performance incidents for which Oracle Health or VA was responsible,
omitting any caused by other parties, such as DOD. The figure does not include loss of redundancy incidents
because these incidents represent no impact on the user. Source: VA OIG analysis of all major performance incident data maintained in Oracle Health’s Lights On
Network from October 24, 2020, through March 31, 2024.
WASHINGTON, DC — Legislators, agency watchdogs and VA leaders are in agreement: The department must stop investing in massive, multiyear IT projects whose intention is to solve a technology need in one fell swoop. Rather, VA needs to address those needs with modular projects that can deliver incremental results swiftly and regularly.
“The scale of the IT systems are massive and presents a lot of challenges,” explained Rep. Matt Rosendale (R-MT), at a House VA Technology Modernization Subcommittee hearing last month. “The department has repeatedly attempted to address these challenges with megaprojects that span years or even decades and cost billions of dollars. This system has not worked well.”
A prime example of this is VA’s EHR modernization project. Entering its seventh year, the system has been installed at only six medical centers at a cost of $10 billion. Another is VA’s Financial Management Business Transformation program. After multiple false starts, the system has only been installed in a handful of offices, and the cost estimate of the project has nearly doubled, Rosendale said.
“Megaprojects sound good in theory, but in reality they’re always overdue, over budget—and that’s unacceptable for veterans and the taxpayers,” he declared.
What Rosendale and technology experts are advocating for are projects that value agile incremental development, small-scale deployments with testing by actual users of the system and component-based architecture that can be upgraded and replaced in pieces without dismantling the entire system.
“When you take that … monolithic approach, you’re developing all your requirements at the beginning, the designing and developing the system. Then, once you deploy 5 or 7 years out, that’s when you realize it’s not going to meet your mission needs. It’s too much of a risk for these agencies to go on with that kind of approach,” explained Carol Harris, director of information technology and security issues at the Government Accountability Office.
Instead, VA should embrace iterative development for complex IT systems, which is where the private sector has been moving for years, she said. “It essentially breaks large monolithic IT projects into small, manageable pieces, so you’re developing functionality in weeks or months instead of years.”
Lynn Overmann, executive director of the Beeck Center for Social Impact and Innovation at Georgetown University, explained how such an approach might work.
“It starts with a small, nimble team conducting discovery sprints with the people who use the services to deeply understand their needs and the [problems] the software aims to solve,” Overmann explained. “By identifying high-risk elements early, such as whether data integration will actually function correctly, the product-development team can test and address critical components of the product before investing significant resources in a full-scale solution.”
This modular approach bypasses other, less obvious pitfalls of large-scale IT projects. Witnesses described how there’s a “last train out of the station” feeling with large projects, as department staff try to make sure the project includes the changes their department needs. This results in larger and more cumbersome initiatives. And if that project fails, agency morale takes a hit, and it becomes harder for VA to inspire confidence in the next attempt.
Kurt DelBene, VA’s chief information officer, agreed with that assessment.
“The whole notion that you can determine the cost of a program and the precise requirements for a program of a large size from the beginning is just very fraught,” he said. “Once we get involved in a project, things evolve. … And the technical complexity of the project invariable increases.”
A modular approach would allow the department to hone what they’re delivering and stack up a series of small successes quickly.
“That also builds success in people’s minds as well,” he said. “They see it rolling out. … They see it getting better.”
DelBene came to VA well after the EHR project was underway. He said that, if given a do-over, he would design it as a modular, incremental project from the start.
“Ironically, that’s where we are ending up,” he said, describing how the program has been installed at six medical centers and then paused to address problems. “In hindsight, that should have been the scope from the beginning. It should have been this first pod, and then we broaden it from there.”
DelBene also admitted that, in that do-over, a commercial system like Oracle’s would probably not have been the immediate choice. Rather, he would have looked first at modernizing VA’s existing VistA system.
“There have been failed attempts to modernize VistA,” he admitted. “But this notion of getting VistA into an incremental modernization where it’s evergreen—that would have been the leading candidate.”
However, DelBene added, the project is beyond the point where it would make sense to turn back.
“I think we are over the hump now, and the merits of [Oracle] outweigh relitigating this question of modernizing VistA,” he said.
Whatever VA’s technology modernization efforts look like next year, the agency will have to achieve them on what DelBene believes is an insufficient budget. Funding for development is down 99.2%, and funding for modernization efforts is down 66.5% in the proposed 2025 budget.
According to DelBene, it’s a maintenance budget focused on sustaining what VA has, rather than moving forward with new programs.
“On a positive front, we are focusing on execution,” he said. “We’re really figuring out how to make it work.”
