Click to Enlarge
SPOKANE, WA — When VA’s new electronic health record (EHR) went live at the Mann-Grandstaff VAMC in October 2020, the system did not have the metrics in place to allow the hospital to measure how the facility was performing, according to a recent study released by the VA Office of the Inspector General.
Those metrics were still missing a year later. It’s an issue so severe that the report calls into question the facility’s ability to maintain its accreditation.
EHR metrics looking at things like organizational performance, quality and patient safety, as well as access to care, are meant to provide a reliable reflection of the state of a healthcare organization. VA uses a collection of metrics known as the Strategic Analytics of Improvement and Learning (SAIL) model for its own internal and external benchmarking, identifying strengths and areas of improvement.
During its investigation, OIG inspectors learned that, of the 103 SAIL metrics, only 13 were fully available for the Spokane facility. On the VA’s public-facing website, there were no performance metrics at all for the facility in FY 2021.
“The OIG is concerned that the lack of organizational performance metrics precludes an understanding of actual performance and data-driven decision-making at the facility,” the report states.
These metrics also are used by the Joint Commission, the nonprofit that accredits the majority of healthcare organizations in the United States, to determine how well a facility is performing. One leader in VHA’s Office of Performance Measurement told inspectors that the 17 metrics needed for accreditation by the commission were unavailable at the facility.
“Another VHA leader told the OIG ‘absolutely not’ when asked about the facility’s readiness for an upcoming Joint Commission accreditation survey,” the report revealed.
The report places much of the blame on Cerner, the company that designed the EHR and has partnered with VA in the roll-out. According to VA staff interviewed for the report, Cerner failed to provide a series of metrics prior to the system’s go-live, and consequently the facility and manufacturer are still trying to catch up.
A facility leader quoted in the report stated, “Early on it was discovered that Cerner did not provide 20 reports with metrics the VA requested in the original contract. Later, we were told we could prioritize the reports we [the facility] had asked for since [go-live] to replace the 20 we didn’t get. Through the approval process, I think 12 were approved, and we are still waiting on some of them to be built.”
Another stumbling block was VA’s lack of access and understanding of the Cerner system prior to go-live. The commercial metrics that came pre-built in the Cerner system were different enough from what VA had been using in its legacy systems that they couldn’t simply be slotted in without significant adjustments.
The example provided by one hospital leader involved the average daily census metric, which identifies how many people are in the hospital every day. The legacy EHR collects that information based on a patient’s arrival time, while Cerner’s metric uses a single midnight check. This one difference in how patient data was recorded prevented VA from being able to use the Cerner metric.
Hospital staff told investigators that none of these “canned metrics” were validated before go-live, that most of Cerner’s metrics did not work as intended, and that 10-months after go-live staff were still figuring out which metrics were usable.
Spokane staff were also caught unawares by the sheer volume of data that the Cerner system provides. One VA leader reported that 10 days after the system’s launch, staff had “roughly eight times the amount of data out of Cerner than we have ever gotten out of VISTA.”
That included more than 1,200 tables and 25,000 individual data fields. Many of those fields were titled differently from the legacy system, meaning that even if the data was familiar, it was difficult to identify.
Staff tried to use the EHR to generate custom metrics, but encountered difficulty doing that, as well.
One facility leader explained to investigators, “For every one of these [new EHR metrics] reports, you must select all the required filters manually. If you select the wrong ones, you must start over. If you request too many services or too long of a date range, the data will just spin and never populate, so you must start over. Each report is different based on what it can handle. Sometimes the report will freeze and never work.”
As with previous OIG reports, the watchdog agency advised VA to address these issues before continuing the roll-out. VA recently paused any further expansion of the EHR, not because of this report, but because of other patient safety issues.
