Predictor or Informer? KLAS Examines EHR Satisfaction Tools

EHR satisfaction and burnout are some of the top concerns of healthcare organizations, as both greatly impact the user experience and productivity. So naturally, if there’s a way to predict and detect potential issues, people need to know about it.

KLAS’ Arch Collaborative team recently published a report examining what the data from Epic’s Signal can tell us about EHR satisfaction and burnout. The Signal tool can be leveraged by Epic organizations to gain access to user-level Physician Efficiency Profile data, “including in-depth data on how physicians use In Basket, orders, notes, and clinical review.” But many organizations have also asked KLAS about what Signal’s data can do in terms of predicting end users’ experiences.

In short, the Signal tool’s basic use is to inform informatics strategies and to compare users in the Epic community. In this report, however, the Arch Collaborative examines data from several Epic organizations to provide a clearer look into the predictive capabilities of the Signal data. While these metrics are specific to Epic, other EHR vendors do have similar efficiency profiles and data tracking abilities, and so we hope all readers find some value from this report.

Predictability is Not the Point

According to the report, “While Signal data is not designed to be a predictor of an individual physician’s EHR satisfaction or feelings of burnout, many high-performing Epic organizations in the Arch Collaborative have found success using their Signal data to spark conversations and guide interactions with physicians.”

The intent of this report is not to confirm whether Signal’s data correlate with EHR satisfaction and physician burnout; it is to reiterate the fact that the data is more informational than anything. If organizations are solely using Signal’s data to predict satisfaction and burnout, they are probably using the data incorrectly.

Organizations that are exploring the tool’s use and that are looking to use the data for other purposes should remember that the tool is better at informing decisions than making them. It can track usage, but it doesn’t tell people whether the usage is good or bad. If organizations want to use the Signal data to predict EHR satisfaction or burnout, they should use it in tandem with other programs and metrics to provide clearer insights.

Prediction Tool vs. Directional Tool

Although the Signal tool is not perfect and its data is not great at measuring certain things, the tool does do a pretty good job of tracking how much time users spend in the In Basket feature and tracking people’s usage outside of normal work hours. In that light, the data is marginally more helpful at predicting user burnout than it is at predicting EHR satisfaction.

Signal is more of a directional tool, providing questions to ask as opposed to direct answers. Some Arch Collaborative organizations have effectively used Signal data to predict EHR satisfaction and burnout, but the data wasn’t the only metric they used.

If an organization that uses Signal data notices that a physician is spending 50 percent more time in In Basket than his or her peers, the organization can use that as an opportunity to talk with that physician and figure out why. The physician may actually be struggling, or that person may just prefer to spend more time in the feature. Therefore, the Signal data doesn’t necessarily direct what an organization focuses on, but it gives them a direction on where to dive deeper so that they have a better understanding of where users need help.

Better Predicting in the Future

While Epic organizations shouldn’t solely rely on Signal data for predictions, its predictive capabilities may increase as Epic continues to develop the tool. Epic is responding positively to feedback and is trying to make changes to tool that reflect that feedback.

The data included in this report is still preliminary, and Epic and KLAS need more data points in order for the findings to become stronger and clearer. If you belong to an Epic organization and are willing to share your data with the Arch Collaborative, be sure to fill out this form.

Connor Bice is a Report Analyst with KLAS Research. To follow KLAS on Twitter, click here.