Putting Innovation To The Test, Part 2: Is It Effective?

The first part of this piece introduced the concept of testing innovations, focusing primarily on the safety aspects as well as the potential burden it can present for physicians, using the same framework that has been adopted for clinical trials. This segment will examine how physician executives can determine whether an innovation will be effective.

Phase 2: Is the Innovation Effective?

The innovation’s testing in this phase should focus on Everett Rogers’ model of innovators and early adopters. The goal of this pilot testing should be determining if the innovation can successfully be incorporated into existing workflows, or if its benefits are large enough to trigger physicians to abandon historical processes. In my experience, there are only a few innovations that have led physicians to adopt new workflows without forcing functions (e.g., EMR access through smartphones, voice-to-text). Unless the innovation shifts a physician’s paradigm, the physician executive may find the proposed change more easily accepted if it conforms to how physicians currently perform their work.

This testing phase should include physicians of different specialties working in different care settings. If the organization relies on volunteers for pilot testing, the physician executive will not gain insight into real or imagined barriers that might prevent the innovation’s adoption from less willing participants. Including thought leaders in this phase can help determine what features of the innovation or subsequent workflow changes might be worth communicating in the next testing phase.

The physician executive and the team responsible for delivering the education about the innovation to physician end-users should consider developing training materials by specialty and workflow to help orient the next group of physician testers as to how the innovation should be applied within their daily work. Without developing these orientation materials, physicians without a vested interest in the innovation’s successful implementation are likely to revert to their “usual” way of accomplishing their work after the training session is complete.

Phase 3: Is the Innovation Effective in a Broader Physician Population?

The goal of this testing phase should be to test the educational materials and intervention in a broader group of users. This phase allows the physician executive to gauge specific metrics that will need to be addressed before widespread implementation:

1. How well is the innovation adopted by the “average physician user?”
2. How much work does it take to support the innovation from the “frontend” and “backend” teams?
3. What is the expected range of the innovation’s adoption?
4. What are the expected benefits from an organizational perspective (e.g., safety, quality, patient satisfaction)?

This information should be presented to the health system’s senior leadership team to verify that the innovation delivers enough benefit to justify the cost to deploy across the organization. A secondary objective of the presentation to the leadership team would be to frame the innovation’s benefits as:

1. What benefits might be expected without explicit senior leadership endorsement including forcing functions (smaller) and;
2. What benefits might be expected with explicit senior leadership endorsement including forcing functions (larger).

Presenting the innovation’s benefits in this way allows the physician executive to gauge how enthusiastic the senior leadership team is about supporting the innovation against the physicians who will resist the change.

At this point, the physician executive should support the innovation’s deployment through whatever channels exist to update the larger physician community. He or she should be prepared to explain the “why” in multiple forums and acknowledge the difficulty in making the change, regardless of the innovation’s magnitude.

Phase 4: Measuring the Innovation’s Effectiveness after It Has Been Widely Deployed

Once the innovation has been deployed, the physician executive should verify that:

• The innovation is still being adopted as expected from the “phase 3” testing
• Unanticipated consequences are not occurring secondary to the innovation
• The organization is realizing the innovation’s expected benefits

Savvier individuals will use the information to highlight the various teams’ contributions to the innovation’s adoption and sustained effect on organizational metrics. When possible, the data may generate hypotheses to develop new innovations to further improve organizational performance.

This post has mapped health IT innovations against the same clinical trial framework used to test new interventions by the NIH and FDA. Applying a higher level of rigor to innovation testing before widespread implementation could reduce the likelihood that an innovation is deployed without the expected level of adoption and subsequent improvements in organizational performance. Over time, organizations with a structured innovation management process may have more resources to devote to addressing organizational needs than organizations that are more permissive with organizational innovations.

[This piece was originally published by Anupam Goel, VP of Clinical Information at Advocate Health Care. To view the original post, click here. Follow him on Twitter at @anupam1623.]