Randy Thomas

Every information technology (IT) project in health care seeks to achieve optimal return on investment (ROI). But exactly how do you measure and achieve this return? It's a common struggle for healthcare providers.

One way to meet this challenge is by employing a disciplined approach to metrics and a benefits-opportunity analysis during the inception of the project, through the go-live stage, and long after. Whether your objective is to reduce avoidable medical errors, improve patient satisfaction with your scheduling process, or reduce your net days in accounts receivable, implementing a consistent and meaningful measurement process is key.

Introducing Metrics

Metrics help identify and measure the current state and potential future of various projects. At least three universal truths can be applied to measuring the success of IT projects:

  • You can't manage what you can't measure.
  • IT projects should deliver measurable value.
  • IT projects should result in a documented benefits realization outcome.

The challenge is using metrics appropriately to achieve the desired gains. Many projects suffer from two common missteps: trying to measure too much and insufficient planning and/or identification of the measurement method.

One way to minimize the potential for these problems is to introduce a well-crafted metrics system at the very start of the IT project-preferably as part of the decision to select and implement the project. Identifying your metrics for success at the beginning not only keeps you focused on what you are trying to achieve, but also can aid in the system selection itself.

Implementing Metrics

The following steps help to determine and implement a metrics system:

  • Identify the objectives for the project and the data and practices that will measure that objective (e.g., reduce medication administration errors).
  • Identify the target rate for each objective (e.g., reduce medication administration errors by 50 percent).
  • Agree on a definition and calculation for each metric (e.g., a medication administration error is when one or more of the "Five Rights" [right patient, route, dose, time, and medication] is violated).
  • Define the time intervals during which to measure each metric.
  • Determine how to capture the data to measure the metric and who will perform this duty.

To define your objectives, develop a team that is well versed in organizational goals and understands operations across the enterprise. The key is to identify the metrics that clearly correlate to the overall objective of the project. For example, if you are implementing a new system for the emergency department (ED) and your goal is to reduce wait times, your key metric should be just that. Ideally, you should select no more than 10 to 12 metrics for any project-and it is perfectly acceptable to measure just two or three things. Make sure the metric is meaningful, but keep it high level and keep it simple.

It is also a good idea to have a mix of metrics, both "hard" (i.e., quantitative) and "soft" (i.e., qualitative). Metrics might include clinical quality, financial return, and satisfaction levels. For example, in addition to measuring the number of minutes of wait time, you might also measure patient satisfaction with ED intake and triage as well as employee satisfaction with the old and new ways of doing things.

When you are establishing your future metrics targets, such as decreased wait times, it is always useful to refer to any industry best practices, benchmark data, or published articles that can help you understand what other similar organizations have achieved. It is also critical to ensure that the executive sponsors of the project and all the members of the project team are in agreement on the targets and understand the benefits that should accrue to your organization if you achieve those targets.

Tracking Performance

Once you have determined your metrics, you should perform a baseline measurement. This measurement focuses on the current state of your process (e.g., ED wait times) before the IT project has been implemented. For example, you may want to determine the current average ED wait time or average wait times at different points during the day. Performing this initial measurement will provide a point of reference throughout the life of the IT project.

At this point in implementing your metrics system, it would also be helpful to integrate data measurement as an automatic function of your new system, thereby avoiding the labor-intensive process of collecting data. Part of your selection criteria in choosing a new system should include whether it will facilitate measurement.

After implementing your IT project, make sure you track its performance. Typically, you should repeat your measurement and analysis steps after operating the new system for 90 to 180 days, which should allow sufficient time for the IT project to demonstrate its effects. This stage is truly "where the rubber meets the road," demonstrating whether you are achieving your targets. If you are on track with what you planned to achieve, you know you should continue what you are doing. If things aren't on track, you have the opportunity to evaluate what might need to change so you can achieve your targets. You also should continue to perform measurements at specified intervals, perhaps every six months, so you can modify your IT projects as the needs of your organization evolve.

Determining ROI

Once the data are gathered, it is time to analyze the information and determine benefits opportunities for your organization. A benefits-opportunity analysis helps predict financial and nonfinancial benefits that will be realized from the implementation of an IT project. Continuing with the ED-wait-time example, the average wait times will likely vary by time of day. Are you targeting a relative reduction in wait times (e.g., 30 percent) or an absolute target (e.g., 45 minutes)? You also will need to determine what changes in resources and processes are required in conjunction with your new system to achieve your target. There will be an economic impact from decreasing wait times in terms of both resources expended and revenue.

With the metrics measured, you can now determine the potential benefit to the organization if you achieve your target. Your metrics can be used to determine the potential ROI for the project. This step links the timing and realization of benefits and costs to the implementation and rollout of the system. From this evaluation, you will be able to determine when you will see the ultimate financial benefit of implementing an IT project. Since you've identified your metrics, collected the baseline data, established your future targets, and understand what changes will need to occur to achieve those targets, you will now be able to perform a standard financial analysis to determine ROI.

Randy Thomas is vice president, Healthlink Incorporated, Houston. Her e-mail address is randy.thomas@healthlinkinc.com.

Did You Know?

One way to ease the system selection process is to ask vendors to provide documentation from their customers for those metrics in which you are interested. These data can prove useful when trying to determine whether a given system is going to help you achieve your goals.

Hang in There!

The first project in which you measure ROI with a structured process will be the hardest. Although the metrics you measure will change depending on the system you implement, the process of metric identification, definition, measurement, and analysis remain consistent. By properly applying metrics, you can clearly identify the benefits your organization realizes through its IT investments.

Publication Date: Monday, March 01, 2004

Login Required

If you are an existing member, please log in below. Username and password are required.



Forgot User Name?
Forgot Password?

If you are not an HFMA member and would like to access portions of our content for 30 days, please fill out the following.

First Name:

Last Name:


   Become an HFMA member instead