Lynn Harold Vogel
When it comes to investing in health IT, one rule is paramount: never proceed without clearly articulating the types of financial and nonfinancial returns you expect to achieve.
At a Glance
With many types of financial investments, the hospital financial community has long-established, and relatively straightforward, practices for projecting and measuring returns. Not so with health IT investments, however, where returns can be difficult to identify and quantify and even harder to track. Yet the success of such an investment depends on measuring the returns-whether they be tangible or intangible, short-term or long-term-because effectively tracking returns is the only way to be sure of achieving them.
1: to commit (money) in order to earn a financial return
2: to make use of for future benefits or advantages
Merriam-Webster Online Dictionary
Whenever you make an investment, whether it's constructing a building, purchasing an MRI, building a new cath lab, or selecting a new information system, you should expect some type of return. Indeed, most hospital leaders regard the size of the potential return as one of the primary considerations in setting investment priorities.
Investments in health IT pose a particular challenge, however, because it's hard to know exactly what kind of returns to expect, how extensive they might be, and how long it will take to achieve them. The uncertainties surrounding the potential returns on any major investment in health IT are enough to make hospital leaders hesitate to take the plunge.
Still, the reasons for investing in health IT are compelling. The technology can improve healthcare effectiveness and efficiency by enhancing the workflow of physicians, nurses, and other clinicians. And because of its ability to enhance patient safety and reduce costs, health IT has been hailed by the president, Congress, and industry leaders as a critical element in the formula to "save" the U.S. healthcare system.
The problem is negotiating the right path to achieve such results. Implementing an advanced clinical computing solution, such as computerized physician order entry or a nursing documentation system, in a way that actually promotes more effective and efficient health care, requires a structured, disciplined approach. And it typically requires that most of a hospital's staff members, including physicians and nurses, fundamentally change the way they perform their work.
What Kind of Return Should You Expect?
In general, the returns you should expect from a health IT investment should be measurable and achievable. Such returns might include improved patient safety and throughput, increased labor productivity, enhanced revenue, and improved workflow processes. Identifying the specific returns to expect from a given project can be challenging, however, because not all health IT investments are alike-even if the purpose is simply to automate manual processes.
For example, measuring the returns on an investment in a patient accounting system capable of processing large volumes of structured transactions may seem relatively straightforward-bring in the computers, and you reduce staff working in the patient accounting area. The financial effect of the staff reductions can be measured directly and easily. But what about measuring the financial effects of generating a more accurate bill, or of sending it more quickly? How do you measure the financial benefits of improved patient satisfaction resulting from a better managed billing process?
Measuring the financial benefits of IT systems that automate work processes in clinical departments such as radiology, clinical laboratory, and pharmacy poses a different challenge. Here, too, you can anticipate the effect of staff reductions. But it's not so easy to measure the financial effects of improved inventory management and tracking the flow of products, such as films, specimens, or medications. Also difficult to measure are the benefits of improved satisfaction of physicians who are able to view clinical data much more quickly and easily, and of patients who receive faster responses from their physicians because the data are more readily available.
Moving up the ladder of IT investment complexity, many providers are investing in enterprisewide clinical systems, such as computerized physician order entry or bar coding at the point of care, which focus on improving quality and patient safety-an important, but even more difficult, return to measure.
Other types of IT investments, such as clinical documentation systems, may enable nursing staff to be more productive. But the new system may only free nurses to perform important tasks that had previously been left undone. So instead of nursing staff reductions, the financial benefit may be higher quality nursing care. Such returns are indirect, accrue over time, and involve multiple interdependencies. Installing a clinical system that enables staff to better document their activities on behalf of patients, for example, may reduce costs of malpractice insurance premiums, but the actual premium reduction may be five years out.
Improved employee satisfaction as a result of a health IT investment can also result in cost avoidance-a kind of return. The University of Texas M.D. Anderson Cancer Center in Houston, for example, estimates the cost of bringing a new employee "up to speed"-including orientation, training, and the time required for the new employee to become fully productive-is about $20,000. But how do you track and record the $20,000 "savings" for each employee who decides to stay? It's difficult, if not impossible, to measure the degree to which improved job satisfaction due to an IT investment will determine an employee's decision to stay.
In short, reducing employee turnover obviously can be a real and measurable benefit, especially in a service industry with high-stress jobs. But in the analysis of potential returns conducted to justify the investment, there's no practical way to account for that benefit in terms of specific dollars.
Choosing the Right Metrics
One of the touchstones for success of a health IT project is identifying the key metrics that will define success-along with the process and time frames for measurement. It's crucial that a discrete set of metrics be identified at the onset. These metrics should include quantifiable financial returns in addition to "softer" types of returns that may yield financial benefits down the road but are harder to measure. Improved employee satisfaction, for example, could be identified based on changes from an established satisfaction baseline.
Consider, for example, the following metrics that could be identified to assess the returns from a computerized physician order-entry system.
Reduction in duplicate lab orders. Measure the rate of orders before and after implementing CPOE for a specific unit, and use the time increment between orders that are the same to determine whether orders are duplicates. Dollars saved can be calculated based on the cost per test and the reduction of duplicates.
Reduction in turnaround time for pharmacy orders. Measure the time it takes from when a physician orders a medication to when it's administered to the patient, before and after CPOE implementation for a specific unit. Although the financial return is indirect and the effect on quality of care is difficult to measure (the assumption being that speedier interventions result in a better therapeutic response for the patent), the improvement in efficiency is a real, and measurable, benefit.
Elimination of medication order transcription in the pharmacy. Measure the time spent on medication order transcription before CPOE implementation and after each unit goes live with CPOE. The time spent should decrease to zero, eventually resulting in reduced staffing requirements.
Reduction in days in accounts receivable. Measure the days in A/R before and after implementation. Improved access to timely physician orders and documentation by the medical records department may improve and speed up coding, thereby reducing the days in A/R attributable to this department.
Reduction in the lag time (in days) between a physician order and the time of billing. Compare the lag times before and after implementation, and between departments. A well-installed CPOE should significantly reduce this lag time. The comparison of lag times also can help identify substandard processes, allowing them to be acted upon quickly.
Reduction in lost charges. Audit charge capture before and after CPOE implementation. This metric should be significantly reduced with CPOE, as charge capture reconciliation processes will most likely be automated, with exceptions quickly identified and resolved.
Other metrics for evaluating the return on a CPOE system might include improved physician satisfaction, reduction in insurance denials due to physician documentation on orders, and improved data on physician productivity.
Dealing with Change
Ensuring a project achieves the initially projected financial returns becomes a management challenge as the project gets under way. Perceptions and assumptions about an IT project can change almost from the start, and the investment returns initially identified for an IT system can change as well. Such changes are likely to occur as you learn more about the system's actual capabilities in your environment and discover additional, unforeseen benefits. Always be willing to reexamine your initial assumptions about the projected return on investment, and make adjustments as necessary.
Avoid the pitfall, however, of allowing the focus to shift from tracking the initially projected returns to negotiating a continually changing "path to the dollars," in which the projected returns change with every unanticipated turn. Even if you must modify the initial model, you should continue identifying and measuring the initially projected returns from the IT investment. To this end, it's helpful to keep to your original set of metrics. You can always add metrics as the systems additional unanticipated capabilities become apparent. For example, your focus for implementing a positive patient identification system for bar-coded bedside medication administration might be reduction of adverse drug events. However, you may realize over time that the system has the added benefit of improved patient satisfaction because patients feel safer.
The challenge of monitoring progress on a health IT investment highlights the difference between such an investment and most business ventures. If you are a CFO and are considering investing in a business venture, it's customary for you to prepare a pro forma statement. If the pro forma doesn't look good, or if the venture doesn't measure up to the pro forma early on, you probably would not hesitate to discontinue the venture. Once an IT investment is under way, however, it's likely you will have changed organizational processes, trained as many as several hundred people to change their behaviors, and invested several million dollars. There's no easy way to call a halt to such a project. Although it's not out of the question to simply stop the investment, many healthcare organizations find such a decision too painful. They often are more inclined to increase their initial investment or simply try to "muddle through," in the hope the expected return will come eventually.
Again, the best way to avoid such circumstances is to start with clearly defined projected returns and to make a commitment to implement all the workflow changes required to take full advantage of the new technology. Refocusing implementation efforts on those process changes that directly effect the project objectives (that is, the metrics) is one of the best ways to get a derailed investment back on track.
The Retrospective Review
It's good business practice always to perform a thorough retrospective review after each IT investment to determine the extent to which expected returns were actually achieved. Too often, hospitals fail to ask objectively, "Was this project really worthwhile?"
To ensure objectivity, it's best to have an independent third-party perform the review. There's an inherent conflict of interest in having the returns reviewed by the same people who made the initial investment decision or led the system implementation. It will be hard for them to shift from the "sales" mode of promoting the project, where they are staking their position on the expectation of positive returns, to the "audit" mode of assessing whether their assumptions were correct. In the words of one industry observer, "It's like having the umpire and the batter be on the same team!"
Also, as was noted earlier, health IT investments often deliver returns over time. Even after an IT system implementation has been completed, the vendor will likely provide updates-sometimes on a monthly, quarterly, or semiannual basis. Sometimes, these enhancements can change the functionality of a system, which can affect the return on investment as well. The return on investment also may increase as staff members incorporate the system's functionality into their workflow, and their knowledge of the system's features and functions increases.
With more complex systems, such as enterprise clinical IT systems, the project may involve so many long-term intangible benefits that it's difficult to determine exactly when the project could be deemed a financial success. This challenge underscores the need to start with a clear set of metrics, both tangible and intangible, at the outset.
There also is always the risk that the retrospective review of a completed IT project might lead you to conclude that the returns are insufficient to have justified the investment in the first place. Unfortunately, being the messenger with such a conclusion is often a thankless task. Be sure you always encourage an honest assessment, without threat of recriminations, and are ready to admit to and learn from mistakes.
The Importance of Accountability
As important as calculating and monitoring returns on a health IT investment is assigning responsibility to someone to see that the returns are achieved. When making this assignment, it's important to keep in mind that, with almost all health IT investments, the true source of returns is not the IT system itself, but what the system enables the organization to do differently. Moreover, these "enabling" effects usually occur in the business or clinical areas of the organization, outside the IT department. For this reason, accountability for investment returns should almost never reside with the IT department alone. Rather, such accountability should, in the very least, be shared between the IT department and representatives from the departments affected by the investment.
Assigning shared accountability addresses a consideration that's of paramount importance: The success of an IT project in achieving its intended objectives and return depends on how decisions are made and who is perceived as a project owner. Certainly, IT staff should be intricately involved at every stage, from initial planning through implementation, and ongoing for updates. But an IT investment that's perceived as "just another IT project," with no buy-in and ownership from those who will be using the system, will almost certainly fail. Thus, ultimately, financial system investments should be led by financial staff, clinical system implementations led by clinicians, and so forth. Only then can you be assured that return targets will be appropriately identified and there will be an ongoing commitment to achieving them.
Patience and Perseverance
Automating a hospital's clinical and administrative processes can bring significant financial returns. You just have to identify what you want to achieve, set the metrics and mechanisms to measure success, balance discipline through the life of the project with flexibility to adapt to new opportunities, and objectively review what worked and what didn't. Remember, the financial returns associated with health IT projects are not always immediate and direct-especially with enterprisewide clinical systems. But that should not stop you from making the commitment to track progress toward those hard-to-measure intangible returns, in addition to the financial returns that are easier to measure. In the end, such an effort will raise your entire organization's awareness of what it means to effectively implement a new health IT system.
Action Steps for Health IT Investment Success
Four steps are key to effectively defining and measuring return from a health IT investment.
Define specifically what you expect from the investment. Are you looking simply for staff reductions? Improved productivity? The ability to produce a better product or offer a higher quality service? A reduction in liability or risk in the products or services you currently offer? In some cases, returns that cannot be easily measured quantitatively may nonetheless be the most important returns to consider.
Evaluate returns using the most appropriate metrics. Such metrics may be easy to track, such as staff reductions. But it's also important to identify metrics for measuring intangible benefits, such as improved physician satisfaction and reduced employee turnover. Also, be sure to include both short-term and long-term returns, because an IT investment, if managed properly and accompanied by appropriate organizational and workflow changes, almost invariably takes time to have an impact.
Make the evaluation of returns and change management an integral part of the project management and reporting process. As the project proceeds and you learn more about the new system's capabilities, consider whether additional returns should be expected or the timeframe for achieving a return should be changed. Are staff who are being trained to use the system discovering new ways to perform their work more productively? Is the investment affecting areas that were not initially identified, and is this effect positive or negative?
Retrospectively review your original investment plan and projected returns. Look back! Not all IT investments achieve the type or level of return originally expected, so learning from the differences between expected and actual returns is important. Make sure that reviews of past projects are collaborative-the failure of an IT investment to deliver expected returns is seldom one individual's or department's fault. Use the results of the retrospective assessment not to punish mistakes but to improve your understanding of what to expect from current and future IT investments.
Publication Date: Saturday, January 01, 2005