Benchmarking Mistakes: The Poisonous ‘Apples-to-Apples’

Top executives and managers in other industries know it is not only acceptable, but necessary to benchmark with other industries to obtain process improvements. For example, a major hotel chain desires to improve guest services. This chain not only has other hotel chains to examine for comparisons, but also can and should look at theme parks or retail corporations. Instead of comparing hotels to hotels, the hotel’s guest service policies are compared with the guest service policies of theme parks, restaurants, and others. Valuable lessons are gleaned from this benchmarking process. While the industries may function very differently, their fundamental guest service processes are common and provide learning opportunities for all parties. Similarly, retailers have made major improvements in inventory acquisition, warehousing, distribution and tracking. Hospitals haven’t typically studied these processes, citing the “uniqueness” of the healthcare and hospital operations as the reason. As a result, many hospitals continue to practice outdated and non-integrated supply transactions, as opposed to making supply management a priority. Many,processes are similar enough across industries for healthcare managers to learn and adapt process improvements from others.

Even though healthcare downplays cross-industry benchmarking because of the uniqueness of healthcare, they also believe that healthcare-to-healthcare benchmarks are valid only with those organizations exactly alike in structure, size, scope, culture, affiliations, physical layout, etc., etc. For example, an outpatient clinic wants to improve its cardiac rehabilitation services but only wants to be benchmarked against clinics offering cardiac rehabilitation services that use Saturdays to deal with overflow, as they do. Healthcare systems also want to benchmark with other systems as opposed to stand-alone facilities. While it is important to determine the way others handle issues, using practice and environmental factors to eliminate potential benchmarking partners reduces the value and learning opportunities for the organizations doing the benchmarking.

Does a Twin Exist?

It is virtually impossible for a hospital to find an identical apple. There are approximately 5,800 hospitals in the United States. Focusing on, for example, only 500-bed, non-profits, can narrow this even further. Simply by adding a few more criteria … academic teaching hospital vs. not; location; managed care penetration; number of buildings; outpatient volumes; etc., simple mathematics shows that a hospital can eliminate all potential hospitals as benchmarking partners.

Requiring multiple and non-relevant criteria narrows the list of acceptable benchmarking partners. Hospitals are complex operations. There are an infinite number of differences among hospitals, and there isn’t one exactly like another. Eric Franz, Manager of Financial Services at OSF Saint Francis Medical Center in Peoria, Illinois, agrees. “There is no twin hospital out there,” says Franz. “It just doesn’t exist. We are unique and we want to be unique.” It makes no sense, then, for hospitals to proclaim their uniqueness while at the same time developing lists of criteria “acceptable” benchmarking partners must meet. For hospitals to use benchmarking effectively, they must accept the fact that their twin doesn’t exist. Then, they can use their resources to learn instead of wasting resources benchmarking their level of uniqueness. Apples-to-Apples

Misconstrued – The Benchmarking Poison

Consider this example of how the value of a benchmark decreases as the hospital attempts to narrowly define acceptable benchmarking partners:
Apples-to-Apples: Benchmark the cost of medical transcription functions at hospitals. McIntoshes-to-McIntoshes: Benchmark the cost of medical transcription functions at hospitals with a centralized transcription department that out sources at least 60 percent of their transcriptions.

New England McIntoshes-to-New England McIntoshes: Benchmark the cost of medical transcription functions at a system-wide set of hospitals with a centralized transcription department that out sources at least 60 percent of their transcription, writing at least 40 different types of reports and an average TAT for History & Physicals of 24 hours. There should be at least three but no more than six hospitals in the system, located at least 10 miles apart, but within a radius of 124 miles.

Similar benchmarking “requirements” surface in many situations.

System-based hospitals only want to be compared to other systems, “preferably one with a similar structure and size”. Why? How will they know if their system structure is a competitive advantage if they don’t compare themselves to different structures or stand-alone hospitals? These highly selective criteria result in a less useful benchmark and less value for the facility that does the benchmarking. Attempting to “benchmark with a similar transcription department” in the above example obscures the impact in-house vs. outsource transcription; centralized vs. decentralized transcription; stand-alone vs. corporate systems has on turn-around time, cost, accuracy, etc., … the exact opposite result expected of a good benchmark.

Mirror, Mirror on the Wall

What do healthcare organizations learn from the search for and ultimately from their “twin” hospital? The search process teaches them that if they add enough criteria, they can reduce their learning pool and maintain the status quo because “there’s no one out there like me!”. If they happen to find a few “twin hospitals” to compare with, they’ll find that their solutions are similar … again reducing the learning opportunities (what can you learn from someone who’s just like you?). Getting an organization to recognize that the perceived “differences” likely point to improvement opportunities is difficult. How much more comforting is it to believe that “my costs could be lower except I have these corporate allocations, and a non-integrated information system, and a high managed care penetration”, than to come to grips with the fact that your costs are higher because of your choices (stand alone vs. corporate) and practices (allowing departments to purchase information systems that don’t interface).

It’s ridiculous to let truly minor differences eliminate cross-organization learning opportunities. According to Franz, there are enough similarities among hospitals to determine where improvements can be made. “The comparison hospitals we used for benchmarking were 80 to 85 percent similar, which is enough to get a good start on this process,” says Franz.

In addition to searching for twins, hospitals similarly search for best practices … thought by many managers to be the holy grail of process improvement. In the transcription example above, you can almost hear the manager thinking “… if I can find the best practice with respect to transcription, my problems will be solved”. The problem here is, similar to the twin hospital, there’s not a single “best practice” for most healthcare operations. A recent survey of hospitals found that many hospitals that had previously outsourced transcription were now bringing the function in-house; while in-house operations were looking for transcription vendors.

Why?

Changes in fit with their culture, their work force, and their environment. The “make” vs. “buy” decision is very dependent on the individual organization. So, while buying transcription services is a best practice for Hospital A, it might be a miserable failure for Hospital B. That is the job of the managers … to sort through their options, coalesce good ideas from multiple sources and come up with the most effective practice for their organization. One hospital cannot simply implement another hospital’s method without adaptation since the cultures, layouts and environments of each are different. Remember … hospitals claim to be unique and therefore they can’t be compared in a benchmark. Why then, would they willingly presume that someone outside their organization knows what the best practice is for them? Instead, a hospital must take bits and pieces from others’ most effective practices and formulate the most effective practice for their organization.

Who’s the Fairest of Them All?

The hospitals and healthcare systems that will be the “fairest in the land” are those who can avoid the common benchmarking mistakes. They will figure out WHAT they want to benchmark. If they want to improve costs, they will benchmark costs and not poison the “apples-to-apples” comparisons with non-relevant criteria such as payer mix, physical layouts, corporate structure, etc., on the way to determining their cost opportunities. The hospitals who use benchmarking as an effective tool will not waste their precious labor resources trying to find a twin hospital because they realize that reduces learning opportunities and encourages managers to think that the status quo is acceptable. Winnowing the list of acceptable learning partners narrows the value and usefulness of the benchmarking results.

The healthcare organizations that will benefit from benchmarking are the ones who realize that the relevant points of difference are driven by their own practices, structures and choices; and, they will make changes accordingly.

Hospitals who gather many effective practices and blend them into a strategy that meets the needs of their organization will benefit. These hospitals know that slavishly mimicking a process without consideration of their own culture, values and needs is managerial malpractice. Rather than comparing New England McIntoshes-to-New England McIntoshes, organizations who understand that the best use of benchmarking is to identify gaps in their performance, will be the ones who will learn from many others in the effort to find the most appropriate apples to improve their own unique processes and performance.