Recently, the Massachusetts Health Care Quality and Cost Council unveiled its new website, My Healthcare Options. While intended to directly support consumers and physicians as a decision-making tool, the site unnecessarily penalizes otherwise good hospitals through inconsistent ratings, by presenting those ratings without context, through a bias against larger hospitals, and by penalizing hospitals for compliance on reporting data transparently.
Although the site attempts to answer such questions as, “My doctor sees patients at two hospitals. Which one should I use?” in many categories, the data is not consistent enough to adequately offer a patient choice. Angioplasty, one of several methods to deal with a heart condition, has no ratings; the variance on quality is so small as not to matter. In other categories, one may find disclaimers, even on where ratings are applied: “Data is not available to calculate Statistical Significance” or “Not different from Average Quality.” Isn’t the point of a rating system the fact that it provides definition?
The danger with the site’s approach to ratings is that it lacks context. Stars are awarded according to the sample size and the measurements are relative. This will lead to more, not less, confusion on the part of patients.
By example, if one searches for a hotel in Boston, selections are ranked based on a star rating according to well-established and widely-available criteria. Hotels must meet a minimum threshold to maintain their ratings. For example, amenities like fitness centers or pools are generally necessary if a hotel wants to achieve three stars. Cities may have numerous hotels at a variety of star levels, but all four star hotels are comparable in their service, regardless of location. For example, Boston maintains twenty-six 4 star hotels while comparatively-sized El Paso, TX boasts only one.
If one searches for a book on Amazon.com, ratings are based on customer experience of that product and these ratings are cumulative. Since books are published with a certain size of audience in mind, the ratings become statistically significant once a few dozen reports are in. Without fail, most books are above average — 2.5 stars or greater. Is that because books are largely a matter of personal taste? Perhaps, but it also stems from the fact that the commercial editorial process ensures that there’s not so many absolutely lousy books published.
The new Council site applies this star motif – well established as threshold criteria in hotels or experience of consumer products online – in a way inconsistent with consumer expectation. In areas with sufficient data, 30% of hospitals must be given either a 1 star or 4 star rating, while 70% must be given a 2 star or 3 star rating. In some cases, the ratings are undermined by disclaimers. In others cases, category ratings are avoided. What if the worst hospital in Boston is still better than the best hospital in El Paso? Wouldn’t that be worth knowing?
Hospitals who have invested in building relationships with patients, their families, physicians and the community at large are right to be concerned with how the Council’s site portrays them. The site is clearly intended to influence volume and bias against larger hospitals. On the homepage, it notes, “This website can help you answer questions such as… Would I get better care for this health problem at my local hospital or a large medical center?”
The correct answer is that it depends: quality measures can be influenced, in part, by pedagogical selection of the staff. Some very fine “large medical centers” are lower ranked than their “local” counterparts, despite the fact the AMCs legitimately provide more options for treatment.
Consider angioplasty again. It is a very safe procedure that requires a less significant investment than an open heart surgery program. It is easier to find staff trained in the procedure. It is also a procedure where the reimbursements are higher than the costs of delivering services. In this case, hospitals of various sizes can offer angioplasty services to their community. If a patient needs open heart surgery, however, the scale of the undertaking is much, much higher. Those patients must be passed on to those “large medical centers.” Aren’t overall outcome measures of heart attacks influenced by these administrative choices, choices which are based financial necessity?
Or consider that the few large medical centers in Massachusetts are located in urban areas, where populations are less homogenous and less likely to receive preventative care. Evidence shows that culture, class, and ethnic background of patients can measurably influence outcomes.
The majority of hospitals operate on razor thin margins – margins which, with the slump in investment portfolios this year, might disappear entirely – and those hospitals depend on a few diagnostic categories, like angioplasty, whose reimbursements fund their emergency rooms. Would the state of Massachusetts prefer all heart patients from Worchester drive to Framingham if that means all the ERs in Worchester will close?
This new site is applying criteria that was never intended to penalize hospitals and yet now does exactly that. To champion transparency as a method of helping hospitals report on and improve their quality and safety measures is very different from assigning them a bell-curve grade based on those results. Already, hospitals are under increased pressure due to recent Medicare’s “never events” policy to not pay for complications created by medical error.
If the rating scale were absolute – as some other national reporting standards are – then we could know how well the average hospital was doing and which hospitals were above or below that average.
If this approach were followed, perhaps all of the rest of the stars would flatten out, eliminating most observable differences. Yet that’s going to happen anyway. As Medicare begins to increase restrictions on reimbursements for medical errors, private insurance will certainly follow suit. The general level of quality, as in book publishing, will be dictated by market forces.
While I do not envy the challenge the Council has undertaken, if the site were recast with a more absolute model, only the truly outstanding, in either direction, would be highlighted. Perhaps this would make for a less compelling website. But instead of pitting hospitals unnecessarily against each other or confusing consumers who are already confused, our state could spend its energy working to learn from the leaders and helping reform those who need it most.
