On Comparing Healthcare Systems

Ashish Jha et al published a paper in this month’s JAMA comparing the US healthcare system to other countries. It’s a fairly robust effort comparing several system features, so the paper made its rounds through medical twitter, although I wouldn’t say that it’s any different from the Commonwealth Fund reports that get published in Health Affairs every year and from which the Jha paper draws heavily (more on this later). I’m not sure how familiar the medical community is with Health Affairs literature, since that’s a journal designed for administrative types that are the mortal nemesis of clinicians, so there may be some utility in publishing this in JAMA. The paper’s basic argument is that the US does not appear to be meaningfully different from the mean of a basket of comparable OECD countries on measures of utilization or social spending and therefore the cause of the high healthcare spending in the US is prices. This is an analysis that we have seen before, notably Uwe Reinhardt’s 2003 paper which reached the same conclusion. Although I do not disagree with the Jha conclusion that the U.S. prices for goods and services are elevated and need reform, the proposition that the U.S. isn’t structurally different from other systems is incorrect and needs to be put into context.

When I started my degree in health policy, it immediately struck me how the U.S. healthcare system does not function as a free market. Without prices set by competitive interaction between patients and providers the system suffers from the chronic information problem Hayek highlighted in The Fatal Conceit where any innovations are stuck in the hospital where they were developed and diffuse through slow channels like academic journals and conferences rather than the much faster market pressures. Even if a large system like Kaiser Permanente is doing something very unique and effective, the rest of the healthcare system may never find out about it because of the lack of signal mechanisms. My intuition was therefore to focus on comparative healthcare research and to look for international perspectives of how a healthcare system could be organized and coordinated. Perhaps we can identify best practices by figuring out what some countries are doing drastically differently to achieve drastically different results? The good news is that there is indeed a rich area to explore, especially in the developing world where healthcare systems can’t afford to waste money like we do (my quick look into Iran’s community health worker system is just one example). The bad news is that that comparative healthcare research is a very new area and the research methodology and data are often questionable. Although comparisons of the US healthcare system to other countries have been made many times in the past, such as this talk from the 1980s by Milton Friedman at the Mayo Clinic in which he compares us to the NHS, the first major thrust for comparative healthcare system research was made by the WHO with their 2000 health system report  in which they attempted to rank 192 countries in a variety of areas, ultimately nominating France as the best healthcare system. Although this pleased the French, the report was deeply criticzed by academic and politicians.

Data Problems

The major problem with the WHO report, and much of comparative healthcare literature in general, is that the data was unreliable. The report provided an 80% uncertainty interval for each country, which obviously leaves a lot of room for error that would be unacceptable in other healthcare literature. These intervals made it difficult to state with a large degree of certainty that any system really was better than another and resulted in large potential swings, with the US potentially ranking anywhere between 7th and 24th. In order to improve the data available and expand our understanding, the Commonwealth Fund began compiling comparative reports on the performance of several English-speaking countries, eventually expanding to include several other Western European and Nordic countries. The Jha paper relies on the Commonwealth Fund surveys for perception and access metrics. Does the Commonwealth data achieve their goal of providing high quality data? I respect the Commonwealth Fund tremendously and appreciate the difficulty of their task, but having worked with their survey I don’t believe that this goal is achieved for the US. The Commonwealth surveys have sample sizes of 1000-2000 per country, with the exception of Canada which generally has samples of 4000-5000 because the Canadian Ministry of Health contributed funds to expand their survey sample size. A sample of size of 1000 may be sufficient for small countries with homogeneous systems like the UK, Switzerland or Sweden, but a basic sanity check tells us that it would be inadequate for competently describing a diverse federal system like the United States, especially when you consider that New York is not Alaska and the 2000 sample size is spread out evenly across the 50 states. 40 responders don’t tell me much about healthcare in NY, especially considering how the differences between NYC and upstate NY are like night and day. In addition to the Commonwealth surveys, the Jha paper also relies on the 2016 OECD Health panel for utilization and spending data. I haven’t personally worked in depth with the OECD data, but my general attitude can be summarized by a gaffe from a former Ukrainian state economist, “the Ministry hasn’t published the report yet because we haven’t decided yet what we want the numbers to say.” The Jha paper itself admits that there’s problems with the OECD data: “When OECD data were not available for a given country or more accurate country-level estimates were available, country-specific data sources were used.” I propose that it’s irresponsible to make authoritative claims like “utilization rates in the United States were largely similar to those in other nations” when the data is compiled from a diverse set of sources of clearly varying reliability.

Analysis Problems

Jha et al suggest that the US system is unremarkable because US utilization for various services is similar to the average utilization of the eleven countries analyzed. For example, US discharges for mental and behavioral conditions is 679 per 100,000 population so the US is unremarkable because it’s close to the average discharge rate of 736. This analytical strategy is fundamentally flawed for two reasons.

The first is that the average behavior of a basket of very different systems effectively has no meaning because you cannot infer any conclusions on the behavior of any single system based on its relationship to the average. If you take the mental health discharges example, if I wanted to design an average healthcare system from scratch, the average of 736 per 100,000 population tells me nothing about how much staff I should hire, what facilities I should construct and which patients I should admit. Furthermore it would be incorrect to describe the US protocols as “typical” because these systems are clearly very different. The mental health discharge rate for Netherlands is 119 per 100,000 population, while the rate for Germany is 1719 per 100,000. These are not systems that are doing the same thing and marginally varying around a mean, there’s a difference by a factor of 14! The US can’t be a “typical” system because a “typical” system clearly can’t exist with such large variability. Either the Germans or the Dutch are doing something very wrong, because I highly doubt that they’re both right. And if they are somehow both right then the entire concept of the average becomes irrelevant because countries deliver care that is appropriate for their population and any nation’s position with respect to the average is irrelevant since the average is not representative of their proper patient population. Another example of this problem is hospital discharges per 1000, where Japan and US have similar rates of ~125 hospital discharges per 1000. Can we safely conclude that Japan and US have similar admission and discharge practices? That would be an impossible conclusion if you knew that Japanese hospitals are simply closed on evenings and weekends. These two systems will clearly behave very differently as a result of this hospital schedule even though they may appear to be similar when only looking at them on a graph.

The second problem with the use of the average of eleven countries as a benchmark to compare US performance to is that the average can be manipulated to serve your needs by adding or removing countries. Why does Jha use these specific countries? They state that it’s to use the wealthiest countries, but that’s a dubious proposition because the US is MUCH wealthier than the rest of the world. If Germany was a US state it would be the 40th poorest state in the union, France would be the 48th poorest. Are these really comparable? We have to therefore admit that we’re taking some liberties in how strict we are in our selection, and if we are going to do that then maybe the analysis should be expanded to the full 34 countries of the OECD? Alternatively, maybe we should be more strict with the selection criteria. Culture obviously has a large impact on population health, and Japan is one included country that’s clearly very different from the rest as reflected by Japan being the lowest on just about every utilization metric. But if we’re removing Japan, are the Germanic countries really that similar to the US? Perhaps it would be best to focus on the four English-speaking countries which will be the most similar culturally? But doing that changes all the averages. It shifts the COPD discharges per 100,000 population from 206 to 252 and suddenly the US utilization rate of 230 goes from the middle of the pack to the lowest utilizer! The point is that there is no such thing as objective data, all data has an opinion, and all comparative analysis of many countries is vulnerable to manipulation by adding or removing countries until you make the US look the way you want it to look. Using an average as a benchmark does not fix this critical flaw.

Knowledge Problems

Finally, a major challenge that I encountered while working on comparative system research data is simply figuring out the truth of what I’m looking at and how the data maps onto reality. Nassim Taleb calls it the pseudo expert problem, where you may have a lot of knowledge about a topic but still have no idea what’s really going on because you don’t have direct contact with the situation. In comparative system research the problem comes from trying to reduce complex systems to simple metrics that can be analyzed and compared, which unfortunately strips out critical qualitative information.

For example, Jha puts forward data suggesting that social spending in the US is similar to other countries, but doesn’t comment that this is only true after adding up public and private contributions to the programs that constitute “social spending”. That’s not exactly what most people would consider as “social” spending and after taking away private contributions the US quickly falls behind. Is it appropriate to combine private and public social spending because Americans are generally wealthier than Europeans and have more disposable income? I don’t know, but the issue is too complicated to be casually glanced over. One more example is hospital beds per 1000, which Jha reports as 2.8 in the US and 2.7 in Canada. These figures may lead a policy analyst to conclude that these systems have similar levels of hospital bed supply and similar problems, but this would be incorrect. Canada has a chronic shortage of hospital beds that does not exist in the US because Canadian law requires patients to be discharged directly to appropriate post-acute care. However, there is a shortage of post-acute care facilities in Canada, resulting in as many as 13% of Canadian hospital beds at any time being occupied by patients who no longer need acute care and are simply waiting to be discharged, sometimes for months. These beds are effectively virtual, resulting in blockages in other areas, such as EDs where it is not uncommon for Canadians to wait over 24 hours just to be admitted. Another example is primary care physician consultations for which Jha reports 4 visits per person per year in the US compared to an average of 6.6 visits per person per year. An analyst may even be impressed with Germany which reports just over 10 visits per person per year. This analysis, however, makes the error of assuming that all consultations are born equal and ignores the fact that American consultations are on average ~22 minutes long compared to only ~8 minutes in Germany. To make matters worse, we don’t even know if the American consultations are of higher quality, because the American doctor may be fiddling with the EHR for 20 minutes with only 2 minutes dedicated to the patient, while the German doctor may be able to dedicate the full 8 minutes to the patient. Baicker and Chandhra raise similar concerns regarding intensity, quality and measurement  in their response to Jha. We almost never know what the reality is!

Conclusion

This rant isn’t meant to discredit the work done by Jha et al, or the effort of the organizations responsible for the data they use in this paper. A lot of work obviously went into this report and it has definitely expanded people’s knowledge of the area, as evidenced by the fanfare it continues to receive on social media. I’m only seeking to express my frustration over the significant analytical errors that are present in most literature comparing healthcare systems. Healthcare is 18% of the economy and any errors have massive ramifications. My own view, to echo Hayek, is that these systems are far too complex to be properly analyzed en masse and instead it is best to focus on very specific topics ie rural nursing home care in Japan and United States. Anything more broad than that is almost certainly going to be wrong and I encourage researchers to hesitate before making firm statements like “it’s the prices stupid”. If we don’t respect the complexities of these systems, how could we possibly expect the New York Times to understand them?

Advertisements

Bezos, Buffett and Dimon walk into a bar

That must be how the healthcare story of the week must have started. Amidst the drama about the State of the Union address and the FISA memo release came a tidal wave of coverage about the announcement made by the leaders of Amazon, Berkshire Hathaway and JP Morgan Chase that they will be collaborating on a new healthcare venture seeking to reduce the burden of healthcare costs. The subsequent media coverage was torn between promises of major disruption and foreboding of incontinent failure. What struck the most was not the announcement itself, but the absolute conviction with which these predictions were made despite there being so much unknown. Here’s what I mean.

The first thing that struck me is the simple question “why these guys?” Nobody seems to be asking this, but it’s critically important. It’s especially strange to have a venture between three different partners. Usually it’s a single company entering into the market, such as with Oscar insurance which also promised disruption. Otherwise it’s two firms cooperating, such as Apple’s venture into the data space while partnering with various healthcare provider networks around the country or the CVS/Aetna merger. But three firms getting together? That’s bizarre. Why not four? It would make perfect sense to throw in a pharmaceutical company into the mix. Why not two? It’s not immediately clear what critical skillset JP Morgan is bringing to the mix. Three? When was the last time you heard of a joint venture between three leading partners? It’s so strange that one has to wonder if they came up with the idea while drinking in a bar or playing golf. It’s probably the golf course, Warren Buffett is on record saying that he doesn’t drink.

The bizarre partner mix brings us to the strategy, which also doesn’t make much sense. Despite the deep conviction of the media analysis, the actual announcement didn’t include ANY specifics. The official statement says that the mission was “to address healthcare for their U.S. employees, with the aim of improving employee satisfaction and reducing costs.” This roughly means nothing, so all we’re left is looking at the core competencies of the companies involved. Much of the subsequent coverage focused on Amazon’s technological innovation and ability to disrupt markets. Is this even true? Does Amazon disrupt? Their Fire OS phones were an abysmal failure, which died off so fast that the media seems to have forgotten they even existed. Similarly their Anime Strike service was shuttered after only a year, while less notorious businesses like Crunchyroll and Funimation are operating just fine. Would people even pay for the Amazon Video service if it was unbundled from Amazon Prime? Amazon acquired Whole Foods but the results so far are mixed. The only business that Amazon ever truly disrupted was the retail market which was done through aggressive supply chain management and competitive pricing. But healthcare isn’t a retail market, it’s a services industry which is an entirely different game (read Baumol’s cost disease). The other business that Amazon has been successful with is Amazon Web Services, which provides an on-demand cloud computing service to individuals and businesses. Amazon could try to leverage this IT expertise, along with their AI projects, to better assess patient data, but this runs into the dual problem of legal restrictions on the usage of patient data and the fact that most patient data is junk (there’s a very good reason why MedPAC is rolling back on MIPS). Amazon also has a relationship with the Express Scripts PBM, but it’s hard to call that a core competence. Berkshire Hathaway is a conglomerate that runs a variety of businesses of which there are too many to describe individually, but my understanding is that they’re also involved with healthcare only tangentially. They don’t run any hospital networks, nor any biotech or pharmaceutical companies. Warren Buffett basically said fairly recently that pharma investment is too difficult to discern and Berkshire Hathaway at the time accordingly only had exposure to positions in vanilla players like Johnson & Johnson and GlaxoSmithKlein. They do provide health re-insurance through the General Re business, but that’s a somewhat different expertise compared to what a health insurer does. Finally we have JP Morgan Chase, which is a bank, they do bank things. Could they be replaced with any other large bank? Probably. So why JP Morgan Chase? Who knows, but I cite my golf course theory. Presumably they’re involved in the project to capitalize this new venture, but the announcement stated that this is going to be a non-profit, which is a hard sell to both shareholders and potential investors.

This brings me to the final thing that I noticed, which was the assumption by the media that this was some benevolent project that seeks to save American healthcare from persistent cost inflation. The joint statement clearly said that the goal was to provide solutions for “their employees”, which indicates that they would start locally before even considering expanding to America as a whole. This isn’t a charitable action, despite calling it a non-profit, but intended to benefit these companies and their shareholders. The question is what that benefit is. The thing that struck me most was the statement of intent to focus on providing “simplified, high-quality and transparent healthcare at a reasonable cost.” That could be a plausible benefit, but is this really something that people at these companies don’t have access to right now and are complaining about? All of these are big brands, one would assume that employees there have access to cadillac plans with low deductibles and broad access. This bares out in the Glassdoor pages, where Amazon’s insurance benefits’ rating of 4.1/5 is supported by comments like “Amazon has the best health insurance benefits I have ever seen.” The same is true for JP Morgan and Berkshire Hathaway. Employees at these companies seem satisfied with their benefits, so something is not right about this announcement. If the employees are not going to benefit, maybe the companies can? It could make sense to establish a non-profit insurer and then transfer the savings that would normally be the insurers markup to the shareholders of the three companies, therefore effectively transferring wealth from the insurer shareholders to parent company shareholders (note, not a charitable action), but this assumes that the new non-profit can operate more efficiently than the existing businesses. Contrary to popular belief insurers aren’t making enormous profits, publicly traded health insurers have a profit margin of only 3% on their revenue. Can Amazon et al negotiate more aggressively with pharmaceuticals and hospitals than their current insurer? It’s doubtful, they don’t have much market power. Although much of the media tried to position the 1 million US employees who work for these companies as a competitive advantage, this isn’t actually a large number. To put things into context the largest US insurer, United, has 70 million subscribers and the 5th largest, Cigna, has 15 million. Amazon+Berkshire+JP Morgan Health just doesn’t bring much weight to the table.

So what’s going on and what’s going to happen? The problem I have with this announcement is the amount that we don’t know and the amount that doesn’t make sense. There’s no company name. There’s no business plan. There’s no competitive advantages. There’s little relevant expertise. The bizarre partner mix makes no sense. All we have is a vague mission statement about improving quality and reducing cost for their employees, but even this doesn’t square out with what employees are saying on Glassdoor. If this announcement was brought in front of a venture capitalist by no-name Jeff, Jim, and John instead of Bezos, Buffett and Dimon they would have been thrown out of the room for wasting people’s time. By their own admission the business is in “early stages” but if you’re just starting out, why even make an an announcement? This is the one question I can’t reasonably answer, why even make an announcement? When nothing makes sense, your assumptions are wrong. This leads me to believe that this isn’t about making a business but all about sending a message. Obviously there’s no proof, just as none of the other theories have proof, but it seems the least incoherent answer. They’re signaling to somebody, most likely insurers but also possibly the government or phama, that this is something they consider to be a problem and are willing to exercise a nuclear option on (ie starting a dubious business) if the rest of the industry doesn’t shape up. It’s about having a strategic bargaining chip on the negotiating table, however credible or real it may (not) be.

 

Turns Out Celebs Are Bad Health Policy Analysts

After Jimmy Kimmel’s impassioned critique of the Graham-Cassidy repeal bill and the canonization of the Jimmy Kimmel Test for evaluating health policy,  celebrities are becoming increasingly active in the nation’s policy debates. I really wish they would stop, because they don’t know what they’re talking about, especially when it comes to healthcare, and the attention that they receive with their celebrity status makes it incredibly irresponsible for them to misinform the public. The newest entrant into the policy arena is Julia Louis-Dreyfus, famous for her roles on Seinfeld and Veep, who announced on Twitter that she has been diagnosed with breast cancer (get well, Julia).

DK06fuqV4AAfa77

The problem is that she then took the opportunity to plug universal health care as the solution to cancer mortality. Why is this a problem? Let’s take a quick look at how poorly the US compares to other OECD countries in breast cancer survival rates:

Capture
source: OECD

Oh. Well. That’s awkward. Despite not having universal health coverage, the United States is the best healthcare system for actually treating women with breast cancer and keeping them alive. That’s because contrary to popular belief, we do actually receive higher quality care than other countries. It’s important to point out that the US is a lot better than some key countries. The Commonwealth Fund’s darling “Best Healthcare System” in the United Kingdom is almost 9% behind the US in 5 year breast cancer survival rates. This is actually a massive problem, because as Julia Louis-Dreyfus correctly points out, 1 in 8 women will be diagnosed with breast cancer. That’s a full 1% of all British women who die because the NHS is failing them on this single important disease. Julia Louis-Dreyfus should study the facts before pushing such a system on American women as well.

By the way, since we’re sharing personal stories, my mother was diagnosed with breast cancer and as an uninsured poor immigrant went through chemo paying for treatment with loans. She lived. It’s certainly tough financially, but it’s not impossible, and the idea that being sick while uninsured is an automatic death sentence is yet another irresponsible falsehood.

Healthcare Reform is Comatose

The Sanders/Klobuchar/Graham/Cassidy CNN healthcare “debate” is a must watch for anybody who cares about healthcare reform in the US. However, it’s not important because it was filled with intelligent and critical discussion of the health care system’s problems and the smartest ways to fix them. On the contrary, it’s essential viewing because it reveals how politically dogmatic and intellectually lazy the two sides have become.

To start off, the bright point of the debate was that both sides seem to finally agree that the ACA isn’t working. When the ACA marketplace’s average deductibles increased to $6,000 and premiums increased by 17% in 2017, the true believers argued that this was a one-time actuarial adjustment as the marketplaces aligned themselves to the demographics of the newly insured population. As we get closer to 2018, this position is becoming increasingly indefensible as it’s becoming clear that exchanges are going to see another wave of large premium increases. Florida, a state that expanded Medicaid under the ACA, is forecasting a 45% increase. It shouldn’t be a surprise that the marketplaces are spiraling since ACOs, which were the primary cost saving mechanism in the ACA, have been a tremendous failure at producing any meaningful results. Importantly, this mechanism has been a failure nationally as well as locally in Massachusetts, which was lauded as a better version of the ACA because it had more robust wealth transfers and higher individual mandate penalties. This is why nobody at the debate tried to argue that the ACA is working. The Democrats praised the law for expanding coverage, but were also eager to admit that something is rotten in the state of Denmark and called out to their Republican colleagues to work together on fixing it together. And this is where the problems begin. Despite agreeing that cooperation will be needed, the ACA has warped policy thought in DC to such a degree that neither side is capable of thinking of solutions outside of the “fund the ACA” vs “defund the ACA” paradigm.

Republicans for their part are completely obsessed with pushing through a repeal bill that also controls Medicaid spending. While their hearts may be in the right place (the CMS trust is still scheduled to run out of money in 2029), this is a fight against reality. Three Senate vote defeats have made it clear that a bill that reduces future spending will not pass and that bill writers need to move into a new direction, including consideration of increasing taxes to stop the fiscal hemorrhage. The proposed Trump tax cuts, however, provide little hope for a balanced budget. Similarly, the desire to completely delegate healthcare policy to the states is also unrealistic. Nicholas Bagley wrote up a reasonable critique pointing out how despite transferring control of funds, the federal government is not going give states the ability to pick and choose federal regulations. What may have worked for welfare in the 90s probably would not work for healthcare because of the entrenched special interests that protect themselves from competition through legal barriers to entry.

Democrats, on the other hand, may be in an even worse position. They quite literally have no idea what to do about the ACA. This is why Hillary’s campaign position on healthcare could be accurately summarized as “we will do stuff.” Despite agreeing with the Republicans that the ACA isn’t working, they seem to have stumbled unwillingly to the bizarre conclusion that the solution is to throw more money at the fire. The Maine and Alaska reinsurance programs Klobuchar championed at the debate is simply taxpayer funded nationalization of sick patients and their costs, and Graham was correct in criticizing it as throwing good money after bad. The other proposal from the Democrats is Bernie’s Medicare for All single payer, which not only throws good money after bad, it throws enough money to bankrupt the nation within a decade (his own words, not mine). The helplessness of the Democratic position is therefore fueling the growth of a malaise among ACA supporters. Where there was once exuberance and joy about the law’s potential, there is now a resigned feeling that this is the best that we can do. Aaron Carroll, ACA proponent and one of the panelists on NYTimes’ recent “Best Healthcare System” tournament, now says that “THERE IS NO WAY TO SPEND LESS, COVER MORE, AND MAKE IT BETTER” (his emphasis, not mine), which effectively concedes that the ACA does nothing to push the health production possibility frontier outward. Aaron Carroll is obviously wrong about this; competition and innovation have been making healthcare cheaper, broader and better for a century now. But more importantly, that statement reveals the lack of imagination that pushes the Democrats to promote more spending. In their view the only way to improve care for everybody is to spend more, so if we’re doomed to spend more then it’s better to do it now rather than later. There’s an internal logic to this fiscal suicide. In addition, because deficit spending and redistribution are critical to this vision, they are also doubling down on centralization of healthcare management in the federal government. Aaron Carroll criticizes the Republican’s state-oriented vision by claiming that none of the states have any idea how to fix this mess. It’s a strange  argument to use to defend the ACA, because it makes the false assumption that the federal government somehow does know what to do and ignores the fact that the ACA itself was a watered down version of a state idea.

Ultimately the biggest problem is that neither of these boring approaches is going to solve America’s problems. The ACA isn’t what’s wrong with healthcare in the US, it just magnified the problems by adding more money into the system. Whether you eliminate it completely or empower it with more money, you’ll still be left with the same structural problems that are driving the irrational healthcare sector. There are critical market focused reforms that need to be made to price transparency, provider market entry, occupational licensing, health insurance purchasing, drug patent law and funding for behavioral health, all of which would immediately and significantly bend the healthcare cost curve. Instead we’re going to keep going back and forth about ACA funding. With so little brain activity going in Congress, the only plausible conclusion is that healthcare reform is comatose for the foreseeable future until something critical breaks.

Good news everyone, health insurance doesn’t work

The passage of the AHCA through the House and the subsequent BCRA written in the Senate has made society at large anxious about the implications of these bills for the health of working Americans. Much of this fear has been fanned by politicians claiming that the bills will kill Americans. Bernie started off the parade:

Then Hillary jumped in:

Elizabeth Warren then accused the GOP of criminal activity:

These cuts are blood money. People will die. Let’s be very clear. Senate Republicans are paying for tax cuts for the wealthy with American lives.

— Elizabeth Warren

And then Nancy Pelosi gave an estimate of how bad the damage will be:

We do know that many more people, hundreds of thousands of people, will die if this bill passes

-Nancy Pelosi

If I was the average citizen, I would be terrified at this point. Fortunately, I’ve been trained to think like an engineer and I don’t get scared until the data tells me that I should be scared. There are several observational reports out there demonstrating that health insurance improves outcomes, some showing no effect, and some even showing that health insurance kills people overall. This is the problem with observational studies; they are especially vulnerable to vulnerable to selection bias and confounding and therefore are inconclusive and unreliable evidence. What we really want is experiments. Fortunately, there have been two major randomized controlled trials (aka real science) of the impact of health insurance coverage on health outcomes and mortality, the RAND health insurance experiment and the Oregon Medicaid experiment. What does the data tell us then? Good news everyone, you have nothing to worry about, because health insurance doesn’t do a damn thing.

The RAND Health Insurance Experiment is a gold standard study that ran between 1978 and 1983 on a sample of 3958 people by providing participants with health insurance with a randomized level of cost sharing ranging from 0% to 95% coinsurance . At the end of the experiment, the researchers concluded that:

For the average participant, as well as for subgroups differing in income and initial health status, no significant effects were detected on eight other measures of health status and health habits. Confidence intervals for these eight measures were sufficiently narrow to rule out all but a minimal influence, favorable or adverse, of free care for the average participant.

The Oregon Medicaid Experiment was the second randomized controlled trial, conducted between 2008 and 2010. In the study state of Oregon expanded Medicaid coverage to a random selection of 6387 beneficiaries out of a total of 12,229 eligible applicants, with the non-recipients being used as the control group. In the words of the researchers:

We found no significant effect of Medicaid coverage on the prevalence or diagnosis of hypertension or high cholesterol levels or on the use of medication for these conditions. Medicaid coverage significantly increased the probability of a diagnosis of diabetes and the use of diabetes medication, but we observed no significant effect on average glycated hemoglobin levels or on the percentage of participants with levels of 6.5% or higher. Medicaid coverage decreased the probability of a positive screening for depression , increased the use of many preventive services, and nearly eliminated catastrophic out-of-pocket medical expenditures.

Two major studies decades apart and they find the exact same thing: increased healthcare utilization, increased total healthcare spending (important finding: prevention doesn’t save money), increased financial security (and the corresponding mental health benefits), but no statistically significant effect on health status (they don’t teach you this part in school). The health insurance didn’t do anything to improve people’s health compared to those who didn’t receive insurance. Keep in mind that these are large studies involving thousands of subjects. They’re also not ‘partisan’ or biased. The Oregon study in particular was run by Finkelstein and Gruber, who worked on the ACA and would’ve loved to show a positive effect on health, they just couldn’t find one.

This is a surprising finding for most people. It’s pretty simple to establish a plausible relationship between health insurance and mortality. Person gets sick, person can’t get treatment, person dies. How can there be no positive impact of health insurance on mortality? There’s several reasons that are not immediately intuitive but are real scenarios. Consider the following:

Case 1. Man gets cancer. Man has health insurance. Man’s cancer has no treatment. Man dies. Health insurance had no effect on health.

Case 2. Man has a bad diet and doesn’t get exercise. Man gets diabetes. Man has health insurance. Doctor tells him what he needs to do. Man continues to eat poorly and not exercise. Man dies. Health insurance had no effect on health.

Case 3. Man gets cancer. Man has no health insurance. Man’s cancer has a treatment. Man partners with a charitable organization and gets his treatment paid for. Man lives. Health insurance had no effect on health.

Case 4. Man has a non-life threatening lesion on a scan. Man has health insurance, so the doctor orders a biopsy to check. Man acquires an infection during the biopsy and dies. He would’ve lived if he had no insurance and didn’t get the biopsy. Health insurance killed the man.

It must be noted how common these situations are. 1 in 25 hospital patients acquire an infection. 1 in 500 hospital patients get killed by a medical error. Medical errors are the 3rd leading cause of death in America. Let that sink in, your average internist is involved in involuntary manslaughter 1-2 times per month. On top of that the expensive stuff that kills people, we’re not even that good at treating it. Overall cancer survival is only 50-60%. We also don’t have the slightest clue on what to really do with heart disease, respiratory disease, diabetes and such, because we don’t know how to stop patients from being themselves. Raj Chetty’s megastudy from last spring (1.4 million individuals observed) showed that the biggest predictors of life expectancy wasn’t income but whether you drank, smoked and exercised. 9 out of the 10 leading causes of death in the U.S. are preventable by changing your behavior. The inconvenient truth is that no amount of health insurance is going to save a smoker from themselves, because you’re just throwing money into a fire. Health care simply does not work for people who don’t care about their health.

The sum of all this is that health insurance has some real cost/benefit tradeoffs, but none of them are to improve health or save lives, and politicians should be more careful about saying things that are scientifically untrue. If you want to save lives, then try prohibition (that went well the first time), because health insurance expansion doesn’t get you there.

(Obviously this is all in the context of marginal effects at US insurance coverage levels, your mileage may vary in other contexts)