An Open Letter to ProPublica about Antibody Testing

They’re not “dangerous.” And when you’re not sure what you’re talking about, please: Say nothing.

On May 2, 2020, ProPublica, a news organization I used to respect, released a video under the title “Why You Can’t Always Trust Your Coronavirus Antibody Test Results.”

Very timely. Antibody tests are starting to become a Thing. Looking at New York City alone, we have the Post reporting on 28 April: “Antibody tests are being rolled out across New York City.” [Link]

the only Infection Fatality Rate (IFR) statistic most people people care about its their own…

We have local news channel WABC offering a consumer segment “What to know about urgent cares now offering antibody testing” [28 April. Link] They report:

On 1 May, Crain’s New York Business writes “CareMount offers antibody testing without restrictions.” [Link (paywall)]

All this in the context of New York State’s aggressive official blood antibody testing program. From Long Island News, 3 May: “NY officials conduct more than 15,000 random antibody tests at supermarkets across the state.”

So what’s not to like? It looks like people can finally get something they need and want.

Apparently, there’s plenty wrong, if you pay attention to ProPublica. It has joined a strange coalition of Antibody Test Deniers.

Here’s a representative headline from The Hill [1 May. Link]:

Alarm bells ring over controversial COVID testing

Are people getting infected from dirty needles or something?

There’s more. The New York Post asks: “Is the coronavirus antibody test a magic bullet — or false hope?” [2 May. Link]

Apparently the Post won’t take neither for an answer.

What are these alarm bells, anyway? Do we need to get the kids out of the house?

No, ProPublica is worried, as we find out by watching to the end of the video:

Social distancing? Where did that come from? And what has it got to do with me getting an antibody test?

The logic below can be followed only if you share a certain elitist, paternalistic, and condescending mindset, and have accepted as true the central diktat: ordinary people are idiots and need to be protected — by us — from themselves.

It goes something like this:

1. A person who suffered through several weeks of Covid is now curious about whether or not they have developed antibodies to the disease. They sure don’t want to go through that again.

2. They go in and get an antibody test at an instant care.

3. The test comes back and says, Yes, they do have antibodies against SARS-Cof-2 in their blood.

4. The person concludes they are immune and starts hugging people.

OMG.

How to stop an outbreak of hugging

How can we stop these positive-testing, probably-immune people from going out and… you know?

We need to confuse them about the quality of their test and the nature of their immunity.

As public policy wonks, this is important to us, because there are some experts out there we don’t like who are using serosurvey results to argue that the Infection Fatality Rate (IFR) of Covid-19 is actually lower than previously thought and — this is why we don’t like them — perhaps social distancing could be lightened up.

We ignore that the only Infection Fatality Rate (IFR) statistic most people care about its their own.

We’ll pretend we’re informing them, but neither we nor they have the attention span to understand and explain all those details and caveats.

Something with math might throw a scare into them.

And confuse. If pushing a meme that has dubious relation to the truth, seek the counsel of experts. Roger Stone says his favorite quote is from The Joker: “Introduce a little chaos. Sow a little confusion.”

You have an idea. Something that will really make people doubt the test. You’ve seen the answer before… long ago… it was…

That’s right! It’s the answer to the Trick Question on the final exam in Introduction to Statistical Epidemiology 101A!

Back in college. The Exam Question you missed. (No matter, the Professor went over that question in class afterwards, because so many got it wrong. You understood the explanation then. Sort of.)

The Trick Question was: “You’re screening people for a rare tropical disease. Only 1% of the population of this African country has it.”

“Your screening test generates false positive results 1% of the time.”

“Compute how confident you are (Positive Predictive Value) in the results of any individual screening test.”

You remember now. That was a stumper. Counter-intuitive.

For rare diseases, even if the test is right most of the time (high Sensitivity and Specificity), a single positive test doesn’t mean that much (it’s going to have a low PPV).

Blood antibodies to SARS-Cov-2 are kind of rare, right? Well, in some places they still are. So maybe we can recycle the Trick Question and get people thoroughly confused about whether or not their positive result is a false positive.

They’ll be afraid to rush out and buy a test that might be wrong. They’ll save money — we are always always a champion of the consumer — and, with no prospect of immunity to confuse them, keep on social distancing like the rest of us have to.

Wait. There is some chance they will get a false positive, right?

Of course. There’s no perfect test.

Okay, then! What is that chance?

Darn! I was afraid you’d ask that. We’re under deadline here!

It’s too much work and too confusing to dig up some real numbers.

So we’ll just make one up.

That’s fair. We’ll make sure we say this example is all hypothetical.

How about: 95% accurate? Sound good?

(Three days later, your bleary-eyed researcher emails you a screenshot of this table from an evaluation by Massachusetts General Hospital (MGH) of 3 “white stick” rapid tests on Pre-COVID-19 blood:

It shows there was exactly 1 false positive result from 3 x 60 pre-COVID-19 blood samples on 3 different rapid tests. The specificity is 99.4%. Not 95%.

Oops. Too late now. The exciting graphic using a false positive rate nearly ten times worse is already finished!)

If you’re the New York Times, you have to be a little more responsible when you argue the case using 95%. You need to cite the U.S. government wherever possible.

The early Cellex rapid test kit has 95% specificity, give or take. And you can make it sound like it’s one with FDA approval. It doesn’t — just a formal Emergency Use Authorization (EUA) — but if you word your story right — “FDA-authorized” — most people won’t notice.

Now you’re ready for the graphics department to do your pretty, if utterly misleading, Infographic.

Here’s one from The San Diego Union-Tribute [3 May. Link]. It discusses a hypothetical antibody test with a 2.5% false positive rate given to 100 employees at a company where the prevalence of blood antibodies is 5%.

Aside from apply the false-positive rate to the wrong number, notice the painfully sloppy language. The test finds people who have antibodies in their blood who were infected (past tense) in the possibly distant past (and perhaps a very few recovering but still infected). The ongoing passive “have been infected with the virus” should be past tense: “had been infected.”

Pretty scary, though. They need to add some social distance between those paper doll figures.

(In the real world, would it be too much to ask this hypothetical employer to talk to those 7.5 employees who tested positive, and ask them if they had the disease? And if they don’t think they did, pay for a second test? Isn’t that what is all doctors say? “I want to do another test?” So do one.)

[Footnote 4 May. Find the Final Exam question confusing? You’re not alone. I note this Correction appended to an article in MIT Technology Journal:

Contra Costa County had problems with this, now MIT… Well, whatever Stanford’s problem is, don’t blame me! I went to Cal.]

It says right here on my iPhone that things made in China are junk…

Scaring people is good. Getting them angry in addition is better.

Alex Jones knows this about pushing memes.

The poor sap getting a false positive is not getting the result of innocent routinely less-than-perfect medical tests.

No. The sap is being lied to!

Lied to! By whom?

The company who made the test, of course.

Some of them are made in China, aren’t they?

It says right here on my iPhone that things assembled in China are junk.

A Congressman has made all this his Thing.

Rep. Raja Krishnamoorthi, an Illinois Democrat, fears for the American Consumer. “What I fear is that they’re going to make dangerous life decisions based on those fraudulent testing results.”

No longer just inaccurate. Fraudulent.

Now, if ProPublica were the investigative reporters they used to be, they might discover that the earliest quotes decrying junk Chinese antibody tests “flooding the market” came from an organization called the APHL.

This is the Association of Public Health Laboratories, whose members will be paid to process the blood antibody tests the rapid point-of-case sticks won’t do, if the FDA keeps the latter out of the market. Concerned no doubt, but lobbyists.

ProPublica should also have noticed that the 2,500 white stick tests purchased by a Loredo, Texas businessman that “didn’t work” were actually perfectly good — as rated by University of California, San Francisco researchers recently.

They were just completely inappropriate for diagnosing acute incoming Covid cases in the emergency room, where the businessman sent them. After being warned beforehand.

Anyway, he came out of it whole, if with only a little egg on his face. He got a good Texas lawyer, refused to pay the Chinese company, and got the entire shipment seized and destroyed by Homeland Security because they were not “FDA Approved.”

I believe no SARS-Cov-2 antibody test is FDA approved, strictly speaking.

Still, the FDA stepping aside during the emergency has been an interesting experiment with the free market.

The lesson the free market has taught is that: in a free market is, you are at liberty to be a knucklehead.

Not even the FDA, I fear, can protect you from yourself.

Follow this blog on Twitter @Will_Bates_sci

Will Bates writes about science, technology, and business. His journalism has appeared in the New York Times, the Wall Street Journal, and numerous magazines.