Will’s White Stick Buyer’s Guide
We rate the rapid SARS-CoV-2 blood antibody tests.
Good news for the followers of this blog: I have declared a Zombie alert.
For some time, the media narrative that SARS-CoV-2 antibody testing is “dangerous,” “inaccurate,” and so on has displayed all clinical symptoms of being a Zombie. This media narrative appears evidence-resistant. Nothing can kill it.
But my Zombie Chronicles are media stories, not science stories. They may be fascinating to those of us who work in the media — but not so such much for those focused on the science itself, or ordinary readers just trying to find practical advice.
So I’m going to move my snide remarks about bad media coverage of antibody testing to a different blog post, for which I will put a link here.
And — the good news is — no rants about the media in this post. Well, maybe just a couple.
Do you need an antibody test at all?
I hate to lose readers, but maybe not.
Covid-19 is something nobody wants to get, but everybody wishes they’d already had
Covid-19 is an odd duck. It’s something nobody wants to get, but everybody wishes they’d already had. Wish-fulfillment is not a great reason to shell out money on a test, even if it’s someone else’s money.
You’re in a different category if you had a positive PCR test and/or were diagnosed. Here, the odds are very good that you developed antibodies. The test will confirm (although just possibly deny) this. So if that’s important to you, go for it. Just remember the rule (explained below) not to test for antibodies sooner than 28 days after the day of the onset of your symptoms.
Health-care workers are in their own category. If they have antibodies, they can return to work with Covid patients at considerably less risk to themselves. If they don’t have blood antibodies, they can at least be assured that their PPE worked. All worth testing to find out.
Now for the just curious.
The result everybody wants — the lottery ticket — is to discover they have antibodies, but had a super-mild case of the disease — one they don’t even remember, or maybe mistook for the flu.
Your odds of winning this particular lottery depend on where you live. The odds favor those in hot spots:
- In Heinsberg, Germany, 22.2% of the seropositive (have antibodies) were asymptomatic (for the nuances of this strange word, see Covid with Style: A Writer’s Guide #1. The hidden dangers of ‘asymptomatic.’).
- In a study of French lycée (high-school) students at the center of an outbreak by Institut Pasteur, 17% seropositive showed no or subclinical symptoms.
For the non-hotspot general population, the odds of testing positive for antibodies presently seem to be in the 4–5% range. (The FDA, interestingly, uses an average seropositive rate of 5% for the U.S.)
The relative rarity of antibodies factors into a number the media like to throw around to warn people off getting tested, the Positive Predictive Value (PPV).
I would ignore that discussion in favor of common sense. Everybody knows any single medical test can be wrong, or inconclusive. That’s why doctors notoriously always want another and another.
If you live in New York, for example, and went through something that may have been Covid, I say, take the test.
Just be prepared not to win the lottery. Your odds of winning are only around 39%, according to Dr. Ania Wajnberg’s fine study of 1,343 New Yorkers at Icahn School of Medicine at Mount Sinai:
Among participants who did not have prior PCR but who were deemed high risk, i.e., people with symptoms consistent with SARS-CoV-2 who were told by a healthcare provider they had presumed infection, lived with someone with confirmed infection, or were healthcare workers themselves, we found 36% of this population had IgG antibodies to SARS-CoV-2. This finding suggests that a majority of participants suspected of having Covid-19 actually were not infected with SARS-CoV-2; however, it may also include a false negative rate of our assay (which has a 92% sensitivity) or insufficient time for participants to mount an IgG antibody response.
Just one nerdy footnote: remember that seroprevalence is not a static statistic. It is only going up, increasing every day with that number under “Recovered” in the Johns Hopkins dashboard.
What are these these “white stick” things?
By now, you’ve seen photos of them. They are technically called a rapid lateral flow immunoassay.
To unpack that:
- rapid, in the sense that the results are readable in 10–15 minutes.
- immunoassay, testing for antibodies in the blood. For SARS-CoV-2, the important one is called IgG. Most test also show a result for IgM, which is produced by the body during the infection, but fades away after. (IgM and IgG explained here).
- lateral flow relates to a chemical/physical principle relied on inside the white plastic shell.
I did a blog post on the white sticks when they were new here. I believe it’s still accurate.
These things go by many names. (I think it’s only me who calls them “the white sticks.”) The British abbreviate lateral flow immunoassay (LFIA). In the U.S., it’s usually LFA. There are lots of similar-looking rapid tests, such as home pregnancy tests.
However, calling these “home” blood antibody tests will, at least in the U.S., likely earn you a letter from the FDA, which don’t want them used at home. This led the lawyers to come up with an obscuring euphemism, “POC blood antibody rapid test.” “Point-of-care” is health industry jargon for a clinic or doctor’s office. The lawyers’ point being that if a nurse gives you the test, that’s OK.
On that topic, none of the tests below are FDA-approved. All come under a blanket FDA Emergency Use Authorization (EUA). A few of the companies such as Cellex have received more formal, written EUAs from the FDA, but this does not make them better tests. It just means the company’s paperwork is farther along.
White stick, or lab test?
As you’ll see when we get into a testing strategy, you may end up with one of each anyway.
It’s generally assumed that the lab tests are “better” than the rapid tests, but this depends on a lot. You would need to know the specs on the equipment used by the lab .Taking blood from a vein is better than taking it from a fingerprick, but you can do that for a rapid test.
Ironically, now that talk of “immunity passports” has been banned, one advantage of having an antibody test done by a lab would be, if you test positive, getting a piece of paper on Quest letterhead that says so.
You don’t get this with a rapid test. My suggestion, if you get a positive result, is to sign the white stick with a print from your bloody finger, then take a time-stamped picture using your cell phone.
I’m half serious. If you can use the photo to talk yourself out of 14-day quarantine on some future trip abroad, it might be a one worth taking.
Your testing strategy
Or, a crash course in SARS-CoV-2 blood antibody kinematics
I’ve written a handy one-page post called SARS-CoV-2 antibody rapid tests: best practices to minimize false positives. It’s full of all the caveats. But to recap the high points in one-liners:
- Don’t bother getting tested less than 28 days after your symptoms appeared. Here’s why:
You could be wasting your time to take the test sooner than 28 days after any symptoms. The IgG antibody isn’t going away. If anything, it appears to get a little stronger as time passes.
- Just ignore the IgM part of the test.
- Use the Mass General Rule — faint lines don’t count as positive — when looking at the result lines.
How we rate the tests: avoiding false positives
The tests work because the sticks contain custom proteins that glom on to the actual antibodies floating in the blood sample. These custom proteins been crafted to go after the SARS-CoV-2 IgG antibody, and nothing else.
Not that they always do. The ability of a test to always glom on to the right thing and nothing else is called its specificity. With specificity, higher is better. There are a few new lab tests now claiming 100% specificity —meaning they never get confused, ever. (We’ll see about that in the real world.)
With the white sticks, we consider 98% specificity the minimum acceptable level. This was the cut-off value used by UK Medicines and Healthcare Products Regulatory Agency when it evaluated rapid tests for use by the NHS. See my coverage here.
The way they come up with this number is mind-numbingly simple. They run the sticks on pre-pandemic (pre-Nov 2019) blood taken from blood banks.
If a brand of stick lights up positive on, say, 1 of 100 blood samples from 2017, it has a specificity of 99%.
These may just be random errors, but probably are not. Blood collected by blood banks from the general population can contain some unexpected things, as we learned the hard way in the early days of HIV.
One thing that could be going on is called “cross-reaction,” which means the proteins contained in the test stick are attaching to something else in the blood that sufficiently resembles IgG to fool them.
One obvious possibility is that its grabbing an antibody for another virus in the coronavirus family. I think it’s mostly (99%) myth that you can get a false positive if you’ve had a cold recently. But, like all good persistent misinformation, the concept contains a tiny kernel of truthiness.
Pregnancy puts hormones in your blood that might confuse the test. So does rheumatoid arthritis and various immune system disorders.
Higher specificity trades off for lower sensitivity. This means you might actually have the antibodies in your blood and get a “not found” result.
In this testing strategy, if you have antibodies but get a (false) negative, you may have wasted some money, but no social harm is done. You won’t come out of the testing center wrongly thinking you are immune and hug grandma by mistake.
Just one comment about those in the media who are terrified by the prospect of people getting false positives on blood antibody tests. They neglect the simple fact that people can take the test twice. The probability of getting two false positives from two different tests (if no other blood factor is in play) is vanishingly small.
Even the FDA says this, but the press does seem to be paying attention:
The regulator noted that all forms of antibody tests can produce false results, based on their “inherent limitations,” and in some cases at least two antibody tests may need to be performed for one patient.
I’ll stop myself there. Over-testing is routinely used in serious research. Institut Pasteur uses three tests to get a positive.
We’re basically looking at one number only; the specificity of the test for IgG at > 28 days. We call any test with IgG specificity lower than 98% unacceptable.
We rated only tests for which we had independent evaluations. We did not accept manufacturer’s specs at face value. (Although we ran across no instance in which they were exaggerated.)
Key to Sources
Denmark/Danish: “Evaluation of nine commercial SARS-CoV-2 immunoassays,” by Ria Lassaunière, et al., Statens Serum Institute, Denmark, Department of Virus & Microbiological Special Diagnostics, posted April 10, 2020. Preprint.
UK: In April, Public Health England asked its Covid-19 scientific advisory panel to arrange an evaluation of rapid test kits. A committee was chaired by Sir John Bell (left) of Oxford. The committee never publicly reported evaluations of individual companies, but some data have leaked out.
MGH: Massachusetts General Hospital (“Mass General”) did its own evaluation of three test kits prior to using them in the field tests I wrote about here. The MGH results were included in an Appendix in the UCSF report below.
UCSF: “Test performance evaluation of SARS-CoV-2 serological assays,” Alexander Marson and Patrick Hsu, Department of Microbiology and Immunology. Marson is from the University of California, San Francisco. Hsu is with the Innovative Genomics Institute at the University of California, Berkeley. Posted online April 27, 2020. MedRxiv link. This study, in my opinion, is a bit odd and seems to confuse more than it enlightens. The data reported in the tables are very useful, however.
FDA: The U.S. Food and Drug Administrations’ “EUA Authorized Serology Test Performance” document, which is mostly guidance for companies seeking Emergency Use Authorizations, but contains some specs that appear to have been supplied by the companies themselves. One suspects these may be more conservative than those touted in the product marketing brochures. Online only, here; scroll down.
ARTG: Australian Register of Therapeutic Goods. The Register is published by the Australian Government’s Department of Health, Therapeutic Goods Administration, which is roughly equivalent to the FDA.
And without any further ado, the ratings.
Top-Rated
Autobio RTA0204 (Zhengzhou, China). The Danish study ranked this best and gave it a 100%. The FDA lists its specificity at 99.4%. Note the parallel track design.
|
|
Innovita (Tangshan) Biological Technology Co., Ltd. (Fengtai, China). On 108 samples of pre-Covid blood, this test gave no false positives in the UCSF evaluation.
|
|
|
|
The Sure Biotech (Hong Kong) test likewise produced no false positives in the UCSF study.
|
|
|
|
Acceptable — Very Good
DeepBlue (Anhui Deepblue Medical Technology Co., Ltd., Hefei, China). The company, like others, also makes home pregnancy and other diagnostic tests. It gave one false positive out of 108 at UCSF. DeepBlue has the dubious distinction of having had 2,500 of its kits ordered by the businessman-owner of a private emergency room in Laredo, Texas, who failed to comprehend that antibody tests are not suitable for diagnosis of acute incoming Covid patients. His crying “foul” lit a Texas-size fire under the “cheap Chinese tests don’t work” meme.
Premier Biotech is a Minneapolis, Minn., company that distributes a test manufactured by Hangzhou Biotest Biotech, Co., Ltd. It gave one false positive out of 108. This was the test used by Dr. Jay Bhattacharya, professor of medicine at Stanford University, for the Santa Clara County, California, survey; the USC/Los Angeles County serosurvey; and his just-completed survey of Major League Baseball (<1%). Its specificity has been well-studied and comes in at 99.5%.
DNK (Dynamiker Biotechnology, Tianjin, China) was very much liked by the Danish researchers. It was co-developed with the Institute of Pathogen Biology of the Chinese Academy of Medical Sciences.
|
|
Bioperfectus (Jiangsu Bioperfectus Technologies Co., Ltd., Taizhou, China). Bioperfectus is a well-respected Chinese company. (They had a PCR test out for coronavirus on January 15th — four days after the first deaths were announced in Wuhan.) For UCSF, their rapid test was 100% perfect at detecting antibodies when present, and only delivered two false positives out of 104, a specificity score of 98%.
|
Wondfo (Guangzhou Wondfo Biotech Co Ltd., Guangzhou, China). Wondfo is a well-known and well-respected maker of diagnostic tests. It’s SARS-CoV-2 blood antibody test is a little different, in that it combines readings for IgM and IgG together. Its ‘Finecare’ sticks can be loaded to run different tests. (Disappointed you didn’t have Covid? There’s always Dengue Fever…) I believe little test cartridges are in that matchbook-looking thing on the left.
Acceptable
|
SD Biosensor (South Korea). This kit made one mistake in the MGH evaluation. MGH put its specificity at 98.3%.
|
UPC Biosciences Inc. (San Jose, CA). The California company is a distributor for kits made by a company called Chinese Peptide Co. The test scored 98.13% at UCSF.
|
|
|
|
|
Acceptable with caveats
These three tests are probably acceptable and possibly very good, but had mixed reviews.
BioMedomics (Morrisville, North Carolina) is emerging as a U.S. national champion, not the least because the privately held North Carolina start-up had [terminated 10 May] a distribution deal with pharma giant BD (formerly Becton, Dickinson and Company). Unfortunately, the BioMedomics kit has a bit of a specificity problem, rating only 96.26% at UCSF. However, at Mass General, following the ‘no faint lines’ rule, the test was perfect, producing no false positives.
The Biolidics test from Singapore also scored perfect at Mass General under the rule. However, the UK lists its specificity at a troubling 91.5%.
|
|
|
OnSite Rapid Test (CTK Biotech). The company has a U.S. office in Poway, Calif. The company claims 99.4% specificity, and the test was very well-liked in Denmark, but the U.K. evaluation put it at 96.6%.
|
|
Not Acceptable
The Cellex test was the first to get a formal Emergency Use Authorization from the FDA. Its test results show why an EUA is not to be confused with real FDA Approved. The Cellex tests scoreed between 93.8% (UK) and 95.6% (company claim) on specificity. Cellex is a Guangdong, China–based company that has a U.S. office in Durham, North Carolina.
DecomBio (Beijing, China). Scored only 91.59% at UCSF.
|
|
|
|
|
|
VivaDiag™ SARS-CoV-2 IgM/IgG Rapid Test (Hangzhou, China). This test, which is apparently no longer for sale in the U.S., got 95.96% at UCSF and also got low marks in the UK, 81.25%–97.1%.
|
On Probation
Bioeasy. (Shenzhen Bioeasy Biotechnology, China). This company recently introduced a rapid blood antibody test.
The company is a for-real maker of diagnostic tests, but became a notorious source of confusion earlier this year when its rapid antigen nose swab test for detecting SARS-CoV-2 virus did not work. This was not an antibody kit. The antigen kit was briefly viewed by some European countries as a White Hope alternative to spinning up proper PCR testing logistics. The antigen test was withdrawn from the market and the kits replaced.
… health authorities in Spain stopped using a rapid diagnostic test (RDT) kit that required a nasopharyngeal (NP) swab to collect specimens to diagnose patients for the SARS-CoV-2 virus, saying the test kits from Shenzhen Bioeasy Biotechnology were unreliable. Turkey also rejected the Shenzen Bioeasy test kits after finding similar problems, according to The Middle East Eye. Ukraine and Georgia also bought kits from Bioeasy, according to published reports.
The Spanish Society of Infectious Diseases and Clinical Microbiology found that the Shenzen Bioeasy NP-swab tests had an accuracy rate of less than 30%, SCMP reported.
The White House coronavirus task force testing expert revealed on Tuesday [5 May] that the CDC was examining antibody tests and they are performing ‘very well.’ The results of a government study on the subject would become available “really soon.”
They can crib off this one, if they want.
You can follow the COVID aB Tracking Project blog on Twitter @Will_Bates_sci
