On 27th April, the Indian Council of Medical Research (ICMR) advised States to stop using kits acquired recently from two Chinese companies – Guangzhou Wondfo Biotech and Zhuhai Livzon Diagnostics.
In its 16th April ‘Guidance on Rapid Antibody Kits for COVID-19’, the ICMR released a list of 23 test systems that received successful validation from the National Institute of Virology (NIV), Pune. This included test kits from the two Chinese companies.
So, what went wrong in just ten days?
We will try to understand antibody tests, what they can and cannot do, and explore a scientific basis to guide us in the future.
Basics of COVID-19 Tests: RT-PCR and Antibody Kits
Two types of tests are deployed for COVID-19. The reverse transcription polymerase chain reaction (RT-PCR) test detects the virus (its RNA) generally in the throat and/or nasopharyngeal swabs, and sometimes also in sputum. It will be positive on the day patients start showing symptoms, and even a few days before that – meaning that infected people are shedding the virus in the asymptomatic phase as well.
The RT-PCR is a confirmatory test. But it is cumbersome to do, requires highly trained personnel and specialized equipment, takes time (~ 8 hours) and is expensive (~Rs 5,000).
On the other hand, antibody tests determine the body’s response to infection by raising antibodies. Most antibody tests determine the presence (or absence) of two types of antibodies – immunoglobulin M (IgM) and IgG. Studies on COVID-19 patients show that IgM antibodies appear in the blood around 7 days after they first show symptoms; IgG antibodies take about 10 days. While the IgM antibodies disappear after 35-40 days, the IgG antibodies persist even after 2 months.
These tests are easy to perform, work with a drop of blood, give results in 15-20 minutes and cost about Rs. 500.
Interpreting Antibody Test Results
For COVID-19, the antibody test is interpreted as follows
A negative antibody test has no diagnostic value; it has to be confirmed with an RT-PCR test. A positive antibody test, however, indicates an ongoing or past infection. It is most useful for testing populations and high-risk groups to understand disease prevalence in a population. For example, results from Germany, the Netherlands and USA show that 2% to 30% of some populations have already been exposed to COVID-19.
Population estimates also allow us to better understand the lethality of the virus. Several such population surveys have shown the mortality to be in the range of 0.2% to 0.7% (average 0.5%), making it about 5 times worse than flu but nowhere close to the 7% case mortality observed globally (3.2% for India).
So, what went wrong with the Chinese kits? Was it a case of improper transit, storage or use?
These should be easy to answer by looking at the control line on each cartridge. If that lights up, the test was working.
But let us assume that kits were stored and used properly. Are there any scientific explanations? This requires digging deeper into the tests and the evaluation done at the National Institute of Virology (NIV). We should also look back at the basics of population screening.
Antibody Test Kit Evaluation by NIV
Each test is defined by its sensitivity – how many true positives it picks as positive; and by its specificity – how many true negatives it picks as negative. While both Chinese tests in question claim >99% specificity, the Guangzhou Wondfo Biotech test has a sensitivity of 86.4%; the Zhuhai Livzon Diagnostics test is 90.6% specific.
Thus, for every 100 true positives, the tests would miss 14 (100 – 86) and 10 (100 – 90) samples, respectively. Compared to RT-PCR (the “gold standard”) a low antibody pickup rate would indicate most people to be in the asymptomatic or early symptomatic phase of the disease.
An evaluation report from NIV was seen to understand how it tests antibody kits. It uses a positive panel of 24 blood samples from RT-PCR positive patients at various days after the appearance of symptoms, and a negative panel of 12 blood samples from people without Covid-19 infection. The positive panel contained 11 samples collected between day 2 to 6 after symptoms, and the remaining 13 were collected between day 7 to 13 days.
If IgM and IgG antibodies appear around day 7 and 10, respectively, the panel effectively has only 13 positive control samples. A test with 86% to 90% sensitivity would on average miss one sample from this small panel. Further, with no blood samples after day 13 of symptoms, the NIV panel does not evaluate the true utility of IgG testing, i.e. past exposure. Thus, a larger and more diverse positive panel is required to evaluate new test kits before their introduction in India.
In screening programmes, one should also consider the positive and negative predictive values of a test. The Positive Predictive Value (PPV) is the probability that subjects with a positive screening test truly have the disease. The Negative Predictive Value (NPV) is the probability that subjects with a negative screening test truly don't have the disease. These would depend upon the prevalence of the disease in a population.
Let us look at two scenarios with 2% and 20% disease prevalence, testing 1000 samples and a test that is 86% sensitive and 99% specific (like the Guangzhou Wondfo Biotech test). This is summarized in the table below and a good explanation provided here.
If the disease prevalence is 2%, in a population screen the test would predict positives with 63% confidence, but this improves to 95.5% when the prevalence is 20%. The test is the same. The key point is to understand the population being tested.
Any test system works well only when it is rigorously evaluated and properly deployed.
If we don’t learn, we are likely to face similar challenges with other test systems as we have faced with the two Chinese kits.
(Dr. Shahid Jameel is former Group Leader of Virology at the International Centre for Genetic Engineering and Biotechnology, New Delhi, India. He is currently CEO of the DBT/Wellcome Trust India Alliance. Views expressed here are personal.)
(At The Quint, we are answerable only to our audience. Play an active role in shaping our journalism by becoming a member. Because the truth is worth it.)