Why two vaccines efficacy cannot be compared? Understanding through Randomized Control Trial
When Pfizer and Moderna first announced the coronavirus vaccine, it was met with a lot of enthusiasm. The results showed more than 94 % efficacy for both vaccines. Even the Russian vaccine, which was received with large scepticism in the western world, had reported more than 90 % efficacy. The vaccines are seemingly very effective in real life as well, as coronavirus cases and deaths have plummeted significantly in the areas where such vaccines are being administered. These vaccines should signal the end of the pandemic and bring life toward normalcy.
However for a large part of the world, especially in the poor regions, it is neither Pfizer nor Moderna that is being administered. Sinovac, the vaccine from China, is the one that is actually being distributed in many underdeveloped and developing countries. Pfizer and Moderna have big logistic challenges which are difficult for a poor nation to facilitate, whereas Sinovac can be easily administered. Nepal is one of the countries that is administrating the Sinovac vaccine. While the Sinovac vaccine drive is moving strongly here, it is hard not to notice that many people are taking this vaccine because this is the only available option, and given the alternative, people would prefer a more “effective” vaccine.
It is easy to understand this sentiment. Initially, it was reported that Sinovac had only about 51 % efficacy. Although studies also showed up to 78% efficacy in different regions; 78 % is still less than 94 %, and many people still are sceptical because of the lower initial rate (of 51%). And even though many reports and news have repeatedly mentioned that 90% efficacy does not translate to 9 out of 10 people being protected after the vaccine and efficacy rate cannot and should not be a comparison index between any two vaccines, people are still likely to prefer the vaccines with a higher rate because it is difficult to understand what these numbers really mean. Thus, I shall dig into the Randomized Control Trial (RCT), the method that is used to test the efficacy of the vaccine, in order to understand better what these numbers actually mean.
The measurement of the efficacy of Pfizer with real data is nicely presented by Dashiell Young-Saver in his article in the New York Times, I recommend you give it a read.
Understanding RCT and Efficacy in Vaccine Trial
In an RCT, a sample is divided into two random groups, the treatment and the control group. Note that the group is divided strictly at random so that both groups have identical traits. A treatment group, in the case of vaccine trials, are the recipient of the vaccine, whereas the controlled group are the one given a placebo. Now suppose the following table show data regarding a particular vaccine trial.
Table 1
From the table, it can be seen that 10% of the people receiving the vaccine contact with the disease. However, of those receiving no vaccine, 30% get infected. The total size of the treatment and control groups are equal in this example. Thus the odds of getting the disease between the treated and controlled group is 1/3. It can also be understood that the chances of getting infected are 30% if no vaccine is provided but only 10% if an individual is vaccinated. Thus only 1/3 of the people who would have been infected are now infected after receiving the vaccine. Or, 2/3 of the people receiving vaccines are not infected.
The efficacy of the vaccine is thus (1- 1/3) or 2/3 or 66.67%.
Now, suppose that two different vaccines are being tested. Initially, we assume that both vaccines are equally effective, and like the example above, only 10 % of the people in the treatment group catch the infection. However, the trial shows a different number of total infections in the control group. The efficacy of the two vaccines is calculated below.
Table 2: Vaccines A
Table 2: Vaccines B
From the result, we find that Vaccine B has more efficacy. But you may have noticed how the high value of infection in the control group for Vaccine B largely affected the efficacy rate of the vaccine.
What if any two vaccines are super and equally effective, and only a small fraction of those receiving the vaccines (as a part of the treatment group) gets an infection but a trial for one vaccine takes place where the infection cases are uncontrolled and rising exponentially whereas the trial for other takes place where strict restrictions against the pandemic are keeping the cases at bay? Like in the example above, the efficacy for equally effective vaccines will differ because the infection rate in the treatment group will remain low in both cases due to the use of the vaccine (which we assumed to be equally effective), but the infection rate in the controlled group in one trial will be high whereas the infection rate in the other trail will be low. An RCT is expected to have identical behavior between the treatment and control group within an experiment, but these are two different experiments that we are comparing.
What does this mean?
It should be a bit clearer by now why two efficacy rates may not be compared. One may even ask why the control group in a different sample shows dissimilar behavior, and whether the trial can be generalized. Why for example, the same vaccine like the Sinovac, has a different efficacy rate between countries such as Brazil, Indonesia, and Turkey? Does the vaccine work differently between race, ethnicity, and sex?
While the in-depth nature of how a vaccine reacts to genes is an advanced medical study, such issues are unlikely to be ignored in trials. Trials make sure the sample is inclusive of any factors that may significantly impact the results. Thus, rest assured that genetics or ethnicity is unlikely to produce a different efficacy rate, and in case it does, the information must have already been available.
(Check the Pfizer trial report to find out how different types of people are included in the study: )
Two things however that might make a huge impact on two different RCT is the period when the trial was done and the geographical area that the trial was done in. The time and area itself should not affect the result, but other factors associated with it may have an influence. In the case of Pfizer, the trial took place between October to November, and 77% of the participants were from the United States. With the presidential election of 2020 happening in the same duration, that period was also the time when coronavirus was surging rapidly in the United States. If the Pfizer vaccine was working, it is easy to see why the efficacy rate would be so high. Given the increasing mobility of people in election time when cases were already high to start with, there would be an exponential rise in the total cases of the infection. But since the treatment group is being protected by the vaccine, the rise will largely impact the infection rate in the controlled group only. Thus, the difference between the infection rate in the treated and the controlled group is going to be big, making the efficacy rate higher.
In a different period where the cases were lower, or in a different place where the infection was more contained, the same vaccine could perhaps show a lower efficacy rate. For instance, China has almost contained the spread of coronavirus in its territory. It is also better equipped than western democracies to impose restrictions on mobility to curb the spread of the virus. Thus, the likelihood of the infection rate in both the treatment and the control group in a trial done in China is going to be low. As the infection rate is already small across both groups, the use of the vaccine in the treatment group is not going to make much difference. Moreover, since the infection rate in the controlled group is already small, there will be little difference in infection rate between the treated and the controlled. As a result, the rate of efficacy of the vaccine will be lower.
Now, you may question that if the vaccine was not working, the treatment group would also have a large infection rate and as a result, the efficacy rate will be low, so less effective vaccines will have a lower efficacy rate right? Categorically, this is true. The less effective vaccine should have a lower efficacy rate. For example, in the above table, if vaccine B had 5000 people infected in the treated group, compared to 8000 infected in a controlled group of the same size, we could easily imply that the vaccine is not good. The efficacy rate in such a scenario is just 37.5 %. However, what I am trying to prove here is that while the less effective vaccine may have lower efficacy, the behavior in the control group can make a significant difference in the rate, so low efficacy may not always imply a vaccine being ineffective.
Thus, with all said and done, one must understand that all vaccines are based on intensive research work and are good to use (if they are authorized). Therefore, efficacy rate should not be a factor for preference over any kind of vaccine, and one should get any authorized vaccines that are available for them.
