How to measure vaccine efficacy: multiple parameters and some limitations
We recently heard about the new Pfizer vaccine to protect us from the SARS-CoV-2 coronavirus. Indeed, it appears that this vaccine is passing the Phase 3 clinical trial brilliantly, with an efficacy close to 90%. This value is not yet definitive, but it is unlikely to drop below 50%, which is the minimum threshold for receiving approval from the Food and Drug Administration, necessary for marketing. Vaccine availability will initially be limited, so several committees of experts are exploring strategic prioritization plans. Healthcare professionals will be the first group (being the category most exposed to the virus), followed by those who are at the highest risk of death or hospitalization: the over 65s and people with certain comorbid conditions.
We all know that vaccines prevent the spread of a (usually viral) disease by stimulating the production of neutralizing antibodies against the pathogen. Another benefit comes from herd immunity, which is achieved when the majority of the population (usually more than 90%) is vaccinated to protect themselves from a disease. In this way, even the small percentage of the population that does not have antibodies is protected (because it is immunodeficient and therefore cannot be vaccinated or is unable to develop immunity), as the virus cannot penetrate the population.
However, it is not automatic that vaccines are able to trigger a strong immune response in all people, or that the contagiousness of a vaccinated person is zeroed. These characteristics are measured after lengthy population studies of people receiving the vaccine.
Vaccines offer both direct and indirect protection. The first is obtained with the vaccination itself, which leads to the production of antibodies, and serves to counteract the symptoms of the disease; the second consists in reducing the infectiousness of vaccinated individuals, so as to reduce the transmission of the virus in the population. A good example is the flu vaccine: initially this was aimed exclusively at the elderly (risk category), aiming for their direct protection. More recently, it has been understood that vaccinating the entire population adds a level of indirect protection, as the contagiousness of the disease is reduced as well. This increases the level of protection of the risk category. Because influenza vaccines induce weaker, short-lived immune responses in the elderly than in young adults, increasing indirect protection has been an effective strategy to reduce the spread of the virus.
In this article I will tell you about population studies to evaluate the effectiveness of vaccines and their limitations.
Phase 3 clinical trials for a vaccine are designed to measure its efficacy and safety on an individual level. To evaluate the efficacy of a vaccine, cases of confirmed symptomatic disease are counted in a group of vaccinated volunteers and in a group of volunteers who received a placebo (the control group). None of the study participants know which group they belong to. If the number of cases of the disease is statistically higher in the control group, then the vaccine is considered effective.
As we have seen, together with direct protection (from symptomatic disease), there are also secondary parameters to consider, such as a possible asymptomatic infection of the vaccinated subject (which is usually not considered by the study as it is not detected), the contagiousness of the vaccinated individuals and direct protection in specific subgroups (for example those at risk). Often these data are difficult to measure, let’s see why.
It is understandable that the effectiveness of a vaccine in a certain category of people (such as those most at risk) affects the protection of the general population. However, unless specifically studied, clinical trials are not designed to establish efficacy in subgroups. In fact, randomized studies (placebo vs. vaccine in volunteers who are unaware of the group they belong to) can provide initial estimates in this regard, but which will have large margins of variability, leaving substantial uncertainty about the true effects in high-risk subgroups. This uncertainty may be exacerbated by the fact that high-risk people participating in the study are more cautious, thus resulting in less exposure to infections, and this reduces their contribution to vaccine efficacy estimates in their category.
How to solve the problem? There are several strategies for studying the effectiveness of a vaccine in a specific subgroup. One possibility is the inclusion in the study of a significant number of high-risk adults and this can be achieved by setting minimum enrollment targets for elderly and / or adults with comorbidities (other existing medical conditions).
Another consideration concerns the rules for discontinuing analyses with the trial still in progress. When preliminary analyses suggest that the vaccine works and does not cause serious side effects (as in the case of Pfizer’s for COVID-19), an ethical problem arises for the selection of new candidates to whom to administer the placebo: in emergency situations, it can become unethical and/or impractical to ask participants of certain subgroups not to receive a vaccine that is already available. To resolve this situation, it could be decided, before starting the trial, to continue it at least until a certain number of confirmed cases of the disease are reached (for Pfizer it was decided to continue up to 164 cases of COVID-19, therefore well over 94 cases ascertained at the beginning of November 2020, when the effectiveness of the vaccine was confirmed first). Studies to evaluate the efficacy and safety of a vaccine in the long term can generate more reliable assessments of age-specific effects.
Vaccines that reduce the symptoms of the disease can also reduce its infectiousness, either by directly reducing the viral load, or by counteracting the symptoms that promote the spread of the virus (for example, coughing and sneezing). But this is not a rule and infectiousness must be measured together with the effectiveness of the vaccine.
To evaluate the vaccine’s impact on the infectiousness of a virus, some studies look at the amount and duration of viral shedding in symptomatic participants. However, by definition this excludes asymptomatic participants. To measure the viral load (amount of the virus in the body) in all study participants, it is necessary to conduct frequent (weekly) viral tests, regardless of symptoms, in order to identify the period of acute infectiousness (when the person is contagious). The Oxford-AstraZeneca vaccine trial is testing participants weekly in the UK regardless of symptoms, but this is not the case in other studies (like Pfizer’s). However, although the weekly tests can provide insights into viral load in vaccinated people, they do not provide clear answers on the effect of the vaccine in preventing the spread of the virus, nor on the relationship between viral load and infectiousness.
Another approach to estimate infectiousness without having to extrapolate the viral load (tests take time and are expensive, if applied to an entire population) consists of studies on family units, i.e. the tracking of family members and other close contacts.
Other parameters that clinical studies need to evaluate concern the long-term safety of the vaccine (therefore pharmacovigilance activity is a must), the duration of protection provided, the effectiveness of a lower dosage, the level of protection against serious infections and death and the potential immune evasion of the virus. All these parameters can only be studied months or years after the vaccine has been commercialized.
It is normal for different vaccines against the same disease to have different characteristics. One may offer better direct protection to categories at risk (the elderly struggle to develop immunity more than the young population), while another could further reduce people’s contagiousness. About the vaccines that will protect us from COVID-19, the situation will be no different. Based on these characteristics, they will be strategically distributed in order to obtain the maximum benefit for the entire population. In order to achieve such a level of knowledge of vaccines, however, it will be necessary to continue population studies for several months after their large-scale distribution.