There has recently been much excitement following the reported success and promise of a 3 year long clinical trial for AIDS vaccine that was carried out in Thailand, the largest trial of its kind in the history of AIDS. The trial, which has the backing of the Army and the National Institute of Allergy and Infectious Diseases among others cost upwards of $100 million. The New York Times reports:
The trial, the largest AIDS vaccine trial in history, cost $105 million and followed 16,402 Thai volunteers.
The men and women ages 18 to 30 were recruited from two provinces southeast of the capital, Bangkok, from the general population rather than from high-risk groups like drug injectors or sex workers. Half got six doses of two different vaccines; half were given placebos.
For ethical reasons, all were offered condoms, taught how to avoid infection and promised lifelong antiretroviral treatment if they got AIDS. They were then regularly tested for three years; 74 of those who got placebos became infected, but only 51 of those who got the vaccines did.
Commenting on the ‘partial success’ is an article in Scientific American:
In an early-morning announcement today, researchers reported that an experimental HIV (human immunodeficiency virus) vaccine effectively reduced the number of people who contracted the virus by nearly a third.
I have some immediate skepticism on the numbers reported and I shall present them in this post in no particular order. But I must state here that no argument of mine will involve special knowledge of HIV literature or host-pathogen dynamics during infection that I have been reading in some detail for my thesis project. These are simple argument and do not involve any fancy statistical inference tools. The SciAm article does mention some points of doubt but these are surprisingly absent in all other articles including that in NYT. Some comments are therefore in order:
1. Firstly, the work that has gone into the vaccine is commendable. Secondly, if historical relativism were some benchmark of success, then it behooves us to acknowledge that the absolute results are promising and ought to be pursued. (Read about the failed Merck vaccine trial in 2007)
2. The vaccine itself is a combination of two vaccines that failed independently in the past. While this does not have to necessarily imply anything, the mechanistic evidence of the new vaccine is wholly unclear and needs to be delineated soon. Furthermore, it was reported that there was no difference in viral loads (or alternatively the chronic phase set point measured in viral RNA copies per ml) between the two infected groups – one that got the vaccine and one that was administered the placebo. Now this is indeed the major reason that confounds the possibility of parsing any mechanistic insight considering that we know the individual drug compositions.
3. Let us consider the null hypothesis that the vaccine is ineffective and evaluate the statistical significance of the results obtained. The statistical significance of a result is a measure of vaccine efficacy. The metric to compute the statistical significance is the p-value which is simply defined as the probability of observing a result at least as extreme as the one observed in the experiment under the assumption that the null hypothesis is true (note however, that it is not the probability of the null hypothesis being true). If this probability is less than a critical value then scientists consider that as a reasonable case against the null hypothesis. As a universally agreed upon heuristic, the critical value of this probability is taken as 0.05.
Let us ask, what is the probability of observing a split of 51 vaccinated individuals and 74 individuals (or lower, in favor of the vaccine) administered a placebo in a cohort of 125 infected individuals, under the condition that the null hypothesis is true (that the vaccine is ineffective)? If this probability is lower than the critical value (0.05) then the results may be considered statistically significant. The expression is the following, with n=125.

It turns out that this sum evaluates to approximately 0.048 (I just calculated a few dominant terms in the summation assuming the maximum term method is valid here). Now this is lower than 0.05 and I assume is the basis for the report of statistical significance as mentioned in the papers. Note however that most successful vaccines in the past have been validated against much lower p values.
4. The sample size is extremely large as compared to the rate of incidence (which is a given considering the nature of HIV infection). Considering the following thought problem. Assume that the vaccine is ineffective and that the rate of incidence of AIDS in a medium risk population is a binomial distribution with probability of infection as 63/8000. Now, we have two groups of 8000, each administered a placebo ; Let us ask what the probability of seeing 51 and 74 infected in the respective groups is. Both numbers are close to being one standard deviation (
) from the mean (63) thus indicating that it is highly likely the result might be confounded by numerical variance. In contrast if the split was 20 and 74, the hypothesis that the former group was administered a placebo would be far more weakened.
5. There is increasing and indisputable evidence that the progression of HIV is profoundly dependent on the genetic constitution of the host, particularly focused on a locus of the human genome coding for a class of proteins called the Major Histocompatibility Complex (MHC). MHC is a highly polymorphic locus – it is very unlikely that two individuals in a population will possess identical MHC genotype (though they may have some alleles, or haplotypes in this context, common). Furthermore, there is a high variation of MHC distribution across geographic locations. Therefore a vaccine that looks successful for a rapidly evolving virus like HIV in Thailand cannot be extrapolated blindly to South Africa or India.
6. Any successful response against HIV has to invoke all the arms of the immune system – the antibody or humoral response and the lymphocyte or cellular response in extremely strategic ways so as to corner the virus. What is more precious than success in a particular trial is coming up with a mechanistic rationale for such a vaccine which also includes modifications across genotypes. In this case, one has to (as I am eagerly) wait for the publication of details of the methods and results of the trial.