• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

Vaccines, Blood & Biologics

  • Print
  • Share
  • E-mail

Section Contents Menu

Vaccines

Statistical Review and Evaluation, April 15, 2011 - MenHibrix

BLA  Number:

125363.0.12

Product Name:

MenHibrix (Meningococcal Groups C and Y and Haemophilus b Tetanus Toxoid Conjugate Vaccine)

Applicant:

GlaxoSmithKline Biologicals SA

Date Submitted:
Action Due Date:

April 15, 2011
October 15, 2011

 

Statistical Reviewer:

 

Tsai-Lien Lin
Lead Mathematical Statistician
Viral and Bioassay Team, VEB/DB/OBE

 

Through:

 

A. Dale Horne
Branch Chief, VEB/DB/OBE

 

 

To:

Joseph Temenak
David Staten

CC:

Meghan Ferris
Barbara Krasnicka
Mustafa Akkoyunlu
Estelle Russek-Cohen
Henry Hsu
DB chron

 

  1. EXECUTIVE SUMMARY

    GSK submitted BLA 125363 Amendment 12 to provide responses to the 88 items listed in the CR letter issued on 6/11/2010. This statistical review covers items 1 and 3 regarding the meningitides serotype Y (MenY) hSBA assay stability.

    Overall, the applicant’s explanations to items 1 and 3 do not seem to address the assay stability issue adequately. A reduction of hSBA MenY titers was observed consistently for re-tested samples from studies Hib-MenCY-TT-005 and -007/-008. The re-tested samples from Hib-MenCY-TT-013 added as sentinel samples in routine hSBA testing of samples from Studies -009 and -010 also showed reduction in hSBA MenY titers from their initial reference values. Assay stability remains an unresolved issue. The data for assay controls covering the testing period from study 005 to studies 009/010 may provide useful information regarding the assay stability.

  2. Background

    GSK submitted BLA 125363 Amendment 12 to provide responses to the 88 items listed in the CR letter issued on 6/11/2010. This statistical review covers items 1 and 3 regarding the meningitides serotype Y (MenY) hSBA assay stability.

  3. Statistical Evaluation

    Item 1

    Because the licensure of this product is based on serology assays only, you must provide scientific evidence showing that the Neisseria meningitidis serotype Y (MenY) hSBA assay is validated and is working reliably and consistently. The –b(4)- in titer observed over time, in particular for clinical study MenCY-TT-005, suggest that the assay performance is neither reliable nor consistent. Please provide scientific evidence showing that the Neisseria meningitidis serotype Y (MenY) hSBA assay is properly validated for its intended use and is working reliably and consistently. Please provide the following evidence in support of the reliability and consistency of this assay:

      1. In the "assay development history" document (Module 5.3.5.4.3. GSK hSBA Assay Development History), you indicated that the three separate retests of samples from the Hib-MenCY-005 clinical study in a MenY hSBA assay yielded a significant         -b(4)--in titer in the majority of tested samples. You have suggested that this effect may be related to the age of essential serum components of the assay.
        Please provide complete data in support of your hypothesis that the proposed changes over time in serum components affect the hSBA titers as observed in the Hib-MenCY-TT-005 study.
      2. Also in the "assay development history" document (Module 5.3.5.4.3. GSK hSBA Assay Development History), you indicate that an additional 12 samples from the Hib-MenCY-007/-008 clinical study were also retested for MenY hSBA and that         -b(4)------- titers were observed in all 12 samples after retest (GMR (b)(4)). Please submit the results of the 12 retested Hib-MenCY-007/-008 clinical study sera.
      3. In a teleconference dated February 9, 2010, you discussed with CBER the possibility that changes in the -----(b)(4)----- upon storage might have altered sensitivity in the hSBA assay. Please provide detailed information on the characterization of the MenY strain S (b)(4) and any changes it underwent over time. If storage of -----(b)(4)----- affects the stability of the assay results as you have suggested, please describe how you will maintain assay stability in the future.

Applicant’s Responses:

  1. Based on the analyses and investigations, the applicant concluded that the hSBA MenY assay and -------(b)(4)------ are stable, but the combined factors of ---------(b)(4)-------- cycles and storage of these samples (------(b)(4)------) for up to ---------(b)(4)--, and the use of a lot of human complement which had exceeded the shelf-life contributed to the –b(4)------ titers observed for the retested study 005 samples.

    The robustness of the hSBA assay has been further evaluated by extension of the        ----(b)(4)---- experiments and a new end-of-run experiment. The linearity data previously submitted to IND (b)(4) were re-assessed. The results were incorporated into the updated validation document submitted in this amendment.
  2. There are 14 Hib-MenCY-007/-008 samples (not 12 as previously stated) retested with varying combinations of ----(b)(4)---- and human complement lots.  The geometric mean of the ratio of the re-test titer/ref titer was (b)(4). The minimum ratio observed is only (b)(4).
  3. Based on a review of the history of the (b)(4) used and the available characterization data, the applicant concluded that the MenY b(4) used and the hSBA assay has remained stable over time.

Reviewer’s Comments:

  1. The applicant’s explanations for –b(4)---- retest titers do not appear convincing. The data from the ---(b)(4)--- experiments showed that the geometric mean ratios of titers after (b)(4) to the reference titers for double positive samples is -------(b)(4)---------- after -------------(b)(4)------------ cycles, respectively. The impact of ------(b)(4)-------- cycles on the MenY titer determination appears to be minimal at least through -----(b)(4)--, and may have some impact (but not at the magnitude seen in the studies 005, 007, and 008 retest data) after -(b)(4)-.
  2. The use of an expired human complement lot and the sample storage condition for studies 005 are no longer present for later studies. Yet, a consistent pattern of                -b(4)----- in retest titer from initial reference value is seen for those retested samples from studies 007 and 008, and the study 013 samples which were retested as sentinel samples during the testing of studies 009 and 010 samples (see comments to item 3).
  3. Control charts for two positive controls ------------------(b)(4)-------------- tested from July 2009 to June 2010 were included in the updated validation report to demonstrate assay stability. Two control charts for each positive control are presented: one for period from July 2009 to January 2010 and the other one for period from February 2010 to June 2010. It appears that the positive controls used were different between the two periods. The positive controls from July/09 to Jan/10 have lower titers and narrower control limits (e.g., QC1 target titer is (b)(4), with control limits at (b)(4) and (b)(4) approximately, as displayed in applicant’s Figure 2 shown below). From Feb/10 to June/10, the titers for the controls are higher and the control limits are much wider (e.g, QC1 target titer is approximately (b)(4), with lower and upper control limits at approximately ------(b)(4)------, as displayed in applicant’s Figure 3 shown below).

    (b)(4)

    (b)(4)

The control charts for QC1, although showed no apparent trend during the one year period from July/09 to June/10, seems to suggest that from July/09 to Jan/10, the majority of the titer values are below the target value and many are below the lower limit. From Feb/10 to June/10, though all individual titers are within the wide limits, the individual values also tend to be lower than the target. Regardless whether the
control charts demonstrate satisfactory assay stability or not, the time period covered by these control charts are several months after the testing of 009/ 010 samples (sera from these two pivotal studies were tested from 1/12  to 2/24/09). These control charts thus do not provide evidence that the MenY immunogenicity results of the pivotal clinical studies are reliable. The control data for the entire period of original testing and retesting of studies 005, 007, 008, and 013 samples and during the studies 009/010 testing will be more informative regarding the assay stability.

Item 3

Regarding the MenY hSBA assay data for the samples from Study Hib-MenCY-TT-013 added as sentinel samples in routine hSBA testing of samples from Studies -009 and -010 (Module 5.3.5.4, pp. 41-42, Table 23), you state that there are 40 samples; however, CBER noticed that there are 52 samples. Additionally, many test results are missing or below the lower limit of quantitation (LLOQ), and the missing values have different codes, e.g., "/," "I," TC," or "TD," without explanations of the codes. Please provide the following information:

  1. A list of sentinel samples that are excluded and the rationale for the exclusion of sentinel samples in your statistical analysis presented in Table 21 (Module 5.3.5.4, pp. 38-39);
  2. A definition for all missing value codes;
  3. An explanation of how test results below LLOQ are treated in your analysis, and an indication of what the "Concordance" values are that are presented in Table 21.

Applicant’s Responses:

  1. A total of 38 sentinel samples were added during the testing of studies Hib-CY-009/010 each week (incorrectly listed as 40 previously in Module 5.3.5.4, pg 40). The sera volumes for 14 of the original 38 samples were insufficient for all of the assay runs and were therefore replaced after week 1 by 14 new samples. Therefore, the total number of samples in the data file is 52 (38+14=52).

    All missing and invalid results were excluded from analyses. For the concordance analysis, values <4 were considered valid.  For the correlation and geometric mean ratio analyses, only the values > 4 are considered valid, and only those samples with valid values at both time points are included in the analyses. The number of samples included in each analysis comparing one time point to the other ranges from 14 to 31.
  2. The definitions of various missing codes are provided by the applicant.
  3. Concordance is defined as the number of concordant observations (both results are either <4 or >4) divided by total number of observations. The analysis results of sentinel samples during the phase III testing are updated. There are some differences from the previous analysis results presented in Table 21 of the original submission.

Results of Sentinel Samples from Studies Hib-MenCY-TT-013
Comparisons with reference titers

 

GMR (wk1/ref)

GMR (wk2/ref)

GMR (wk3/ref)

GMR (wk4/ref)

MenC

1.00

0.89

0.81

0.78

MenY

0.81

0.63

0.67

0.60

 

r (wk1/ref)

r (wk2/ref)

r (wk3/ref)

r (wk4/ref)

MenC

0.94

0.97

0.96

0.96

MenY

0.64

0.78

0.78

0.83

 

Concordnce (wk1/ref)

Concordance (wk2/ref)

Concordance (wk3/ref)

Concordance (wk4/ref)

MenC

81.25%

91.43%

96.15%

83.87%

MenY

96.43%

96.15%

97.30%

91.43%

Comparisons from week to week

 

GMR (wk2/wk1)

GMR (wk3/wk1)

GMR (wk3/wk2)

GMR (wk4/wk1)

GMR (wk4/wk2)

GMR (wk4/wk3)

MenC

0.94

0.87

0.96

0.79

0.83

0.84

MenY

0.99

1.08

1.01

1.00

0.94

0.95

 

r (wk2/wk1)

r (wk3/wk1)

r (wk3/wk2)

r (wk4/wk1)

r (wk4/wk2)

r (wk4/wk3)

MenC

0.97

0.98

0.98

0.98

0.98

0.98

MenY

0.94

0.94

0.92

0.95

0.96

0.94

 

Concord (wk2/wk1)

Concord (wk3/wk1)

Concord (wk3/wk2)

Concord (wk4/wk1)

Concord (wk4/wk2)

Concord (wk4/wk3)

MenC

86.67%

86.36%

92.31%

92.59%

86.67%

86.36%

MenY

100%

100%

100%

93.33%

94.29%

93.33%

Numbers in red changed from previously submitted results.

Reviewer’s Comments:

  1. It was noted that many missing values of those tested samples are due to “TC” (too concentrated) or “TD” (too dilute). In those cases, they were supposed to be repeated, but were not done. These missing values are not missing at random. There is information about them. According to the applicant’s explanation, “TC” means that less than 2 dilution points of the curve have –b(4)-------- and the sample has to be repeated at a lower dilution. Thus, we know that those titers are below certain levels. Excluding a large proportion of samples with TC code from statistical analysis will likely bias the result. Most notably, only 20 out of the 38 samples tested at week 1 have valid titers (i.e., almost half of the data are missing.) Out of the 18 missing values, 8 are due to TC and 7 are (b)(4).
  2. The GMR results show that during the period from week 1 to week 4, no assay drift was observed. However, when comparing to the initial reference titers, statistically significant –b(4)---of MenY titer is observed at weeks 2-4 (GMR= --------(b)(4)----------- respectively.) The observed –b(4)----- in titer is smaller at week 1 (GMR=(b)(4)), possibly due to the impact of large amount of missing data as discussed above. If all week 1 TC values are replaced with the titer estimated from their week 2-4 values, then the GMR at week 1 becomes similar to the GMRs at weeks 2-4. The assay drift could have happened prior to the re-testing. Since studies 009/010 samples were tested about two years after sera collection, it is not known whether the 009/010 assay results are reliable enough for the immunogenicity endpoints in these pivotal clinical studies.
  3. The concordance, defined as the number of concordant observations (both results are either --(b)(4)--), does not assess the agreement across the assay range which is needed when GMT is the immunogenicity endpoint (e.g., for lot consistency evaluation). Furthermore, the result of the concordance analysis is not adequate for evaluating the quality of studies 009/010 assay results for the following reasons: (1) many samples included in the analysis which contribute to the concordance rate have an initial titer (b)(4) and remain unquantifiable through the 4 retests; it is not known whether these are pre-vaccination samples; (2) few samples have titer near the cutoff. Therefore, this analysis does not provide confirmation of the quality of studies 009/010 post vaccination immunogenicity endpoints data.
  4. Conclusions
    A –b(4)----- of hSBA MenY titers was observed consistently for re-tested samples from studies Hib-MenCY-TT-005 and -007/-008. The re-tested samples from Hib-MenCY-TT-013 added as sentinel samples in routine hSBA testing of samples from Studies -009 and -010 also showed –b(4)------ in hSBA MenY titers from their initial reference values, though no further drift was observed from week 1 to week 4 of the 009/010 testing period. The applicant’s explanations for items 1 and 3 do not seem to address the assay stability issue adequately. The retest data of sentinel samples from study 013 tested in the 009/010 testing provide only limited information due to a large amount of missing data. Assay stability remains an unresolved issue. The data for assay controls covering the testing period from study 005 to studies 009/010 may provide useful information regarding the assay stability.
  5. Comments to Applicant
    1. In your response to item 1, you concluded that the hSBA MenY assay and ------------(b)(4)- are stable, but attributed the –b(4)----- of MenY titers for the re-tested samples from study Hib-MenCY-TT-005 to the combined factors of ------------------------------------------(b)(4)-------------------------------------------------------------, and the use of a     -------(b)(4)-------- lot which had exceeded the shelf-life. These factors/conditions were either improved or no longer present for the re-tested samples from studies -007/-008 and -013, yet, a consistent pattern of reduction in MenY titer from the initial reference titer was observed. Specifically, CBER has the following comments on the data presented in your updated validation document 2009-V-008-R:

      a). The results of your extended --------(b)(4)------- experiment showed that the assay is robust for up to -------------------------(b)(4)---------------------------------. These data do not seem to support your hypothesis for the observed –b(4)----- in MenY titers. Please comment.

      b). You presented quality control charts for positive controls in the MenY hSBA assay (QC1 and QC2) for the testing period from July 2009 to June 2010 to demonstrate assay stability. We noticed that in the QC chart for Control 1                   -------(b)(4)------- for the period from July 2009 to January 2010 (Section 4.3.8, Figure 2, page 30), many data points are below the lower limit. For the period from February 2010 to June 2010 (Section 4.3.8, Figure 3, page 30), the target value for Control 1 (-----(b)(4)------) is changed to a higher level. Although all data points are within the control limits, the range between the lower and upper control limits becomes much wider. Please explain why you believe that the hSBA MenY assay is stable.

      Furthermore, the time period covered by these QC charts (July 2009 to June 2010) begins several months after the testing of studies 009 and 010 samples (Jan 2009 to February 2009) is completed. Thus, these QC charts do not provide information regarding the assay stability during the testing of samples from the clinical studies supporting this BLA application. Please submit the QC charts for controls covering the testing period from study -005 to study -010, including details on how the control limits were set.
       
    2. A large amount of missing data in the MenY assay data for the samples from Study -013 added as sentinel samples in routine hSBA testing of samples from Studies -009 and -010 may bias the results of assay stability evaluation, especially for week 1. Out of the 38 samples tested in week 1, only 20 samples have valid titer results.  Eight samples have missing value code “TC”, meaning that they were supposed to be retested at a lower dilution because less than 2 dilution points of the curve have –b(4)---------. Since these missing TCs are not missing at random, excluding these samples could make the GMR at week 1, relative to the initial reference, higher than the true ratio had those TC samples been re-tested (based on their titers at weeks 2-4). Overall, the GMRs during the four weeks clearly suggest that reduction in MenY titers from the initial reference values is also present for these sentinel samples from study -013. The concordance analysis may also not be useful for evaluating this unidirectional (-b(4)----) assay stability issue and its potential impact on the clinical studies results, because there are many samples with b(4) titer initially and few samples near the cutoff point. Please comment.