• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

Vaccines, Blood & Biologics

  • Print
  • Share
  • E-mail

Section Contents Menu

Vaccines

Serology Review Memo, January 19, 2012 - MenHibrix

 

Date:               January 19, 2012
 
To:                  File for 125363/0
 
From:              Freyja Lynn, Consumer Safety Officer, DBPAP/OVRR
 
Through:         Jay E. Slater, M.D., Director, DBPAP/OVRR
 
Subject:           Serology Review Memo for BLA Supplement 125363/0/21 (MenHibrix)
Response to CBER CR letter of September 21, 2011
 
Sponsor:         GlaxoSmithKline (GSK)
 
Documents Reviewed:
 
BLA supplement 125363/0/21, file name: “efficacy1-3.pdf.”
 
Summary/Background:
 
Due to ongoing concerns regarding the quality of the meningococcal serum bactericidal assays (SBA) used to assess the efficacy of the Group Y component of the vaccine, CBER issued a second CR letter on 21 September 2011, with three questions related to the serology. GSK submitted a partial response (125363/0/19) on 26 October, 2011 and this partial submission was discussed in a telecon between GSK and CBER on 8 November, 2011. Based on the questions posed in the CR letter and the additional feedback from the telecon, GSK submitted a full response to the CR letter on 1 December, 2011.
 
Review of Response to CR letter issued 21 September 2011:
 
CBER questions are listed below bolded. Summary of my review of their response follows each item. Suggested CBER questions and comments to be sent to GSK are found at the end of the memo.
 
1.                  In response to Item 1a, you indicated that study 005 sera were not handled according to the SOP. You indicated that the study 005 sera ----------(b)(4)----------------------------------------------------------------------------------------------------------, which is significantly more than the validated ---------(b)(4)---------. You hypothesized that     ----------(b)(4)----------- cycles may have led to --(b)(4)--- hSBA titers in the Men Y retest. You then tested this hypothesis by retesting immune sera subjected to            ---------(b)(4)--------- cycles in the hSBA. The results of your extended ----(b)(4)---     -------- experiment showed that the assay is robust for up to -(b)(4)- and there may be a small impact at -(b)(4)--. Thus, these data do not support your hypothesis that the observed --(b)(4)-- in MenY titers is attributable to excessive -------(b)(4)---------. In addition, you have also suggested that the -(b)(4)- of the study 005 sera may have played a role in the --(b)(4)-- of hSBA titers, but you have not tested this hypothesis and have not established the --------(b)(4)-------- at which hSBA titers begin their       --(b)(4)--. We conclude that the reasons for the --(b)(4)-- in study 005 hSBA titers remain unknown. Please provide any additional information you may have that would explain the decrease in study 005 hSBA results.
 
The response is adequate. However, note that the Bland & Altman figures shown in Figure 2 on pages 16 and 18 (studies 006 and 008) indicate the disparity of results between the repeated analyses is greater for samples with lower titers, indicating a potential loss of precision at the low end of the assay range.
 
2.         In response to Item 1b, you presented a table of reference and retest hSBA values for selected samples (Table 2) tested in the Y assay. We note that the retesting of these samples demonstrated a lower value in the retest for 11 of the -(b)(4)- samples. Nine of the -(b)(4)- retest hSBA values are greater than two-fold lower than the reference values and two of the -(b)(4)- retest hSBA values are greater than or equal to four-fold lower than the original values. Thus, these data do not support the stability of the hSBA for the Men Y over time. 
 
In response to Item 3a, you provided data relevant to the reliability of the hSBA for Men Y. Specifically, you presented a table of hSBA values from the Y assay for the sentinel samples included in study HIB-MENCY-TT-013 (Table 5). We note that seven out of -(b)(4)- samples show a greater than four-fold discrepancy between the highest and lowest reported values. Four samples show results both above and below a titer of -(b)(4)-, including one sample with a titer in the --------(b)(4)-------------------------. The samples with only one replicate provided are not included in the totals. In addition, a substantial amount of data is missing from the table which precludes a complete assessment of assay stability. 
 
Together, the ---(b)(4)-- titers seen in the repeat analyses for samples from Study Hib-MenCY-005 (refer to item 1 above), in conjunction with the ---(b)(4)--- titers and the discrepancies in the data submitted in response to Items 1b and 3a (refer to item 2 above), raise concerns with regard to the ability of the hSBA assay for the Y strain to produce reliable and consistent data over time. While it is acknowledged that sample storage may have been one factor leading to ---(b)(4)-- hSBA titers in the Men Y retest, adequate control of the assay during the sample analysis of the pivotal studies is critical. In this regard, we request the following additional information:
 
a.        To evaluate whether small changes in the assay over time would have affected all groups from a given study equally, please provide the blinding and randomization scheme for analysis of the samples from the pivotal studies.
 
The blinding was not complete as the assay technicians were aware of the visit number for each subject and the samples are assayed in the order in which they are received, however the randomization of the subjects does provide blinding and randomization with regard to the study groups. Additionally, the -------------(b)(4)------------- system helps to prevent operator bias. The response is adequate. GSK should consider improving the technician blinding and the randomization of the samples in the assay with regard to visit.
 
b.         Given the apparent instability of the hSBA for the Y strain, please address the following:
 
i.          Please provide data that demonstrate that the ------(b)(4)------ algorithm maintains consistent assay performance across changes in control and complement lots. Please provide a trending analysis for the ------(b)(4)----- values that demonstrates consistent assay performance within control and complement lots. Please show that the ------(b)(4)----- algorithm is independent of sample titer, i.e., that the variance of the -------(b)(4)------ ratio is constant relative to titer.
 
The data regarding the consistency of the assay performance across changes in controls and complement lots are sufficient to support that no major shifts in assay performance appear to have occurred as a result of the reagent changes. However, the -----(b)(4)---- of control values before trending analysis may mask changes. The effectiveness of the ---(b)(4)--- control values to monitor assay performance as compared to non ---(b)(4)--- control values has not been demonstrated. The benefits versus risks of -----(b)(4)---- have not been fully explored. If            ----(b)(4)---- is not shown to improve assay performance or correct for changes across reagents, then ----(b)(4)---- should not be performed.
 
Figure 3, page 21 shows the trends of the ------(b)(4)---- factor over the time during which the samples from pivotal studies were performed. The data indicate that the distribution of the          ------(b)(4)----- factor was not performing as expected as the distribution of the factors was centered well below -(b)(4)- for most of the assays performed. In particular the --(b)(4)--- value appears to be trending lower for the time frame during which samples from studies 009 and 010 were analyzed (January 2009 – February 2009). The control charts include only the ---(b)(4)--- values for the controls, therefore the actual values of the controls cannot be examined to determine what caused that decrease in -----(b)(4)------ factors. The use of --(b)(4)--- control values to track and trend control performance is misleading as ----(b)(4)----- forces the values of the controls into mirror images of each other. Performance of each control cannot be evaluated.
 
The data referenced on page 25 on the impact of the -----(b)(4)---- factor on the precision of the assay with respect to sample titer indicate that the -----(b)(4)----- factor does not improve the precision of the assay, and in fact for the MenC assay, the -----(b)(4)----- actually decreases the precision for samples less than a titer of -(b)(4)-. Given that the -----(b)(4)----- factor does not improve assay precision, it should not be used.
 
The data presented in Figures 5 and 6 (page 26) and additional analyses performed using the data supplied by GSK also do not indicate that the -----(b)(4)----- provides any benefit with regard to assay precision or stability. The precision profiles, the sample value trends over time and the distribution of the variances all indicate that ----(b)(4)----- does not improve assay performance.
 
The ------(b)(4)---- factor has been consistently centered below 1, with the mean -----(b)(4)-----   ----- of -(b)(4)- during the sample analysis for the pivotal studies. Therefore raw low titer values are increased by the ------(b)(4)----- factor creating a gap in the data between samples that are negative and those greater than -(b)(4)-. This has effectively shifted the LOD from ----(b)(4)----. 
 
ii.         Please present the analysis that demonstrates that the four-parameter model can be appropriately fitted to the bacterial count data generated in the assay. Please describe how the a and d parameters for each sample are determined and controlled. Please comment on whether the curve fitting is constrained, and if so, please explain how it is constrained. Please provide the basis for the criterion that each sample has an R2 greater than -(b)(4)-.
 
GSK has not provided sufficient justification to use the four parameter --------(b)(4)----------        ------------ to estimate ---(b)(4)---. In general best practice is to use the simplest relevant data reduction algorithm. The curve fitting for the (b)(4)- is unconstrained and the a and d parameters are not controlled. To support the use of the -(b)(4)-, GSK compared the non -(b)(4)- values obtained using the -(b)(4)- data reduction algorithm to the discontinuous (noninterpolated) values and to the values interpolated using simple two point linear regression. The discontinuous titers are equal to or lower than the titers estimated using the -(b)(4)- as expected. However, the titers using the -(b)(4)- should be no more than twice the value of the discontinuous titer based on the assumption that the interpolated titer would be between the two dilution points, one above and one below the level at 50% reduction in cfu. The data indicate that the interpolated titers are as high as three times the discontinuous titer, implying shifts of the data beyond what would be expected. The comparison between the values estimated by (b)(4) and those estimated using linear regression demonstrate good agreement between those two methods, which is somewhat inconsistent with the finding that the (b)(4) is in some cases estimating values higher than two fold of the discontinuous titer. The linear regression estimate can not be more than two fold higher than the discontinuous titer by definition and therefore one would expect to see some disagreement between the (b)(4) (which shows some values greater than two fold higher than the discontinuous titers) and the linear regression models (which by definition cannot be greater than two fold higher).  
 
To justify the criterion for the R2 of greater than (b)(4), GSK refers to IND (b)(4), serial (b)(4) in which responses to a January 17, 2008 letter from CBER were submitted. GSK states that the criterion would “reject most of the aberrant killing curves.” No data were submitted to the IND in serial (b)(4), nor to this BLA to support that contention. The criterion of (b)(4) is unsupported.
 
While for the purposes of this BLA, the -(b)(4)- data reduction algorithm does not appear to be affecting the stability of the Group Y SBA, the use of the -(b)(4)- algorithm, and the system suitability criteria, should be revisited for future studies with regard to the effect of the (b)(4)- on precision and accuracy of the assay.
 
iii.        You presented quality control charts for positive controls in the MenY hSBA assay (QC1 and QC2) for the testing period from July 2009 to June 2010 to demonstrate assay stability. We notice that in the QC chart for Control 1 (-------(b)(4)------) for the period from July 2009 to January 2010 (Section 4.3.8, Figure 2, page 30), many data points are below the lower limit. For the period from February 2010 to June 2010 (Section 4.3.8, Figure 3, page 30), the target value for Control 1 (--------(b)(4)-------) is changed to a higher level. Although all data points are within the control limits, the range between the lower and upper control limits becomes much wider. In light of these observations, please explain why you conclude that the hSBA MenY assay is stable.
 
GSK states that the two controls have different target values because they were different samples. They go on to state that the CV’s for the two controls were also different, thus accounting for the wider control limits. In Table 9, they report the CV of the first control as 15% and the CV of the second control as 28-37%. They state that these CV values are acceptable as the “normally expected” CV is -(b)(4)-. GSK additionally explains that acceptance ranges are calculated using the log10 transformed values for each control, according to a formula that incorporates an alpha risk fixed to 1% to have only 1% of false warnings by out of limit controls.
 
The precision should be reasonably constant across the assay. Controls should reflect this precision so that the control adequately represents the samples in terms of precision.  An expected assay precision of -(b)(4) is not consistent with a control whose variability is only 15%. Either the control is not typical, or the assay variability is not under control.
 
The use of the log10 transformed values was not consistently applied to the control limits. For example the target and limits for the final limits for ---------------------(b)(4)--------------------        -------------------- are estimated using the arithmetic scale (see Supplement 13, pages 96 – 97). Also the CV (standard deviation divided by the mean) is not generally used for assessing variability of log normally distributed values.
 
The data presented indicate that samples from studies 009 and 010, which are pivotal to this application, were run in January and February of 2009. According to the time lines provided, the lot of complement was not changed during the course of the sample analysis. However, the time lines, control trending charts and ------(b)(4)----- factor trending chart raise two issues regarding the stability of the assays and the ability to interpret the clinical data. The first is that both controls used during that time period appear to have been held to “temporary” criteria, i.e. the limits set for the controls were much wider than the subsequent “final” range (see supplement 7, figure 4 on pages 80 and 81). The wider limits appear to have led to accepting assays that would have been rejected if the final ranges had been used. As a result, the variability of the assay during the time the temporary range was used would may be higher than when tighter limits were used. A second issue is the performance of the -----(b)(4)----- factor during the time frame in question (see Figure 3, page 22, and my comments regarding -----(b)(4)----- above under 2b.i.). The distribution of the ------(b)(4)---- factor was centered well below 1 with values lower that 0.2 common. The trend during that time was towards a lower ------(b)(4)----- factor over time. The randomization of the samples during testing would help to mitigate any trends in the assay performance, however, the variability in the ------(b)(4)----- factors may indicate increased variability.
 
 
iv.        The time period covered by these QC charts (July 2009 to June 2010) began several months after the testing of samples from studies 009 and 010 (Jan 2009 to February 2009) was completed. Thus, these QC charts do not provide information regarding the assay stability at the time the testing of samples from the clinical studies supporting this BLA was performed. Please provide data that support the stability of the assay covering the actual testing period from study -005 to study -010. Data that would be supportive include all QC charts for controls with trending analyses, reagent qualification data for any new controls or complement introduced during the analysis of samples from a given study, and all sentinel data. A detailed and continuous time line depicting the changes in controls and complement lots during the entire testing period should also be included.

As stated above under Item 2.b.iii, the control charts and time lines indicate that no critical reagents were changed during analysis of studies 009 and 010. The chart of the ----(b)(4)------ factor indicates a downward trend during analysis of the samples (see Figure 3, page 22). The controls used during analysis had preliminary limits which were much wider than the final limits set after completion of the sample analysis for studies 009 and 010 (see supplement 7, Figure 4, pages 80-81).  Complete review of the trending of the control values cannot be performed as all the control data are presented using the ---(b)(4)--- values. While no substantial trend in the data can be seen with the information presented, the precision of the assay was not clearly controlled.
 
3.                   We are concerned that missing data for the samples from Study -013 added as sentinel samples in routine hSBA testing of samples from Studies -009 and -010 may have biased the results of the Men Y assay stability evaluation, especially for week 1. Out of the -(b)(4)- samples tested in week 1, only 20 samples have valid titer results. Eight samples have a missing value code “TC”, meaning that they were supposed to be retested at the lower dilution because less than 2 dilution points of the curve have -(b)(4)- killing. Since these missing TCs are not missing at random, excluding these samples could make the GMR at week 1, relative to the initial reference, higher than the true ratio had those TC samples been re-tested (based on their titers at weeks 2- 4). Overall, the GMRs during the four weeks clearly suggest that a reduction in MenY titers from the initial reference values is also present for these sentinel samples from study -013. Also, the concordance analysis may not be useful for evaluating this unidirectional --(b)(4)-- assay stability issue and its potential impact on the clinical studies results, because there are many samples with titers b(4) initially and few samples near the cutoff point. Please comment.
 
The data and explanations regarding the sentinel samples from study 013 repeated during the analysis of samples from studies 009 and 010 are not consistent with the reagent qualification criteria and results as presented in response to Item 2b.iv. The b(4) in titers between the original and retest values for the sentinel samples is said to be due to a change in the complement lot between the original test and the retests. Complement Lot -(b)(4)- was used in the original testing of study 013 while lot -(b)(4)- was used during the retesting. However, during reagent qualification, each lot of complement was tested with a panel of positive samples and the geometric mean ratio (GMR) of positive samples analyzed using the new versus the old reagents was required to be between ---(b)(4)--- (see page 39). The GMR between the lots -(b)(4)- and     -(b)(4)- during qualification was 0.99. The data presented on page 42 and in Figure 3 (page 22) indicate that the mean ------(b)(4)----- factor shifted dramatically from -(b)(4)- when lot -(b)(4)- was replaced with -(b)(4)-. The GMRs for the sentinel samples between the original and retest values, which span the shift between lots -(b)(4)- and -(b)(4)-, range from --(b)(4)- (see page 42). The fact that these GMRs are consistent during the four week time period during which the sentinel samples were retested indicates that the assay does not appear to be fluctuating wildly, but overall the assay performance raises concerns regarding the level of control afforded by the reagent qualification and the effectiveness of the ------(b)(4)----- algorithm. 
 
The reagent qualification is designed to prevent large shifts in reported results when reagent lots are changed. The criteria indicate that the shift must be between -----(b)(4)----. However, the GMR comparing the -(b)(4)- sentinel samples results across the lots of complement indicate that the shift was as much as -(b)(4)-, which is beyond the limit proscribed for acceptance of a new lot. The implication is that the reagent qualification scheme is not adequately screening new lots of complement to prevent shifts wider that --(b)(4)--.
 
The -----(b)(4)----- factor does not appear to be adjusting the values as one would expect to counteract the effects of reagent changes. If the assay shifted due to a change in the lot of complement, the premise is that the ------(b)(4)---- factor will adjust for that shift. Yet despite a large drop in the -----(b)(4)---- factor, the retesting data indicate that the GMR between samples tested with complement lot -(b)(4)- versus complement -(b)(4)- is out of the expected range of    -(b)(4)-. The ---(b)(4)-- algorithm does not appear to be effectively mitigating the impact of reagent changes on the assay. 
 
Figure 11 is the trending plot for retest of samples from study 013 from January 2009 to August 2011. GSK states that this figure represents normal assay variability and stability. While the stability of the assay over time is supported by this figure, the overall assay precision is not, with ratios of the retest to the reference ranging from approximately -(b)(4)-. This is a -(b)(4)- fold range.
 
Combination or comparisons of results that span reagent lots should be made with caution. The effects of changing reagents and the ability of the –b(4)-------- factor to control the assay are unclear. Randomization of the samples run with in a study should mitigate assay shifts, however the precision of the assay is not fully supported.
 
Recommendation
 
The group Y SBA was likely stable during the analysis of the pivotal clinical samples, however the precision was not well controlled. To be conservative, I recommend the clinical endpoints be recalculated using -----(b)(4)----, discontinuous data with both ----(b)(4)-- as cutoffs. Alternatively, if the interpolated and normalized data are used, the clinical endpoints should be estimated using -----(b)(4)---- cutoffs. 
 
Suggested comments for GSK:
 
Item 2.a.
 
  1. Please consider full blinding of operators as to visit number. Additionally, analysis should be delayed from the study start to ensure that a mix of visits are analyzed together and early samples are not run predominantly at the beginning of the sample analysis and late samples run late in the analysis. Please acknowledge.
 
Item 2.b.i
 
  1. Overall, the data indicate that no major shift in the assay occurred during analysis of the samples from studies 009 and 010. However, the use of the ------(b)(4)---- factor to stabilize the assay and improve precision has not been demonstrated. The data referenced on page 25 on the impact of the -----(b)(4)----- factor on the precision of the assay with respect to sample titer indicate that the -----(b)(4)----- factor does not improve the precision of the assay, and in fact for the MenC assay, the ------(b)(4)----- actually decreases the precision for samples with a titer of -----(b)(4)-----.  The data presented in Figures 5 and 6 (page 26) and additional analyses performed using the data supplied by GSK also do not indicate that the -----(b)(4)----- provides any benefit with regard to assay precision or stability. The precision profiles, the sample value trends over time and the distribution of the variances all indicate that ------(b)(4)---- does not improve assay performance.  Additionally, the -----(b)(4)----- factor has been consistently centered below 1, with the mean --------------(b)(4)------------- during the sample analysis for the pivotal studies. Therefore raw low titer values are increased by the -----(b)(4)----- factor creating a gap in the data between samples that are negative and those greater than (b)(4). This has effectively shifted the LOD to ---(b)(4)---. Given that the -------(b)(4)----- factor does not improve assay precision, it should not be used. See additional comments under Items 2b.ii, iii and iv. Please recalculate the clinical endpoints using non -----(b)(4)----, discontinuous data.
 
Item 2.b.ii.
 
  1. The data indicate that the use of the -(b)(4)- data reduction algorithm has not contributed substantially to assay instability. However, the effects of the use of the -(b)(4)- may be affecting assay precision. As the serological correlate is based on discontinuous titers, please recalculate the clinical endpoints using non -----(b)(4)--- discontinuous titers to demonstrate the results are consistent regardless of the data reduction algorithm. 
 
Best practices in assay validation recommend that the simplest data reduction algorithm be used. We do not find your justification for using the -(b)(4)- data reduction algorithm to be sufficient and recommend that you reconsider the use of a linear interpolation between the points above and below the point of -(b)(4)- killing, or the use of the discontinuous titer. If you wish to continue use of the -(b)(4)- algorithm in future submission, we will require additional information regarding the system suitability criteria, the rate of nonconvergence, and impact of the -(b)(4)- on assay precision. This comment applies to hSBA for all serogroups. Please acknowledge.
 
Item 2.b.iii and iv.
 
  1. Control of the precision of the assay during routine use is not well supported.  
    1. The precision should be reasonably constant across the assay. Controls should reflect this precision so that the control adequately represents the samples in terms of precision. An expected assay precision of 45% is not consistent with a control whose variability is only 15%. Either the control is not typical of clinical samples, or the assay variability is not under control.
    2. The use of the log10 transformed values was not consistently applied to the control limits. For example the target and limits for the final limits for                  -------------------------------(b)(4)------------------------------- are estimated using the arithmetic scale (see Supplement 13, pages 96 – 97). Also the CV (standard deviation divided by the mean) is not generally used for assessing variability of log normally distributed values.
    3. The rejection rate of the assays is based a formula that incorporates an alpha risk fixed to (b)(4) to have only (b)(4) of false warnings by out of limit controls. The control charts presented in Supplement 7 indicate that for several controls, the rejection rate is much higher than (b)(4). This would indicate that the control limits are not set correctly or that the assay precision is not under adequate control.
    4. During analysis of samples from studies 009 and 010, both controls used appear to have been held to “temporary” criteria, i.e. the limits set for the controls were much wider than the subsequent “final” range (see supplement 7, figure 4 on pages 80 and 81). The wider limits appear led to accepting assays that would have been rejected if the final ranges had been used. As a result, the variability of the assay during the time the temporary range was used may be higher than when tighter limits were used.
    5. Figure 3 page 22, and Supplement 9 show the trending of the -----(b)(4)------ factor. The -----(b)(4)---- factor varies considerably, from greater than ---(b)(4)-------. Even within a very short period of time (less than a month), the factor varies from ----(b)(4)---, indicating substantial assay variability. 
    6. Figure 11 is the trending plot for retest of samples from study 013 from January 2009 to August 2011. GSK states that this figure represents normal assay variability and stability. While the stability of the assay over time is supported by this figure, the overall assay precision is not, with ratios of the retest to the reference ranging from approximately -(b)(4)-. This is a -(b)(4)- fold range.
 
The data suggest that the precision of the assay may not be adequately controlled and you estimate the normal variability to be up to -(b)(4)- fold. The serologic correlate is based on seropositivity in the hSBA, traditionally those subjects whose serum titers are greater than or equal to -(b)(4)-. Given the uncertainty with which the values in your assay are estimated and the need to link the results to the traditional -(b)(4)-, please recalculate the clinical endpoints for -(b)(4)- fold and -(b)(4)- fold greater than the -(b)(4)----------. Alternatively, as requested above, please calculate the clinical endpoints using the non     ------(b)(4)-----, discontinuous data.
 
Item 3.
 
  1. The assay appears to be subject to shifts in performance, potentially related to reagent lot changes. Reagent qualification and ------(b)(4)----- do not appear to be adequately controlling the shifts in performance. The drop in titers between the original and retest values for the sentinel samples from 013 that were tested during analysis of sera from 009 and 010 is said to be due to a change in the complement lot between the original test and the retesting. Lot -(b)(4)- was used in the original testing of study 013 while lot (b)(4) was used during the retesting. However, during reagent qualification, each lot of complement was tested with a panel of positive samples and the geometric mean ratio (GMR) of positive samples analyzed using the new versus the old reagents was (b)(4), well within the reagent qualification limits of beween ---(b)(4)--- (see page 39). The data presented on page 42 and in Figure 3 (page 22) indicate that the mean -----(b)(4)---- factor shifted dramatically from -(b)(4)- when lot -(b)(4)- was replaced with -(b)(4)-. However, the GMRs for the sentinel samples between the original and retest values, which span the shift between lots -(b)(4)- and -(b)(4)-, range from -(b)(4)- (see page 42). 
    1. Reagent qualification is designed to prevent large shifts in reported results when reagent lots are changed. The criteria indicate that the shift must be between        -(b)(4)-. However, the GMR comparing the 013 sentinel samples results across the lots of complement indicate that the shift was as much as -(b)(4)-, which is beyond the limit proscribed for acceptance of a new lot. The implication is that the reagent qualification scheme is not adequately screening new lots of complement to prevent shifts wider that ---(b)(4)---.
    2. The -----(b)(4)--- factor does not appear to be adjusting the values as one would expect to counteract the effects of reagent changes. If the assay shifted due to a change in the lot of complement, the premise is that the ------(b)(4)---- factor would adjust for that shift. Yet despite a large drop in the -----(b)(4)----- factor, the retesting data indicate that the GMR between samples tested with complement lot -(b)(4)- versus complement -(b)(4)- is out of the expected range of ---(b)(4)---. The --------(b)(4)------- algorithm does not appear to be effective at mitigating the impact of reagent changes on the assay. 
    3. The factor appears to have substantially shifted in early 2009 so that it is no longer centered on -(b)(4)- even though controls, ----------(b)(4)------- and complement lots have all changed throughout that time.
 
As studies 009 and 010 were run without changes in critical reagents, and the samples were randomized by group in the assay, the effects of reagent changes on the precision and accuracy are mitigated. However, combination or comparisons of results that span reagent lots should be made with caution. The effects of changing reagents and the ability of the -----(b)(4)---- factor to control the assay are unclear. The precision of the assay is not fully supported. The use of the -----(b)(4)------ factor in future studies will need to be supported by additional data that indicate that the factor improves assay precision for all serogroup hSBA. Please acknowledge.