Final Statistical Summary Review for PMA P970003/S50 (Original and Various Amendments), Vagus Nerve Stimulator (VNS) Therapy System for Depression, Cyberonics, Inc.
I. Introduction
The VNS system is indicated for the adjunctive long-term treatment of chronic or recurrent depression in patients who are experiencing a major episode that has not had an adequate response to two or more antidepressant treatments. This review summarizes the important statistical issues and results for pivotal D-02 study (VNS plus Standard of Care), observational D-04 (Standard of Care alone), and D-02/D-04 comparison. The primary efficacy endpoint is the comparison of average rate of change per month (slope) and average change from baseline between D-02 and D-04 patients for the evaluable patient population. The secondary efficacy endpoint is the comparison of proportions of Response (defined as ³ 50% decrease in scores from baseline) at 12 months.
The Hamilton Rating Score for Depression (HRSD-24) is the primary efficacy endpoint in the D-02 pivotal study. However, since HRSD-24 scores were collected only at baseline and at 12 months in the D-04 observational study, the Inventory of Depressive Symptomatology Self Report (IDS-SR) was used as the primary efficacy endpoint in the D-02/D-04 comparison via repeated-measures linear regression (RMLR) analyses. The RMLR requires both patient baseline and multiple post-baseline measurements to estimate average rate of change per month (slope) and the difference of two true slopes for D-02/D-04 comparison.
II. Multi-Center Study Data
There are 22 sites (centers) which participated in either D-02 or D-04 studies. Of these 22 sites, 12 sites participated in both D-02 and D-04 studies (called overlapping sites), 9 sites enrolled D-02 patients but no D-04 patients, and 1 site enrolled D-04 patients but no D-02 patients. The numbers of patients for “evaluable” and “12-month completer” patient population, separately by all participating and overlapping sites, are shown in Table 1.
Table 1. Number of Patients (N)a by “All Sites” and “Overlapping” Sites,
D-02/D-04 Study
|
Site |
D-02 Long-Term |
D-02 Evaluable |
D-02 12-Month Completers |
D-04 Evaluable |
D-04 12-Month Completers |
|
All (22) |
233 |
205 (185 Unipolar, 20 Bipolar) |
177 |
124 |
112 |
|
Overlapping (12) |
165 |
147 |
128 |
120 |
108 |
a. Sample size (N) was justified for the comparison of two response proportions, the secondary efficacy endpoints; not for the comparison of two slopes, the primary efficacy endpoint. The detailed distribution of patients by clinical site (Non-overlapping and Overlapping) is shown in Table 10, Appendix 1.
III. D-02 Pivotal Study
The D-02 study included patients whose HRSD-24 ³ 18 anytime during the 12- month follow-up, and HRSD-24³ 29 at acute phase (at 3 months). The following Table 2 provides a brief summary for D-02 group patients
Table 2. Brief Summary for D-02 Study, All Sites
|
|
Acute (3 months) |
Long-Term (1-year) |
|
Study Design |
Double-blind, randomized, parallel, Active VNS versus Sham control, Multi-center (22) |
(Delayed-treatment group) |
|
Follow-up |
Baseline (2), Implanted, 2 weeks,…3 months |
Monthly in the first year, quarterly thereafter |
|
Clinical Outcome |
HRSD-24 Score, Primary; IDS-SR Score, Secondary |
HRSD-24 score (Per-Protocol) |
|
Primary Endpoint |
Comparison of two response proportions |
Average rate of change per month (slope)-Repeated-Measures Linear Regression (RMLR) |
|
Result |
HRSD-24 (N = 221) 15% (17/111) VNS, 10% Sham (11/110) p = 0.31 (Fisher’s exact) IDS-SR ( N = 215) 17.4% (19/109)VNS, 7.5% ( 8/106) Sham, p = 0.039 (Fisher’s exact) |
Evaluable ( N = 205) [Slope = -0.45, Standard error = 0.05, 95% CI: (-0.55, -0.34), p<0.001 to reject the true null hypothesis (slope = 0)] 12-Month Completers ( N = 177) [Slope = -0.47, Standard error = 0.06, 95% CI: (-0.58, -0.36), p<0.001] |
IV. D-02 and D-04 Comparison
Propensity Score (PS) Adjustment
Since D-04
is an observational study (Standard of Care alone), evaluation of true device effect must control for
potential bias or confounding effect in differences for individual patient demographic characteristics and
clinically important baseline
covariates between D-02 and D-04 group patients.
The PS approach is to derive an overall summary composite score of sponsor’s selected 17 patient binary or continuous covariates [age, gender, bipolar versus unipolar depression, lifetime electroconvulsive therapy (ECT) use, length of current major depressive episode (MDE) in months, average number of lifetime episodes of depression, percent of patients received ECT in lifetime, percent of patients received ECT in current MDE, percent of patients with suicide attempt in lifetime or in the past 12 months, and others]. The purpose of PS analysis is to reduce bias in non-randomized, observational studies, such as in the D-02/D-04 comparison. Statistical logistic regression is used to predict D-02 treatment assignment conditional on the individual patient’s covariates. The resulting individual patient predicted probabilities of receiving active treatment
group
(D-02) and control (D-04) groups were then ordered to form a 5-subgroup or quintiles based on the estimated
propensity scores. For example, the
first quintile group contains
approximately 20% of the patients with the lowest D-02 PS and the last group contains approximately 20% of the
patients with the highest D-02 PS.
PS can only adjust for observed
covariates, not for unobserved ones. PS analysis may not eliminate all
selection bias, particularly hidden bias.
In my previous reviews, this reviewer asked the sponsor to provide the following information regarding their PS analysis:
Justify selection criteria for fitted logistic regression model:
Graphical display (e.g., Bar chart) of the distribution of PS quintile means (for continuous covariates) or quintile proportions (for binary covariates) between
D-02 and D-04 patients;
For each selected patient covariates (17), prepare statistical analyses for both before-and-after PS adjustment between D-02 and D-04 patients. Explain the degree of covariate unbalance before PS adjustment and covariate balance (or unbalance) after adjustment;
Explain 2-way analysis of variance including main effect (treatment group, PS quintile) and their interactions.
The sponsor has responded to the above comments in the March 17, 2004 Amendment # 4.
Repeated-Measures Linear Regression (RMLR) Analysis
The RMLR analysis is used to evaluate average rate of change (slope) and average change in IDS-SR scores from baseline to the 12-month follow-up. SAS PROC MIXED was used to analyze the 12-month follow-up data. No missing data imputation is needed to run SAS PROC MIXED since missing data are assumed to be missing at random (MAR), which means that probability of missing data is independent of future observed data. The last observation carried forward (LOCF) analysis was also prepared by the sponsor for comparison purpose. The patient covariates used in the general mean response mixed model include several fixed-effect study factors [9 pooled sites with some pooled sites containing only D-02 patients (see Table 18.2, March 17, 2004 submission), treatment (D-02 versus D-04), 5-level grouped PS quintiles, baseline IDS-SR score, indicator variables for follow-up time at 3, 6, 9, and 12 months, and treatment by time interactions]. The spatial power covariance structure is used to count for correlation among different follow-up times.
In the March 26, 2004 email, FDA asked the sponsor to respond to the following questions:
Provide an analysis the IDS-SR primary efficacy endpoint in the RMLR analysis, HRSD-24 secondary efficacy endpoint, and Response/Non-Response (see following section) proportions from only those sites that enrolled for both D-02 and D-04 (overlapping sites, see Appendix 1) for unipolar and bipolar patients combined.
Repeat the above analyses for censored patients (i.e., additions or changes in either antidepressant drugs or ECT).
The sponsor has responded to the above FDA’s comments. For censored patients approach in RMLR analyses, the sponsor indicated that “IDS-SR raw scores were censored such that the value from the patient’s IDS-SR measurement obtained prior to their first increase in the antidepressant resistance rating (ARR) score was carried forward and replaced all of the patient’s subsequent, non-missing, IDS- SR measurements.”
Please note that the current ECT addition or change during the follow-up was not discussed above.
Comparison of Proportions of Response
(for 12-Month Completers)
The comparison of two response proportions (³ 50% reduction from baseline in IDS-SR or HRSD-24 scores), defined as one of several other secondary efficacy endpoints, is discussed in this summary review. However, the direct, simple comparison of two response proportions between D-02 and D-04 patients, without adjusting for individual patient baseline IDS-SR or HRSD-24 scores and other clinically important patient covariates, is subject to potential bias. The 12-month completers are the patients who were in close compliance with the scheduled follow-up, may provide the “best-case” scenario as compared to those who did not complete the 12-month study. The selected cutoff point (³ 50% from baseline, response; else, non-response) is also subject to measurement
error, variability of the IDS-SR or HRSD-24 scores, and serial correlation of repeated-measures data. Statistical logistic regression is more appropriate than simple comparison of two proportions by taking important patient covariates (site, baseline IDS-SR or HRSD-24, and others) into the model building. Sensitivity analysis may be helpful to evaluate robustness of Response/Non- Response outcomes by various cutoff points. Appropriate statistical methods for pooling of multi-center data, such as meta-analysis, stratified categorical data analysis, will provide precise evaluation of true treatment effect for comparison of two response proportions.
Concordance Between HRSD-24 and IDS-SR Scores in D-02 Study
As discussed above, the IDS-SR score was used in the RMLR analyses to estimate the average rate of change per month (slope) for D-02 and D-04 patients. Sponsor’s justification is that the HRSD-24 score data were collected only at baseline and at the 12 months in the D-04 study, and that the IDS-SR score had been shown as a “good predictor” of the HRSD-24 score from the published literature.
In FDA’s major deficiency letter of March 1, 2004 and FDA’s E-mail dated March 31, 2004, we do not agree that concordance studies reported in the published literature are sufficient to support IDS-SR as a “good predictor” of HRSD-24. Due to wide variability of paired IDS-SR/HRSD-24 scores from patient to patient, pooled-patient correlation and regression analyses are not appropriate. FDA asked the sponsor to prepare the following analyses:
Calculate correlation coefficients between IDS-SR and HRSD-24 scores for each individual patient and a pooled estimated correlation coefficient over all patients by an appropriate statistical method.
Likewise, calculate corresponding results for estimated slopes and their standard errors for each individual patient.
Provide the fitted linear regression model (intercept, slope, or higher terms), the estimated parameter values, standard errors, 95% confidence intervals, and squared multiple correlation coefficient (R-Square) to show goodness-of-fit of sponsor’s regression equation to the observed paired IDS-SR/HRSD-24 data.
Provide graphical displays for all individual patient paired HRSD/IDS-SR data to show all observed individual patient data pairs, and the fitted regression lines.
The sponsor has responded to all of the above FDA’s requested issues. However, the linear regression model assumes that all individual patient IDS-SR/HRSD-24 pairs are independent, but they are actually correlated in the more proper longitudinal data analyses. Nevertheless, this reviewer believes that linear regression analyses based on individual patient paired IDS-SR/HRSD-24 data may still be applied to verify the sponsor’s claimed IDS-SR as a “predictor” of HRSD-24.
Unipolar and Bipolar Patients
In Question # 9 of FDA’s major deficiency letter of March 1, 2004, we asked the sponsor to prepare separate and combined analyses for unipolar and bipolar patients for both Response/Non-Response secondary efficacy endpoint, and RMLR analyses for the primary efficacy endpoint. The sponsor’s responded in their March 17, 2004 submission. The unipolar/bipolar subgroup analyses were not discussed in the original study design. The sample size for bipolar patients is too small to provide any statistically valid conclusion. The distribution of sample size in D-02 and D-04 comparison is shown in Table 3.
Table 3. Distribution of Number of Patients in the D-02 and D-04 Comparison, by Unipolar/Bipolar Patients, IDS-SR (HRSD-24) Scores
|
Group |
D-02 |
D-04 |
|
Unipolar |
|
|
|
Evaluable |
163 (164) |
97 (91) |
|
12-Month Completer |
156 (157) |
97 (91) |
|
Bipolar |
|
|
|
Evaluable |
17 (17) |
15 (13) |
|
12-Month Completer |
17 (17) |
15 (13) |
|
Combined |
|
|
|
Evaluable |
180 (181) |
112 (104) |
|
12-Month Completer |
173 (174) |
112 (104) |
V. Statistical Analyses Results
PS
Analyses
All graphical displays for each of 17 covariates for D-02 and D-04 comparisons are shown in Attachment 20 of March 17, 2004 submission. The graphical displays appear to be acceptable to examine comparability of D-02 and D-04 patient populations with respect to these 17 covariates after PS adjustment.
The before-and-after PS comparisons for 17 covariates are also shown. The statistically significant differences in some covariates between D-02 and D-04 group patients before PS adjustment were non-significant after PS adjustment. For example, a statistically significant p-value <0.001 in percent of patients who received ECT in current MDE between D-02 and D-04 groups became non- significant (p = 0.434) after PS adjustment. Although some PS quintile by treatment interaction is also shown for percent of patients who received ECT in their lifetime and length of current MDE and treatment, the sponsor’s PS adjustment procedures via logistic regression model appear to be acceptable. The final PS quintile by treatment frequency distribution is shown in Table 4.
Table 4. Treatment (D-02/D-04) by PS Quintile Frequency Distribution (Evaluable Patients)
|
PS Quintile Group |
D-02 ( N = 205) |
D-04 ( N = 124) |
|
|
1 |
22 (10.9%) |
43 (34.7%) |
|
|
2 |
39 (19.4%) |
26 (21.0%) |
|
|
3 |
36 (17.9%) |
29 (23.4%) |
|
|
4 |
48 (23.9%) |
17 (13.7%) |
|
|
5 |
56 (27.9%) |
9 ( 7.3%) |
|
|
Total |
201 (100%)* |
124 (100%) |
|
[*: 4 patients excluded from PS analysis]
The above frequency distributions are statistically acceptable. Patients belong to each of the above 5 PS quintile groups were coded as the categorical variable (5 levels) in the RMLR analyses.
RMLR
Analyses, D-02 and D-04 Comparison,
IDS-SR Scores
The following Figure 1 shows the observed and predicted (by RMLR) mean IDS- SR scores by baseline, and each of 4 quarters (9 pooled sites, evaluable patients). The predicted mean IDS-SR scores appear to be close to the observed scores. The predicted differences in mean IDS-SR scores between D-2 and D-4 patients showed smaller improvement than these observed scores between D-02 and D-04 patients. For example, at Quarter 4, the predicted difference (D2 – D4) is -4.8 (33.7 – 38.5) and the observed difference (D2 – D4) is -6.6 (32.6 – 39.2).
The following Figure 2 shows the corresponding mean IDS-SR scores for 12- month completers
For primary effectiveness endpoint (IDS-SR), the differences of average rate of change per month (slope) and their 95% confidence intervals (CI) for Evaluable and 12-month completers are shown in Table 5.
Table 5. Difference of slope (D2 – D4) and the 95% CI,
All Sites, Unipolar/Bipolar Patients Combined, All Sites
|
Patient Population |
Difference (Std Error) |
95% CI for Difference |
|
Evaluable |
-0.397 (0.1) |
(-0.59, -0.21) |
|
12-Month Completers |
-0.452 (0.1) |
(-0.65, -0.26) |
Clinical interpretation is needed to decide whether or not the above results are clinically acceptable.
The sample size for the above D-2 and D-4 comparison is shown in Table 6
Table 6. Sample Size (N) by D-2/D-4 Comparison, All Sites
|
Group |
Baseline |
Q 1 |
Q 2 |
Q 3 |
Q 4 |
|
D2 (N) |
201* |
200 |
195 |
183 |
177 |
|
Missing |
0 |
1 |
6 |
18 |
24 |
|
D4 (N) |
124 |
120 |
119 |
116 |
112 |
|
Missing |
0 |
4 |
5 |
8 |
12 |
*: 4 evaluable patients did not have IDS-SR and/or PS scores available
I have revised part of sponsor’s reported proportions of Response/Non-Response for IDS-SR and HRSD-24 scores, for 12-month completers, as shown in sponsor’s Tables 3 and 4, Volume 19, Clinical Summary (See Tables 7-A and 7-B below)
Table 7-A. FDA’s Revised Proportions of Response for 12-Month Completers, IDS-SR Scores, All Pooled Sites
|
12-Month Data |
D-02 |
D-04 |
p-valuea |
|
Responseb |
22 % (38/173) |
12% (13/112) |
0.027 |
|
LOCF Response |
22% (39/176) |
12% (13/112) |
0.027 |
|
Complete Responsec |
15% (27/180) |
4% (4/112) |
0.001 |
|
LOCF Complete Response |
13% (27/204) |
3% (4/124) |
0.003 |
a. Fisher’s two-sided exact test
b. ³ 50% decreasing change from baseline; c. IDS-SR £14
Table 7-B. FDA’s Revised Proportions of Response for 12-Month Completers, HRSD-24 Scores, All Pooled Sites
|
12-Month Data |
D-02 |
D-04 |
p-valuea |
|
Responseb |
30 % (54/181)) |
13% (13/104) |
0.001 |
|
LOCF Response |
27% (55/205) |
13% (13/104) |
0.004 |
|
Complete Responsec |
17% (31/181) |
7% (7/104) |
0.018 |
|
LOCF Complete Response |
16% (32/205) |
7% (7/104) |
0.029 |
a. Fisher’s two-sided exact test;
b. ³ 50% decreasing change from baseline
c. HRSD-24 £9
Please note that, in Tables 7-A and 7-B, p-values calculated from direct pooling of all cell frequencies (Response/Non-Response by treatment group) over all sites (non-overlapping and overlapping), without preparing appropriate statistical modeling approach or meta-analysis, may be invalid.
Under the section for RMLR, in the FDA’s March 26, 2004 email, we asked the sponsor to reanalyze the IDS-SR and HRSD-24 score data from only these sites that enrolled both D-2/D-4 (overlapping sites) and for censored patients (i.e., additions or changes in either antidepressant drugs or ECT). The following FDA’s revised Tables 8-A through 8-D are for Tables 24-1 through 24-4 shown in the sponsor’s responses of April 2, 2004 to the FDA’s email dated March 26, 2004 (Amendment # 6).
FDA’s Revised Table 8-A
IDS-SR Scores-D-02/D-04 Comparisons (Overlapping sites for both D-02 and D-04 patients only), Evaluable Patient Population (Unipolar and Bipolar patients combined)
|
|
D-02 |
D-04 |
P-value |
95% CI (Least square mean) |
RR(95% CI)b D2 – D4 |
|
N at Baseline |
147 |
120 |
|
|
|
|
Baseline Average |
42.7 |
43.6 |
|
|
|
|
12 Month Data |
N = 131 |
N = 108 |
|
|
|
|
Average |
33.8 |
39.4 |
|
|
|
|
Average change from baseline (SD) |
-8.9 (13.3) |
-4.2 (12.1) |
0.003 |
(-8.9, -1.8) |
|
|
LOCF average change from baseline (SD) |
-8.4 (12.8) |
-4.6 (12.2) |
0.021 |
(-7.1, -0.6) |
|
|
Response (% of Subjects)c |
19.8 (26/131) |
11.1 (12/108) |
0.076a |
|
1.8 (0.94, 3.4) |
|
LOCF Response (% of Subjects) |
17.7 (26/147) |
11.7 (14/120) |
0.227 |
|
1.5 (0.83, 2.8) |
|
Complete Response (% Subjects)d |
13.0 (17/131) |
2.8 (3/108) |
0.0045 |
|
4.7 (1.4, 15.5) |
|
LOCF Complete Response (% Subjects) |
11.6 (17/147) |
2.5 (3/120) |
0.0048 |
|
4.6 (1.4,15.4) |
|
|
|
|
|
|
|
a By Fisher’s two-sided exact test
b. Risk Ratio (RR) = [P(Response) for D-02]/[P(Response) for D-04]
Example: for Response, the estimated RR = (26/131)/ (12/108) = 1.78
c Response: ³ 50% decreasing change from baseline
d. Complete Response: IDS-SR £ 14
Primary effectiveness endpoint [Difference in two slopes, D2 – D4 and 95% CI: -0.32 per month, 95% CI: (-0.52, -0.12), p = 0.002 to reject true null hypothesis (difference = 0)] (see April 2, 2004 Amendment # 6)
FDA’s Revised Table 8-B
HRSD-24 Scores-D-02/D-04 Comparisons (Overlapping sites for both D-02 and D-04 patients only), Evaluable Patient Population
|
|
D-02 |
D-04 |
P-value |
95% CI (Least square mean) |
RR (95% CI)b |
|
N at baseline |
147 |
120 |
|
|
|
|
Baseline Average |
27.4 |
27.7 |
|
|
|
|
12 Month Data |
N = 130 |
N = 100 |
|
|
|
|
Average |
19.7 |
23.0 |
|
|
|
|
Average change from baseline (SD) |
-7.7 (8.8) |
-4.7 (7.6) |
0.020 |
(-5.1, -0.4) |
|
|
LOCF Average change from baseline (SD) |
-6.9 (8.9) |
-4.7 (7.6) |
0.113 |
(-4.0, 0.4) |
|
|
Response (% of Subjects)c |
27.7 (36/130) |
11.0 (11/100) |
0.0018a |
|
2.5 (1.3, 4.7) |
|
LOCF Response (% of Subjects) |
25.2 (37/147) |
11.0 (11/100) |
0.0055 |
|
2.3 (1.2, 4.2) |
|
Complete Response (% Subjects)d |
16.9 (22/130) |
5.0 (5/100) |
0.0065 |
|
3.4 (1.3, 8.6) |
|
LOCF Complete Response (% Subjects) |
15.6 (23/147) |
5.0 (5/100) |
0.013 |
|
3.1 (1.2, 7.9) |
|
|
|
|
|
|
|
a. By Fisher’s two-sided exact test
b. Risk Ratio (RR) = [P(Response) for D-02]/[P(Response) for D-04]
c. Response: ³ 50% decreasing change from baseline
d. Complete Response: HRSD24 £ 9
FDA’s Revised Table 8-C
IDS-SR Scores After D-02 Censoring Only-D-02/D-04 Comparisons (Overlapping sites for both D-02 and D-04 patients only), Evaluable
Patient Population
|
|
D-02 |
D-04 |
P-value |
95% CI (Least square mean) |
RR (95% CI)b |
|
N (at Baseline) |
147 |
120 |
|
|
|
|
Baseline Average |
42.7 |
43.6 |
|
|
|
|
12 Month Data |
N = 131 |
N = 108 |
|
|
|
|
Average |
36.0 |
39.4 |
|
|
|
|
Average change from baseline (SD) |
-6.7 (13.3) |
-4.2 (12.1) |
0.026 |
(-7.5,-0.5) |
|
|
LOCF Average change from baseline (SD) |
-6.1 (12.8) |
-4.6 (12.2) |
0.160 |
(-5.5, 0.9) |
|
|
Response (% of Subjects)c |
16.8 (22/131) |
11.1 (12/108) |
0.26a |
|
1.5 (0.78, 2.9) |
|
LOCF Response (% of Subjects) |
15.0 (22/147) |
11.7 (14/120) |
0.47 |
|
1.3 (0.7, 2.4) |
|
Complete Response (% Subjects)d |
8.4 (11/131) |
2.8 (3/108) |
0.095 |
|
3 (0.86, 10.6) |
|
LOCF Complete Response (% Subjects) |
7.5 (11/147) |
2.5 (3/120) |
0.097 |
|
3 (0.85, 10.5) |
|
|
|
|
|
|
|
a. By Fisher’s two-sided exact test
b. Risk Ratio (RR) = [P(Response) for D-02]/[P(Response) for D-04]
c. Response: ³ 50% decreasing change from baseline
d. Complete Response: IDS-SR £ 14
Primary effectiveness endpoint [Difference in two slopes, D2 – D4 and 95% CI: -0.18 per month, 95% CI: (-0.38, 0.02), p = 0.079 to reject true null hypothesis (difference = 0)] (see April 2, 2004 Amendment # 6)
RMLR
predicted mean IDS-SR (Table 6.2.37, Amendment # 6)
Quarter D-2 (SE) D-4 (SE) D2-D4 95% CI*
1 37.93 (0.59) 38.47 (0.60) -0.54 (-1.70, 0.64)
2 37.01 (0.65) 38.09 (0.69) -1.08 (-2.38, 0.24)
3 36.35 (0.71) 37.96 (0.81) -1.61 (-3.10, -0.12)
4 36.58 (0.77) 38.72 (0.98) -2.14 (-3.84, -0.54)
(*The average of two standard errors was used as pooled SE for 95% CI, N unknown)
FDA’s Revised Table 8-D
HRSD-24 Scores-After D-02 Censoring Only -D-02/D-04 Comparisons (Overlapping sites for both D-02 and D-04 patients only), Evaluable Patient Population
|
|
D-02 |
D-04 |
P-value |
95% CI (Least square mean) |
RR (95% CI)b |
|
N (at 12-Month) |
147 |
120 |
|
|
|
|
Baseline Average |
27.4 |
27.7 |
|
|
|
|
12 Month Data |
N = 130 |
N = 100 |
|
|
|
|
Average |
22.5 |
23.0 |
|
|
|
|
Average change from baseline (SD) |
-4.9 (9.1) |
-4.7 (7.6) |
0.581 |
(-3.9, 1.7) |
|
|
LOCF Average change from baseline (SD) |
-4.3 (8.9) |
-4.7 (7.6) |
0.910 |
(-2.1, 2.4) |
|
|
Response (% of Subjects)c |
18.5 (24/130) |
11.0 (11/100) |
0.14a |
|
1.7(0.86,3.3) |
|
LOCF Response (% of Subjects) |
16.3 (24/147) |
11.0 (11/100) |
0.27 |
|
1.5(0.76,2.3) |
|
Complete Response (% Subjects)d |
7.7 (10/130) |
5.0 (5/100) |
0.59 |
|
1.5(0.54,4.3) |
|
LOCF Complete Response (% Subjects) |
6.8 (10/147) |
5.0 (5/100) |
0.79 |
|
1.3(0.48,3.8) |
|
|
|
|
|
|
|
a. By Fisher’s two-sided exact test
b. Risk Ratio (RR) = [P(Response) for D-02]/[P(Response) for D-04]
c . Response: ³ 50% decreasing change from baseline
d. Complete Response: HRSD24 £ 9
In Tables 8-A through 8-D, the 95% confidence interval (CI) for the difference
(D2 – D4) of two average changes from baseline, rather than the calculated p- values, would allow better clinical evaluation, because p-values are simply used to reject the true null hypothesis that two average changes = 0 against the alternative hypothesis that two average changes ¹ 0.
In Tables
8-C and 8-D for censored patients (overlapping sites), as described previously, that “IDS- SR or HRSD-24 raw scores were censored
such that the value from the patient’s IDS-SR
or HRSD-24 measurement obtained prior to their
first increase in the antidepressant resistance rating score was carried forward and replaced all of the patient’s
subsequent, non-missing, IDS-SR measurements.
In Tables 8-C and 8-D (Censored patients from overlapping sites, see Appendix 1), most of statistical results failed to support that D-2 patients showed superior IDS-SR or HRSD-24 results to those for D-4 patients, except for average change from baseline comparison at 12 months for IDS-SR only.
Concordance between IDS-SR and HRSD-24 Scores
Due to wide variability of paired IDS-SR/HRSD-24 scores from patient to patient, FDA requested the sponsor to calculate the estimated correlation coefficient and its 95% confidence interval (CI), the estimated regression intercept and slope and their 95% CI, the unadjusted (for degrees of freedom) R-Square (R2), which measures the “proportion of total variation about the mean HRSD-24 explained by the fitted regression equation”, from each individual patient. The R2 evaluates how well IDS-SR, predicts HRSD-24 score. R2 ranged from 0% (worst prediction fit) to 100% (perfect prediction fit). In Figure 3, the histogram of R2 shows relatively poor to fair prediction with mean R2 of 0.55, ranging from 0 to1, for 235 evaluable patients from the D-2 study. In Table 9, the average simple Pearson correlation coefficient of 0.7 with 95% CI (0.67, 0.73) between IDS-SR and HRSD-24 scores, again indicates that IDS-SR is not a “good predictor” of HRSD-24.
Table 9. Summary of Correlation Coefficient and Regression Slope, for 235 Evaluable Patients, Paired IDS-SR/HRSD-24 Scores, All Sites
|
Parameter |
N |
Mean |
SD |
Median |
Min |
Max |
Lower 95% CL |
Upper 95% CL |
|
Pearson Correlation Coefficient |
235 |
0.7 |
0.25 |
0.77 |
-0.26 |
1.0 |
0.67 |
0.73 |
|
Slope |
235 |
0.55 |
0.25 |
0.56 |
-0.76 |
1.49 |
0.51 |
0.58 |
VI. Conclusion
The IDS-SR does not to be a “good predictor” of the HRSD-24 based on the distribution of R2 values (Figure 3) and sample correlation coefficients (Table 9) of 235 D-2 evaluable patients.
Due to small sample size (Table 3) for bipolar patients, no valid statistical analyses can be prepared, but a clinical decision is needed.
The required sample size analyzed in this PMA was based on neither the comparison of two true slopes (Primary effectiveness endpoint) nor mean responses in this repeated-measures/longitudinal data analyses. No minimum clinically detectable difference in two slopes or mean HRSD-24 or IDS-SR was defined at the study design stage in order to estimate the required sample size with pre-specified power, type I error, estimated variability of the data, number of follow- up visits, and correlation among repeated measures.
A clinical decision is also required to decide several important issues, such as pooled sites, potentially important hidden covariates in the PS analysis, non- overlapping sites or overlapping sites, and censored or non-censored analyses. The validity of statistical inferences from comparison of two proportions of Responses pooled over all non-overlapping and overlapping sites, without any appropriate statistical modeling approach, such as meta-analysis, is highly questionable. Clinically important patient covariates, such as patient baseline IDS-SR or HRSD-24 measurements, clinical site, and others, must be considered in the comparison of two proportions via statistical modeling approach or stratified categorical data analysis.
For censored and overlapping sites, no statistically significant differences in primary effectiveness endpoint (Difference in two slopes, IDS-SR, Table 8-C) and secondary effectiveness endpoint (Difference in two proportions of responses or difference in average change from baseline, HRSD-24, Table 8-D) were found.
Due to above statistical issues, such as questionable concordance between HRSD- 24 and IDS-SR, questionable pooling of multi-center data for comparison of proportions of responses, statistically insignificant findings from censored and overlapping sites (Tables 8-C and 8-D) for IDS-SR primary effectiveness endpoint (Slope) and HRSD-24 secondary effectiveness endpoint (Response proportions), it is unclear whether the effectiveness claim of D-02 over D-04 group patients has been demonstrated.
Appendix 1
Table 10. Distribution of Number of Patients by Clinical Site
(10 Non-overlapping and 12 Overlapping Sites), D-02 and D-04 Study
|
Site |
Long- Term, D-02 |
Evaluable D-02 |
12-Month D-02 |
Evaluable D-04 |
12-Month D-04 |
|
040 |
16 |
15 |
15 |
6 |
5 |
|
041 |
10 |
9 |
7 |
3 |
1 |
|
042 |
10 |
9 |
7 |
- |
- |
|
043 |
13 |
12 |
10 |
8 |
7 |
|
044 |
17 |
13 |
10 |
12 |
12 |
|
045 |
18 |
18 |
15 |
13 |
10 |
|
046 |
13 |
11 |
10 |
2 |
2 |
|
047 |
9 |
7 |
7 |
- |
- |
|
048 |
7 |
7 |
7 |
- |
- |
|
049 |
12 |
10 |
9 |
11 |
11 |
|
050 |
10 |
9 |
8 |
8 |
6 |
|
051 |
9 |
7 |
3 |
- |
- |
|
052 |
9 |
8 |
7 |
- |
- |
|
053 |
4 |
3 |
2 |
- |
- |
|
054 |
13 |
12 |
12 |
15 |
14 |
|
055 |
8 |
5 |
4 |
- |
- |
|
056 |
9 |
9 |
9 |
- |
- |
|
057 |
6 |
5 |
5 |
16 |
14 |
|
058 |
19 |
17 |
14 |
14 |
14 |
|
059 |
18 |
16 |
13 |
12 |
12 |
|
060 |
3 |
3 |
3 |
- |
- |
|
071 |
- |
- |
- |
4 |
4 |
Total (All) 233 205 177 124 112
Total 165 147
128 120 108
(Overlapping)