M E M O R A N D U M

 

DEPARTMENT OF HEALTH AND HUMAN SERVICES PUBLIC HEALTH SERVICE

FOOD AND DRUG ADMINISTRATION

CENTER FOR DRUG EVALUATION AND RESEARCH

                                                                             

DATE:                        February 2, 2004

 

TO:                              Paul Seligman, M.D., M.P.H., Acting Director

                                    Anne Trontell, M.D., M.P.H., Deputy Director

Office of Drug Safety

Immediate Office, HFD-400

 

THROUGH:                Mark Avigan, M.D., C.M., Director

                                    Division of Drug Risk Evaluation, HFD-430

                                    Gerald DalPan, M.D., M.H.S., Director

                                    Division of Surveillance, Research, and Communication Support, HFD-410

                                   

FROM:                        Allen Brinker, M.D., M.S., Epidemiologist Team Leader

                                    Parivash Nourjah, Ph.D., Epidemiologist

                                    Karen Lechter, Ph.D., Survey Analyst

                                    Division of Drug Risk Evaluation, HFD-430

                                    Division of Surveillance, Research, and Communication Support, HFD-410

                                    Office of Drug Safety

 

SUBJECT:                  PID D030417

Drug: Isotretinoin

Topic: Review of Isotretinoin Patient Survey Materials and Data with Interest in Compliance During the First Year of the System to Manage Accutane Related Teratology (SMART) Program.

 

 

EXECUTIVE SUMMARY

 

The analyses presented herein are based upon data supplied by the primary drug sponsor, Hoffmann-La Roche[1], and subcontractors Slone Epidemiology Unit, Boston University Medical School [Slone] and The Degge Group with S.I. International [Degge/SI].  Other data resources include the IMS HEALTH National Prescription Audit (NPA) and National Disease and Therapeutic Index (NDTI).

 

The main findings of the report based on the Divison of Drug Risk Evaluation (DDRE) analyses are as follows:

 

·       Absolute participation in the Accutane Patient Surveys [Survey(s)] increased from 16% - 19% in the year before SMART to 22%-26% in the first year of SMART.

 

·       In comparison to two observational cohorts of Accutane-brand isotretinoin and/or generic isotretinoin recipients, the youngest users of isotretinoin appear to under-represented among participants in the Degge/SI cohort and this difference appears clinically significant.  Further comparison to these observational data also suggests patients receiving isotretinoin from non-dermatologists appear to be underrepresented in the Survey. In general, results from the patient survey should be considered with caution due to possible error from a low enrollment rate, recall bias, social desirability bias, bias due to both unit non-response and item non-response, and poor questionnaire design.

 

·       In the Degge/SI cohort, 76% of Survey participants reported they signed two consent forms, 4% reported they signed only one form, 9% reported that they did not sign any consent forms, and 11% were uncertain about whether they signed a consent form or not.

 

·       In the Degge/SI cohort, 92% of Survey participants reported that they received a prescription with yellow Acutane Qualification sticker. The remaining women reported there was no sticker, did not answer the question, or reported someone else handled the prescription for them.  

 

·       Data reported from the Slone Epidemiology Group suggests that there has been some improvement in the rate of ANY pregnancy testing prior to initiating therapy with isotretinoin, from 77%-85% to 91% in the two year interval 2QTR2001 through 1QTR2003.  However, the improvement appears to have plateaued shortly after new labeling requirements (April 1, 2002).

 

·       In the Degge/SI cohort, approximately 68% of apparently fertile and sexually active women reported two pregnancy tests prior to initiation of isotretinoin, a labeled requirement for isotretinoin therapy.  Approximately 7% of apparently fertile and sexually active women reported no pregnancy testing prior to initiation of isotretinoin.  Of menstruating, apparently fertile sexually active women, 28% had a pregnancy test within the first 5 days of their menstrual period immediately before starting isotretinoin.

 

·       In Quarterly reports to the sponsor and FDA, the Slone Epidemiology Group has reported a low (<=1%) prevalence of sexual activity without birth control based on a denominator of ALL Survey respondents.  After restriction to a denominator of apparently fertile, 15-45 yr-old sexually active participants, this rate becomes ~3%.

 

·       In the Degge/SI cohort, 4.2% of apparently fertile and sexually active 15-45 year-old participants reported no form of birth control.  In total, 46.4% of apparently fertile and sexually active 15-45 year-old participants reported use of “appropriate” birth control consisting of two methods, at least one of which is a “primary” birth control method (e.g., oral contraception).

 

·       Regarding the presence of a prescription Qualification sticker at initiation of isotretinoin treatment, pregnancy testing was generally high both in the presence of a sticker (91%) and in the absence of a sticker (91%) for apparently fertile, 15 to 45 year old survey participants. No difference was seen after imputation for records with missing data or upon restriction to sexually active Survey participants. Overall 9% of Survey participants who reported a qualification sticker was present also indicated a pregnancy test was not done.

 

·       The presence of a prescription Qualification sticker did not correlate with use of birth control at initiation of isotretinoin treatment.  Among sexually active, apparently fertile 15 to 45 year old Survey participants, any birth control was noted in 97% of enrollees with a sticker and 96% of enrollees without a sticker. No difference was seen after imputation for records with missing data.

 

·       Review of data reported from the Slone Epidemiology Group suggests that there has been some improvement in the rate of ANY pregnancy testing during isotretinoin therapy, from ~70% to ~85% in the two year interval 2QTR2001 through 1QTR2003.  However, the increase appears to have plateaued soon after initiation of the SMART program and remains at ~85%.  Based on DDRE analyses of the Degge/SI DAT2 cohort, 81% of apparently fertile, 15 to 44 years old Survey participants report monthly pregnancy testing in two consecutive months during treatment with isotretinoin.

 

·       The presence of a prescription Qualification sticker did not appear to correlate with performance of the pregnancy test during isotretinoin therapy.  Pregnancy testing was generally high both in the presence (99%) and in the absence (98%) of a sticker for apparently fertile, 15 to 45 year old survey participants. Similarly, the presence of a prescription qualification sticker did not appear to effect compliance with birth control during isotretinoin therapy.  Among sexually active, apparently fertile 15 to 45 year old survey participants, any birth control was noted in 99% of enrollees with a sticker and 100% of enrollees without a sticker. No difference was seen after imputation for records with missing data.

 

·       DDRE has reviewed additional information to the sponsor’s “1 Year Report” and Degge/SI dataset in the form of Quarterly reports for 2QTR2003 and 3Qtr2003 from Degge/SI (on Accutane-brand isotretinoin) and Slone Epidemiology Group (for both Accutane-brand isotretinoin and generic isotretinoin). Due to differences in survey, survey methods, and operating procedures, it is problematic to compare across these disparate surveys.  However, a review of selected and generally comparable variables (e.g., pregnancy testing, birth control practices) suggests similar compliance with these selected attributes as for April 1, 2002 through March 30, 2003 as described immediately above.  Independent FDA validation of results in quarterly reports is not possible.

 

·       The Degge/SI dataset contains 15 reports of pregnancy among 4277 women on their first-course of isotretinoin therapy.  The observed pregnancy rate for first-course users within the Degge/SI cohort is thus 15/4277 = 3.5 / 1000. Since this rate is censored, it likely represents an underestimate of the rate realized when all these Survey participants complete follow-up. However, this rate is virtually identical to the rate as reported by researchers from the Slone Epidemiology Group for their Accutane Survey cohort of 2.9 per 1,000 women.

 

INTRODUCTION

 

Goals/Objectives of SMART

 

The System to Manage Accutane Related Teratogenicity (SMART) program was initiated on April 10, 2002 to address two main goals raised at a public meeting in September 2000.[2]

1) No woman should begin Accutane therapy if she is pregnant;

2) No pregnancies should occur among women taking Accutane.

 

Tools of the SMART program; Required Steps in Process and Patient labeling[3]

 

The S.M.A.R.T. program requires the following:

·       Prescribers read the S.M.A.R.T. "Guide to Best Practices" provided by Roche, and then sign and return the Letter of Understanding certifying their knowledge of the measures to minimize fetal exposures to Accutane. In addition, Accutane prescribers are strongly encouraged to participate in a half-day Continuing Medical Education (CME) course developed by the manufacturer that includes specific, practical information about pregnancy prevention.

·       Prescribers receive self-adhesive Accutane Qualification Stickers which are to be attached to the prescription form. These stickers indicate to the pharmacist that the patient is "qualified" to receive Accutane, i.e., the female patient has had negative pregnancy tests, as well as education and counseling about pregnancy prevention. The pregnancy test is to be repeated every month throughout the Accutane treatment course, and no prescriptions should be given for more than a one month supply of Accutane.

·       All female patients must have two negative urine or serum pregnancy tests before the initial Accutane prescription is written.  Additionally, patients must have a negative pregnancy test prior to receiving subsequent prescriptions, regardless of whether or not they are sexually active. Patients who are, or might become, sexually active with a male partner must also select and use two forms of effective contraception simultaneously for at least one month prior to initiation of Accutane therapy, during therapy, and for one month following discontinuation of therapy.  Patients are required to sign a Patient Information/Consent form about Accutane and birth defects, in addition to the General Consent Form that all patients receive about other potentially serious risks (e.g., suicidality, etc.). Finally, female patients must be given the opportunity to enroll in the Accutane Survey. This confidential survey collects data on the utility of S.M.A.R.T. in reducing pregnancies among female Accutane users.  This survey will help to identify aspects of S.M.A.R.T. that could be improved upon.

·       Pharmacists dispense Accutane only upon presentation of a prescription with the special Accutane Qualification Sticker.  Moreover, pharmacists dispense a maximum one-month supply of Accutane, fill prescriptions within seven days from the date of "qualification," (defined above) and provide a Medication Guide for patients with each Accutane prescription. Requests for refills (i.e. more Accutane without a new prescription) and phoned-in prescriptions are not to be filled.

 

METHODS

 

Background on Patient Survey

 

Accutane (isotretinoin, Hoffmann-La Roche) was initially marketed in the US in 1982.  Isotretinoin is an effective medication for the treatment of severe recalicitrant acne, although it has potent teratogenic effects.  To minimize the risk of patients becoming pregnant while taking isotretinoin and avoiding exposure during pregnancy, the sponsor designed and implemented the Accutane Pregnancy Prevention Program (PPP) in 1988.  In 1989, the Slone Epidemiology Unit of Boston University designed and conducted the Survey of Accutane Use in Women (Slone Survey / Accutane [Patient] Survey) to evaluate the PPP program.  The general format[4] of the survey remains largely unchanged.   In the original survey, women who chose to enroll in the program were randomized (at initiation of therapy) to either a During and After Treatment [DAT] arm (consisting of 2 or more questionnaires, DAT-1, DAT-2, and DAT-3) or an After Treatment [AT] arm.  The DAT arm was restricted to a total enrollment of 5,000 women per year.  Between 1989 and 1994, women in the DAT arm were queried by phone while women assigned to the AT arm were queried by mail.  Women who enrolled but failed to respond to mailed questionnaires were contacted by air courier or, if needed, telephone.  Since 1995, mailed questionnaires have been used as the primary contact instrument for both the DAT and AT arms with follow-up contact by air courier or telephone for non-responders.  The Slone Epidemiology Unit was the sole administrator of the Accutane patient survey from inception through September 2002.  During the fourth quarter of 2002, Hoffmann-La Roche re-awarded the contract for the survey of enrollees using Accutane-brand isotretinoin to Degge/SI.  Sponsors of generic isotretinoin introduced into the market during the fourth quarter of 2002, elected to use the Slone Epidemiology Group to conduct the survey, a regulatory requirement, on their behalf.  The Slone Epidemiology Unit has continued to survey women who enrolled before the SMART program, those who enroll with cards addressed to Slone Epidemiology Unit, and those who declined transfer to Degge/SI.  The DAT questionnaire was also modified to include new questions specifically designed to measure compliance with SMART.[5]  The new questionnaire was introduced in 4QTR2002 by Degge/SI and in 2QTR2003 by Slone for generic isotretinoin.  Chart 2 provides a timeline for Accutane/generic isotretinoin patient surveys[6].

 

Data Sources

 

The analyses presented herein are based upon data in a report supplied by the primary drug sponsor, Hoffmann-La Roche[7] and primary dataset provided by subcontractors Slone Epidemiology Unit, Boston University Medical School [Slone] and The Degge Group with S.I. International (Degge/SI).  Other data resources include the IMS HEALTH National Prescription Audit (NPA) and National Disease and Therapeutic Index (NDTI).

RESULTS

 

1.         SURVEY PARTICIPATION, REPRESENTATIVENESS, AND LIMITATIONS

 

1.1       Absolute Participation in Patient Surveys

 

Absolute participation in the Patient Survey by quarter, as provided to FDA by Hoffmann-La Roche is shown in table 1.  These data appear to show that, although participation has generally been poor, participation is increasing. 

 

Table 1.  Trends in participation in Accutane Patient Survey per Sponsor

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

Survey Enrollees

7,903

7,897

8,211

10,044

8,533

7,560

10,249

10,289

New female starts

49,795

47,402

54,348

31,262

42,642

34,424

38,528

19,761

Enrollment Rate

17.3%

16.8%

15.1%

32.1%

20.0%

22.0%

26.6%

52.1%

 

To this end, the sponsor states,

 

“Accutane Survey enrollment rate of 28.2% [for the first year of SMART], based on a denominator of new female patient starts, represents an approximate 10 percentage point increase in Survey enrollment rate from the 4 quarters prior to SMART and reverses a downward trend in Survey enrollment rates observed in a retrospective review.”

 

In order to provide for an independent assessment of quarterly enrollment since initiation of the SMART program (April 1, 2001) though March 31, 2002, DDRE produced an independent estimate of unique females on Accutane therapy for the 8 quarters of interest.  These data, along with quarterly Survey enrollees (numerators) shown in Table 1, permit calculation of enrollment rates.  The approach used to calculate quarterly estimates of unique females on Accutane is based on the following assumptions: 1) there is a steady-state condition of drug usage for well-established drugs such as Accutane; and 2) the average total number of Rxs per unique individual until the end of therapy is 3.7.[8] This estimate of average number of Rxs per unique patient is provided in a large, observational study of Accutane recipients followed to the apparent end of an Accutane treatment course.  The assumptions listed previously, and lack of proven applicability of this metric to the population of Accutane recipients are limitations of this analytic approach.  Steps taken to calculate enrollment rates were as follows: First, retail female prescription volume for the 8 quarters of interest was obtained by dividing the observed quarterly prescription volume by two and then summed.  [Sum Rx volume to females over the eight quarters of interest = 1,334,000 Rx.]  Second, the total number of unique females on Accutane for the 8 quarters of interest was estimated by dividing the total female Rx volume by 3.7.  This is outlined in the following equation:

Unique females (total for 8 quarters) = (1,334,000 Rx) (unique patient / 3.7 Rx[9])

= 360,500

 

Finally, total female utilization (360,500) over the 8 quarters was multiplied by the fraction of total utilization for any specific quarter to result in an estimate of unique females for each quarter.

 

Table 2. Trends in participation in Accutane Patient Survey per DDRE based on estimate of unique females on therapy.

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

Survey Enrollees

7,903

7,897

8,211

10,044

8,533

7,560

10,249

10,289

Rx volume for quartera

377,000

338,000

394,000

399,000

295,000

248,000

292,000

325,000

Rx volume to femaleb

188,500

169,000

197,000

199,500

147,500

124,000

146,000

162,500

% of total female utilizationc

14.1%

12.7%

14.8%

15.0%

11.0%

9.3%

10.9%

12.2%

Estimate of total unique femalesd

50,900

45,700

53,200

53,900

39,900

33,500

39,500

43,900

Enrollment Rate

15.5%

17.3%

15.4%

18.6%

21.4%

22.6%

26.0%

23.4%

a  US Data Source: IMS Health, IMS National Prescription Audit PlusTM, Accutane and generic isotretinoin for 3/01 to 6/03; accessed 7/28/03.

b  Based on assessment of 1:1 male: female Rx distribution; FDA custom calculations were based on IMS Health data.

c  Based on division by female Rx volume of quarter by total Rx volume to females (1,334,000); FDA custom calculations were based on IMS Health data.

d  Based of multiplication by percent of total female utilization of quarter by 360,500; unique female count of 360,500 based on an average of 3.7 Rxs per patient (see text); FDA custom calculations were based on IMS Health data.

 

The analyses provided by the sponsor and by DDRE are highly similar in all eight quarters before and after initiation of SMART except for 1QTR 2002 and 1QTR 2003.  In these quarters, the DDRE analysis of enrollment is approximately half of the enrollment estimate by the sponsor.  Since the two analyses use the same numerator data, the difference for the two quarters of interest must be due to differences in the denominators for 1QTR 2002 and 1QTR 2003.  This difference is captured in Tables 1 and 2.  In these tables, the respective denominators for 1QTR 2002 and 1QTR 2003 are approximately half of each other. In order to investigate trends due to seasonality, average duration of treatment in relation to season were explored within the AdvancePCS claims database.  No differences were observed based on standard calendar quarters (data not shown) for 2002.  Thus, the reason the two models result in almost identical counts for six quarters yet diverge strikingly for 1Qtr02 and 1Qtr03 is not readily apparent.  Based on DDRE analyses, absolute participation in the patient surveys increased from 16% - 19% in the year before SMART to 22%-26% in the first year after implementation of SMART.

 

1.2       Representativeness

 

The sponsor includes the following summary with regard to the representativeness of Accutane Survey participants:

 

“The representativeness of Accutane Survey respondents to all Accutane female patients in the United States was demonstrated for age, prescriber, payer and geographic region of the country. While this does not ensure absolute comparability of all Survey respondents to all Survey non-respondents, it suggests that the convenience sample is reasonably representative of the population on these selected parameters.”

 

DDRE would agree with the conclusion of the sponsor that the Survey represents a “convenience sample.”  With reference to the sponsor’s data (not shown) for clinical characteristics of Slone and Degge/SI cohorts versus data on the isotretinoin-user population at large, we would note the following:

 

  • Age distribution - differences are mild but noticeable

·       Young (< 15 years) and 45+ age groups are underrepresented in the Accutane Survey populations when compared to the female Accutane population/reference population[10] (For < 15 year olds: SEC 13.2%, Degge/SI 9.6%, and reference population 17.3%; For 45+ age group SEC 3.6%, Degge/SI 5%, and reference population 10.9%). 

·       The 20-29 year age-group is over-represented (SEC 35.7%, Degge/SI 37.2%, and reference population 24.9%). 

  • Geographic Region - Generally similar, east somewhat under-represented in the survey population when compared to reference population[11] (SEC 15.6%, Degge/SI 16.5%, and reference population 21.2%)
  • Payor Type - Based solely on Degge/SI cohort; unable to compare given the large fraction (12%) of unknown responses.
  • Prescriber - Data suggest that up to 20% of Accutane prescriptions are generated by non-dermatologists.  However, non-dermatologists account for only 5% of prescribers in the Slone cohort and 10% of the Degge/SI cohort.[12]  This is also noted by the sponsor.

 

The Degge/SI primary dataset as provided to FDA by the sponsor does not include data on geographic region nor payer type.  DDRE is unable to generate tables comparable to the sponsor’s.  Data on age and prescriber was provided by the sponsor in their dataset. DDRE analyses of these data are outlined as follows. The age distribution of the Degge/SI cohort is compared to the distribution of recipients of isotretinoin (both Accutane-brand and generic) within the IMS Health NDTITM and with recipients of Accutane-brand isotretinoin with coverage managed by AdvancePCS (described presented previously within the Isotretinoin Utilization section of this document[13].) In brief, the IMS Health National Disease and Therapeutic Index™ (NDTI™) is an ongoing survey designed to provide descriptive information on the patterns and treatment of disease encountered in office-based practice in the continental United States by collecting data on drug products mentioned during visits to roughly 2,000 - 3,000 office-based physicians.  The data are projected nationally to reflect national prescribing patterns.  These data are displayed in Table 3 below.  In comparison to the IMS Health NDTI, it appears the youngest isotretinoin users are underrepresented in the Survey.  While the NDTI is designed to be representative, it is limited by size.  Thus, the age distribution of Degge/SI participants was also compared to isotretinoin recipients with drug benefits managed by AdvancePCS, which is not designed to be representative but is quite large.  These data also suggest that the youngest isotretinoin users appear to be under-represented among Survey participants.  Furthermore, while it is inappropriate to subject these data to statistical testing, these comparisons provide evidence that there is a large and clinically meaningful difference in Survey participation for the youngest recipients of isotretinoin.

 

Table 3. Age distribution of the Degge/SI cohort compared to female Accutane recipients with drug benefits managed by AdvancePCS drug benefits and female recipients of isotretinoin (all brands) within the IMS Health NDTI.

Age group (years)

Degge/SI cohort*

(as submitted)

AdvancePCS**

(4/02-3/03)

IMS Health NDTI***

(4/02-3/03)

Up to age 19

1917 (35%)

43%

45%

20-29

2092 (38%)

28%

30%

30-39

918 (17%)

16%

16%

40+

536 (10%)

13%

9%

Total

5463

100%

100%

*excludes 6 records where age was missing  + 432 records on generic isotretinoin

**Advance PCS™ Dimension Rx, accessed 11 December 2003

***IMS Health, IMS National Disease and Therapeutic IndexTM, 4/2002-3/2003, accessed 15 December 2003; FDA custom calculations were based on IMS Health data

 

Second, in DDRE analyses of the Degge/SI dataset, 94% of DAT-2 participants indicated their Accutane prescriber was a dermatologist.  In comparison, approximately 80% of recent isotretinoin prescriptions in the IMS Health NPATM were associated with dermatologists.[14]  Thus, as shown in analyses by DDRE and the sponsor, patients receiving care from non-dermatologists are underrepresented in the Survey.  It is not known if there is differential encouragement for the patient survey between dermatologists and non-dermatologists or if compliance with isotretinoin labeling and pregnancy varies between dermatologists and non-dermatologists.

 

1.3       Overview of Study Limitations due to Internal Variability

 

In consideration of the generalizability of the Survey, the sponsor notes,

 

“Examination of invalid and absent responses to the Accutane Survey indicated relatively high proportions of invalid and absent responses to several key questions that measure S.M.A.R.T. compliance.”

 

The Slone Survey, implemented in 1989, was designed to, “assess the compliance of physicians and patients with the Accutane Pregnancy Prevention Program and to identify the rate of pregnancy during treatment with isotretinoin and during the month after treatment.” Appendix 6[15] provides a comprehensive discussion on both the internal variability and external generalizability of the Survey. DDRE/DSRCS analyses also indicated a high percentage of missing responses on key variables. For instance, patient survey information on contraception may be incomplete or unreliable due to the complexity of the questionnaires.  In addition, some questions may be skipped inappropriately due to misinterpretation of survey instructions.  While the sponsor does not address what inference should be gleaned from “invalid and absent responses,” we take the position that these issues introduce biases, degrade interval validity, and preclude statistical testing of Survey results. Furthermore, it is our position that any changes between calendar quarters represents too short a time interval to detect meaningful changes; larger changes over time may be considered more robust, but not definitive evidence of change.

 

2.         DAT-1 SURVEY (INITIATION OF TREATMENT)

 

2.1       DAT-1 Univariate Analyses

 

2.1.1    Timing of Enrollment relative to the start of Accutane therapy

 

Timing of enrollment relative to the start of therapy with isotretinoin is important to assure accuracy of responses. The sponsor reports 73% of enrollees enroll within 1 month of starting treatment with isotretinoin.  Approximately 27% of participants enroll at a time point of > 1 month.

 

Based on a dataset provided by the sponsor, 5489 women responded to a DAT-1 questionnaire sent by Degge/SI between October 2002 and April 1, 2003.  Twenty (20) of the respondents reported no plans to start Accutane therapy and have been excluded from all further DDRE analyses.  Of the remaining 5469 respondents, 5334 women started treatment with Accutane, and 135 women reported they will start Accutane therapy later or did not provide an answer.  Of this baseline dataset of 5469 women, 80.9% completed the questionnaire within 30 days of enrollment.  Twelve percent (12%) responded later than 30 days, and 7.1% had an invalid or missing date.Thus, we concur with the sponsor in assessment that most enrollees in the Survey do so early in therapy. Among the baseline dataset of 5469 respondents, 1416 (26%) reported they had undergone a previous course of Accutane therapy.

 

2.1.2    Consent forms

 

The SMART program requires women to sign two consent forms, one required of all patients on isotretinoin and the other required only of female patients. The sponsor does not include an analysis of the Degge/SI cohort for consent.

 

Based of DDRE analyses of the Degge/SI cohort, 76.3% of women reported that they signed two consent forms, 3.7% reported they signed only one form, 9.3% reported they had not signed a consent form, and 10.7% were uncertain about whether they signed a consent form or not.

 

2.1.3    Medication Guide

 

A Medication Guide should be distributed with each prescription. The sponsor does not include an analysis of the Degge/SI cohort for receipt of a Medication Guide within the Degge/SI cohort in their “1 Year Report.”

 

Based of DDRE analyses of the Degge/SI cohort, 81% of respondents reported receipt of a Medication Guide.  Four (4) percent reported that a Medication Guide was missing and 15% reported they were unsure or did not answer the question.

 

2.1.4    Patient Recall of Qualification Sticker

 

The sponsor’s analysis of patient recall of a qualification sticker is consistent with the analysis of the Pharmacy Compliance Survey[16] .  Based on data from the 3rd and 4th quarters of SMART, the sponsor notes 97% of survey enrollees recalled the presence of a qualification sticker.

 

In DDRE analysis of the Degge/SI cohort, a yellow Accutane Qualification sticker on the prescription was reported by 92.1% of the 5469 participants. Another 2.3% reported the Accutane qualification sticker to be missing and 5.6% of respondents were unsure or did not answer the question.

 

2.1.5    Compliance with 7-day Limit of Qualified Prescriptions

 

According to the new label for isotretinoin, the qualification sticker signifies that a patient was “qualified” for therapy upon receipt of the prescription and the prescription is to be filled within 7 days.  The sponsor conducted no analyses within the Degge/SI Survey on distribution of receipt and fill dates for Survey participants.

 

In DDRE analyses of Degge/SI survey data for women who started Accutane and further reported the presence of a Qualification sticker (n=4403), 89.9% initiated treatment within 7 days of receiving their Accutane prescription. Among this same group of participants, 93.2% filled their prescriptions within 7 days of receipt.  These data are compared to an analysis of the Pharmacy Compliance Survey[17], where, after DDRE review, 94% to 99% of prescriptions with qualification stickers were filled within 7 days.

2.1.6    Pregnancy Testing

 

2.1.6.1 Any Pregnancy testing: The sponsor provided the trends in any pregnancy testing prior to starting isotretinoin therapy based on the DAT-1 survey data collected by SEC and Degge/SI. As shown in table 4 below, SEC data show that the percentage of women reporting ANY pregnancy testing during the initial four quarters of SMART appears constant at about 92%.  It follows that 7% to 8% of women reported NO pregnancy testing.

 

The data for the Degge/SI cohort are generally similar for the 4QTR02 and 1QTR03. The percentage of women reporting ANY pregnancy testing is reported as 88% and those reporting no pregnancy testing as 9%. The difference between the two analyses may be in the larger fraction of women classified as “unknown” in the Degge/SI survey (3%) versus Slone (<1%).

 

Table 4. Any Pregnancy Testing per Sponsor

 

Pre-SMART

(1/1/95 – 3/31/02)

1st qtr SMART

(2Qtr02)

2nd qtr SMART

(3Qtr02)

3rd qtr SMART

(4Qtr02)

4th qtr SMART

(1Qtr03)

Total under SMART

 

Slone

Any test

27753 (76%)

1189 (92)%

1280 (93%)

1096 (92%)

911 (92%)

4476 (92%)

No test

8361 (23%)

96 (7%)

101 (7%)

93 (8%)

82 (8%)

372 (8%)

Unknown

233 (1%)

4 (<1%)

1 (<1%)

2 (<1%)

3 (<1%)

10 (<1%)

Degge/SI*

Any test

-

-

-

1413 (86%)

2103 (89%)

3516 (88%)

No test

-

-

-

174 (11%)

186 (8%)

360 (9%)

Unknown

-

-

-

53 (3%)

65 (3%)

118 (3%)

*Degge/SI began enrolling patients in survey on 11/20/02 (4Qtr02)

 

The sponsor also conducted analyses to examine the relation between pregnancy testing and age (data not shown).  These data suggest that pregnancy testing is related to age to some degree, with slightly higher compliance among women aged 16-39 (86 to 92% receiving any pregnancy testing).  Poorer compliance with testing at the extremes of the age distribution was among relatively small numbers of survey participants at those extremes (78 to 86% for any pregnancy testing).

 

While the sponsor provides a reference for ANY pregnancy testing (as shown in Table 4 above) based on the interval 1995-March 2002, this large time interval averages results during a time of public heightened attention and regulatory attention to reducing isotretinoin-exposed pregnancies.  Therefore, trends in pregnancy testing before start of Accutane therapy based on DAT-1 surveys have been compiled by DDRE for the four quarters prior to April 1, 2002 and the following quarters of SMART based on Slone Survey quarterly reports (Table 5). 

 

 

 

 

 

 

Table 5. Trends in Pregnancy Testing before start of Accutane therapy based on DAT-1 surveys (aggregated from Slone Survey quarterly reports) for the four quarters before SMART and the first four quarters of SMART.

Era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

N

1101

1050

1102

1141

1116

1178

1190

966

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

Any test

80

77

81

85

92

92

92

91

No Test

19

21

18

15

8

8

8

8

Unknown

1

1

1

<1

<1

<1

<1

1

 

As in the analysis supplied by the sponsor, some 8% of enrollees in the Slone Survey report no pregnancy testing prior to initiating therapy with isotretionoin following April 1, 2002.  These data also suggest that there has been some improvement in the rate of pregnancy testing prior to initiating therapy with isotretinoin, from 77%-85% to 92%.  However, the improvement is not as great when the comparison is to the recent past and appears to have stabilized / plateaued shortly after requirements per new labeling (April 2002).

 

In DDRE analyses of the Degge/SI cohort (N=5469), there were 5334 women who started treatment with Accutane. All analyses of pregnancy testing or birth control further exclude 249 women who reported a reproductive status of menopause or who had undergone hysterectomy and another 341women of age >=45 years or age <15 years. Of the remaining 4744 women, the number of available patients for any specific analysis is further decreased by exclusion of records with missing information. Of 4719 apparently fertile women aged 15 to 45 years who provided information on sexual activity, 1846 (39.1%) reported no sexual intercourse during or before treatment, 1855 (39.3%) women reported sexual activity before and during Accutane treatment, and 1018 (21.6%) women reported sexual intercourse before starting Accutane treatment but not during treatment.

 

DDRE analyses of pregnancy testing are outlined in Table 6.  Of 4596 apparently fertile women aged 15 to 45 years who provided information on pregnancy testing, 1806 (90.8%) reported pregnancy testing prior to initiating treatment with Accutane.  This is generally unaffected upon stratification by sexual activity status.  Any pregnancy testing was reported by 92.1% among women reporting sexual activity before and during Accutane treatment, 92.7% among women reporting sexual intercourse before starting Accutane treatment but not during treatment, and 88.5% among women reporting no sexual intercourse before or during Accutane treatment.

 

Table 6. Patient report of pregnancy testing, all and stratified by sexual activity status, for apparently fertile women 15-45 years of age who provided data on pregnancy testing.

 

 

N

At least one pregnancy test

Two or more pregnancy tests

Total*

4596

90.8%

66.0%

Sexually active since starting Accutane therapy

1806

92.1%

68.3%

Sexually active before but not during Accutane therapy

993

92.7%

66.8%

Not sexually active before or during Accutane therapy

1797

88.5%

63.0%

*Excludes women who report: (1) age younger than 15 years old or older than 45 years, (2) women who report they have undergone hysterectomy or menopause, (3) women who have not started Accutane treatment, and (4) records with missing values.

 

2.1.6.2 Two pregnancy tests: According to the Accutane label, women should not get the first prescription for Accutane before assessment by 2 pregnancy tests.  While this criterion is not new, the number of pregnancy tests was not included as a question in the old Survey.  The number of pregnancy tests is now included as a question in the new Survey.  Per the sponsor’s analysis, 63% of apparently fertile participants report two pregnancy tests prior to starting Accutane (data not shown).

 

Among the subset of 1806 apparently fertile women 15-45 years of age women who reported sexual activity since starting Accutane therapy, 68.3% reported two pregnancy tests in the month prior to starting their treatment (Table 6, shaded area).

 

2.1.6.3 Pregnancy testing in relation to menses: According to new labeling for isotretinoin, the first test must be done when the prescriber decides to prescribe Accutane, and the second pregnancy test must be done during the first 5 days of the menstrual period preceding initiation of Accutane therapy, assuming the woman menstruates.  Compliance with testing around menses is generally poor at 27% (data not shown), suggesting that clinicians cannot adopt visit / treatment cycles in sequence with recommendations for testing in relation to menses.

 

To address whether the test was taken at appropriate time per approved labeling, the number of days between start of menses and start of Accutane treatment was examined on the subset of women who received pregnancy testing between these two events (Table 7). 

 

Table 7. Distribution of days between starting menses and starting Accutane treatment for apparently fertile women aged 15-45 years who had a pregnancy test sometime between these two events.

 

N

0-5 days**

6-10 days

11-20 days

21-30 days

>30 days

Total*

1851

27.1%

23.9%

26.0%

15.6%

7.2%

Sexually active before and during Accutane treatment

767

28.1%

23.7%

24.6%

16.3%

7.2%

Sexually active before Accutane but not during

429

26.8%

24.5%

25.9%

15.9%

7.0%

Not sexually active before or during treatment w/Accutane

655

26.3%

23.8%

27.8%

14.7%

7.5%

*Excludes women who report: (1) age younger than 15 years old or older than 45 years, (2) women who report they have undergone hysterectomy or menopause, and (3) and women who have not started Accutane treatment. Furthermore, 1524 women were not included in the analysis because of missing values for pregnancy testing, date of starting Accutane treatment, or date(s) of menses.

** “correct” per labeling

Based on DDRE analyses, 27.1% of apparently fertile women aged 15-45 years old on whom data were available received pregnancy testing during the first 5 days of menses. As is further shown in Table 7, compliance with pregnancy testing during menses (for menstruating participants) appeared to be unaffected by sexual activity status.  [It is noteworthy that the data for about 45% of apparently fertile women aged 15-45 is unusable due to missing values for either pregnancy testing, dates of menses, or isotretinoin therapy start date.]

 

2.1.7    Sexual Activity / Birth Control

 

2.1.7.1 Sexual activity: The isotretinoin label states that women should use two separate forms of contraception for at least 1 month before and during therapy, at least one of which should be a “primaryError! Bookmark not defined.” method. The sponsor investigated the reproductive status and contraceptive practices for women enrolled in the Degge/SI survey which suggest a large fraction of women report they are not sexually active (50%).

 

2.1.7.2 Birth Control: Among the participants in the Degge/SI survey, approximately 26% note they are sexually active and use 2 forms of birth control, one of which is primary.  An additional large fraction of women (18%) report they are sexually active and do not use 2 forms of birth control.  The sponsor further stratified the data by age, where non-compliance with use of “two forms” of contraception is shown to generally increase in direct relation to age through age 40 to 44 years where it peaks at 39%.  Non-compliance appears to fall thereafter, although there is only one remaining age band beyond age 40-44 years.  It should be noted that within the age band with the most members (20-29 years, n=1577)), non-compliance with 2 forms of birth control (one primary) among sexually active enrollees is reported at 20%.

 

DDRE further compared trends in fertility/contraceptive status of enrollees in the Slone survey in the four quarters before April 1, 2002 and the following four quarters (Table 8).  These data are derived from “Pregnancy Risk Category” as used in Slone Survey quarterly reports and show status (per Slone categorization) at initiation of Accutane therapy based on DAT-1 surveys.

 

Table 8. Fertility/contraceptive status of enrollees at initiation of Accutane therapy in the Slone survey in the four quarters before SMART and the first four quarters of SMART. 

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

N

1148

1103

1153

1177

1165

1233

1234

1024

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

Hysterectomy /Postmenopausal

4

5

4

3

4

4

4

3

Not Sexually Active, Birth Control*

28

29

28

33

32

34

35

38

Not Sexually Active, No Birth Control

26

28

26

25

26

23

25

23

Sexually Active, Birth Control

39

36

38

37

36

36

35

25

Sexually active, No Birth Control

<1

<1

<1

<1

<1

<1

<1

<1

Unknown

1

1

1

1

1

2

2

1

*Refers to at least one method of birth control including either primary methods (e.g., oral contraception, tubal ligation, injectable/implantable/insertable/patch contraceptives, vasectomy, IUD) or secondary methods (e.g., diaphragms, latex condoms, cervical cap)

Each survey enrollee cohort contains a small group (up to 1%) classified as “sexually active, not on birth control.” In aggregated analyses of these Quarterly Reports, this 1% “rate” does not change over the SMART era and thus appears unaffected by new labeling requirements. It should be noted that this 1% “rate” is based on a denominator of ALL Survey respondents.  After restriction to a denominator of apparently fertile, 14-45 yr-old sexually active participants, this rate becomes ~3%.

 

Trends on contraceptive type (e.g. birth control pills) stratified by age bands and are included in Appendix 4[18]. Although the data for 15 –24 year old enrollees may suggest an increase in use of a “primary” birth control method (51%-60%), variability precludes assessment of a definitive trend since initiation of SMART. These data further appear to indicate that the fraction of enrollees classified as “Not sexually active but on birth control” increased after initiation of SMART (28% to 38%).

 

Of 4719 apparently fertile women aged 15 to 45 years who provided information on sexual activity, 1846 (39.1%) reported no sexual intercourse during or before treatment, 1855 (39.3%) women reported sexual activity before and during Accutane treatment, and 1018 (21.6%) women reported sexual intercourse before starting Accutane treatment but not during treatment.  In DDRE analyses of the Degge/SI cohort, 95.8% of apparently fertile and sexually active 15-45 year-old participants reported that they used some form of birth control early in the course of treatment with Accutane (Table 9); 4.2% of such women reported no form of birth control.  This is consistent with the recalculated rate of ~3% from Slone with restriction to sexually active women.  In total, 46.4% of apparently fertile and sexually active women reported use of “appropriate” birth control consisting of two methods, at least one of which is a “primary” birth control method (e.g., oral contraception).

 

Table 9.  Distribution of birth control method used among apparently fertile women 15-45 years of age within the DAT-1.

 

 

 

Any form of birth control

Any primary method of birth control

Appropriate per Accutane label**

Total

4688*

70.2%

56.8%

23.7%

Sexually active since starting Accutane therapy

1841

95.8%

83.0%

46.4%

Sexually active before but not during Accutane therapy

1008

76.0%

57.7%

24.2%

Not sexually active before or during Accutane therapy

1838

42.0%

30.0%

0.7%

*Excludes women who report: (1) age younger than 15 years old or older than 45 years, (2) women who report they have undergone hysterectomy or menopause, (3) women who have not started Accutane treatment, and (4) records with missing information on birth control.

**Appropriate per Accutane label requires two forms of which one must be a “primary” method (e.g., oral contraception, tubal ligation, injectable/implantable/insertable/patch contraceptives, vasectomy, IUD). As per the Accutane label, women who claim absolute abstinence may elect not to use birth control. Thus, the latter two rows of the table may include women who chose absolute abstinence and this may result in lowering for percent appropriate. There is no question in the patient survey for abstinence: sexual activity used as a surrogate.

 

Further analyses (data not shown) were conducted to examine changes in pregnancy testing or birth control over time among apparently fertile and sexually active women aged 15-45. Between 4QTR2002 and 1QTR2003, we observed a 3% absolute increase in reporting of two pregnancy tests and a 4% absolute increase in reporting of using birth control prior to starting Accutane. Furthermore, the use of a “primary” method of birth control increased 2% and “appropriate” use of birth control declined by 2%.

 

2.2       DAT-1 Bivariate Analyses

 

The sponsor does not include bivariate analyses as outlined in the following section.  Thus, this section includes only DDRE analyses. Data (as tables) are included in this section.  The overall effect of the sticker at initiation of therapy (thus through the DAT1 instrument) is shown in DDRE analyses of sticker and pre-therapy pregnancy test (Tables 10 through 13) and sticker and birth control (Tables 14 & 15).  Due to the potential bias of imputation of missing data, these tables are shown initially with exclusion of missing records, and then with imputation of missing data as a negative result [“missing = NO”].

 

2.2.1    Relationship between Qualification Sticker and Pregnancy Testing

 

The overall effect of the sticker at initiation of therapy is shown in DDRE analyses of sticker and pre-therapy pregnancy test in Table 10.  As shown, the sticker does not appear to relate to performance of the pregnancy test, as pregnancy testing was very high both in the presence of a sticker (91%) and in the absence of a sticker (90%) for all apparently fertile women 15 to 45 year old enrollees. Nine (9) percent of Survey participants who report a Qualification sticker report no pregnancy test.

 

Table 10. Relationship between Qualification sticker and pregnancy testing for all, apparently fertile*, 15-45 year-old enrollees. [Missing=excluded]

 

Qualification sticker

Yes

Qualification sticker

No

Row total

Pregnancy test

Yes

3908

(91%)

90

(90%)

3998

Pregnancy test

No

392

(9%)

10

(10%)

402

 

4300

100

4400**

*excludes women reporting hysterectomy or postmenopausal

**excludes 319 records with missing data for either qualification sticker or pregnancy test; pregnancy testing includes any pregnancy testing performed at physician’ office or performed at home but reported to the physician; restricted to women who reported starting Accutane therapy.

 

As shown in the following table (Table 11), imputation of “NO” for missing data does not appear to change our assessment.  Although compliance with pregnancy testing falls slightly, it does so

both in the presence of a sticker (89%) and in the absence of a sticker (85%) for all apparently fertile women 15 to 45 years old.

 

 

Table 11. Relationship between Qualification sticker and pregnancy testing for all, apparently fertile*, 15-45 year-old enrollees. [Missing=NO]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Pregnancy test

Yes

3908

(89%)

268

(85%)

4176

Pregnancy test

No

495

(11%)

48

(15%)

543

 

4403

316

4719**

*excludes women reporting hysterectomy or postmenopausal

**pregnancy testing includes any pregnancy testing performed at physician’s office or performed at home but reported to the physician; restricted to women who reported starting Accutane therapy.

 

No effect of the Qualification sticker is seen after restriction to sexually active, apparently fertile women 15 to 45 year old enrollees, where the delta between groups is 1% (Table 12).

 

Table 12. Relationship between Qualification sticker and pregnancy testing for Sexually active, apparently fertile*, 15-45 year-old enrollees. [Missing=excluded]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Pregnancy test

Yes

1575

(92%)

41

(91%)

1616

Pregnancy test

No

136

(8%)

4

(9%)

140

 

1711

45

1756**

*excludes women reporting hysterectomy or postmenopausal

**excludes 90 records with missing data for either qualification sticker or pregnancy test; pregnancy testing includes any pregnancy testing performed at physician’ office or performed at home but reported to the physician; restricted to women who reported starting Accutane therapy.

 

As was seen above, little change is noted with imputation of NO for missing responses (Table 13, below). Since the vast majority of these enrollees noted the presence of the qualification sticker, inference about the “effect” of the sticker is limited. 

 

Table 13. Relationship between qualification sticker and pregnancy testing for Sexually active, apparently fertile*, 15-45 year-old enrollees. [Missing=NO]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Pregnancy test

Yes

1575

(90%)

90

(87%)

1665

Pregnancy test

No

168

(10%)

13

(13%)

181

 

1743

103

1846**

*excludes women reporting hysterectomy or postmenopausal

**pregnancy testing includes any pregnancy testing performed at physician’ office or performed at home but reported to the physician; restricted to women who reported starting Accutane therapy.

 

2.2.2    Relationship between Qualification Sticker and ANY Birth Control

 

Per the new isotretinoin label, the Qualification sticker is intended to document that the patient has received education and counseling on pregnancy prevention.  DDRE conducted analyses of Qualification sticker and birth control (as ANY birth control) among sexually active Survey participants (Tables 14 and 15) in order to study the potential impact of the Qualification sticker.  As above, the first table (Table 14) excludes records with missing data while the second table (Table 15) is based on imputation of NO for missing data.  As was seen above for pregnancy testing, there does not appear to be a strong relationship of the Qualification sticker to compliance with birth control as birth control was noted in 97% of enrollees with a sticker and 96% of enrollees without a sticker (Table 14). Three (3) percent of sexually active Survey participants who report a Qualification sticker further report no use of ANY birth control.

 

Table 14. Relationship between qualification sticker and ANY birth control for Sexually active, apparently fertile*, 15-45 year-old enrollees. [Missing=excluded]

 

Qualification sticker

Yes

Qualification sticker

No

Row total

Any birth control

Yes

1671

(97%)

70

(96%)

1741

Any birth control

No

44

(3%)

3

(4%)

47

 

1715

73

1788**

*excludes women reporting hysterectomy or postmenopausal

**excludes 54 records with missing data for qualification sticker; birth control is defined as those who report currently use of ANY birth control method as well as any report of tubal ligation or vasectomy in partner.  ANY birth control use as shown in this analysis is probably an overestimate as women who reported vasectomy in their partner were included as ANY birth control regardless of whether they indicated another option as required per approved labeling.  This analysis approach was necessary because of the poor structure of the questionnaire with regard to birth control using tubal ligation and vasectomy leading to potential for confusion.  In addition, vasectomy may not pertain to all potential partners; restricted to women who reported starting Accutane therapy.

 

With imputation of a “NO” for missing information, compliance with any birth control fell slightly for both arms, to 95% among those reporting a Qualification sticker and 92% for those indicating no Qualification sticker (Table 15), but was otherwise high in both arms.

 

Table 15. Relationship between qualification sticker and ANY birth control for sexually active, apparently fertile*, 15-45 year-old enrollees. [Missing=NO]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Any birth control

Yes

1671

(95%)

70

(92%)

1741

Any birth control

No

95

(5%)

6

(8%)

101

 

1766

76

1842**

*excludes women reporting hysterectomy or postmenopausal

**Birth control is defined as those who report currently use of ANY birth control method as well as any report of tubal ligation or vasectomy in partner.  ANY birth control use as shown in this analysis is probably an overestimate as women who reported vasectomy in their partner were included as ANY birth control regardless of whether they indicated another option as required per approved labeling.  This analysis approach was necessary because of the poor structure of the questionnaire with regard to birth control using tubal ligation and vasectomy leading to potential for confusion.  In addition, vasectomy may not pertain to all potential partners; restricted to women who reported starting Accutane therapy.

 

3.         DAT-2 SURVEY (DURING TREATMENT)

 

NOTE: The sponsor’s submission (“1 Year Report on the SMART Program”) was conducted before accrual of enough DAT-2 questionnaires to be informative.  Thus, the following section describing DDRE analyses of DAT-2 data from the SI/Degge cohort does not have a direct corollary within the “1 Year Report on the SMART Program.”  These data permit inference on SMART attributes/compliance during the middle and later windows of Accutane therapy.  It should be anticipated that the sponsor will also conduct similar analyses to present at the Advisory Committee.

 

A dataset sent to FDA in October 10, 2003 included 3569 women who enrolled in the Degge/SI Accutane Survey from November, 2002 through March 31, 2003 and who returned a DAT-2 questionnaire. One hundred twenty-two (122) of these women apparently never completed a DAT-1 questionnaire and have only a DAT-2. About 47% (N= 1670) of these women reported that they were still taking Accutane.  Thirty-five (35) percent (N=1246) reported that they completed treatment.  The status of remaining 18% (N=652) could not be determined.

 

3.1       DAT-2 Univariate Analyses

 

3.1.1    Qualification Sticker

 

As in the DAT1 questionnaire, DAT-2 participants are asked about the presence of a qualification sticker on prescriptions.  Monthly prescriptions are required for Accutane therapy lasting over 30 days since no refills are permitted under the SMART program.  As shown in Table 16, 94.7% of DAT-2 respondents who continued on therapy with Accutane indicated the presence of a qualification sticker on their prescriptions. 

 

Table 16.  Percent distribution of Qualification sticker from DAT2 for participants who continued on therapy with Accutane.

Total number of sample

1670

Yes

94.7%

I saw the prescription but not I did not see a Qualification sticker

0.8%

Other*

4.5%

*Other includes missing values, someone else filled the prescription, participant did not see the prescription.

 

3.1.2    Pregnancy Testing

 

3.1.2.1 Per Slone Survey: Pregnancy testing during Accutane therapy based on DAT-2 surveys for the interval April 1, 2001 through March 30, 2003 is show in Table 17.  As with compliance with pregnancy testing at initiation of therapy, the proportion of women reporting any testing appears to increase in close association with the SMART program, from ~70% to ~85%.  As seen before, the increase appears to plateau soon after initiation of the SMART program and remain at ~85%.

 

Table 17. Trends in pregnancy testing during Accutane therapy based on DAT-2 surveys (based on quarter of enrollment, based on Slone Survey Quarterly Reports)

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03*

N

1281

1352

1212

1324

1308

1419

1224

1206

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

Any test**

66

66

71

81

87

86

85

87

No Test

33

33

29

18

12

14

14

12

UK

1

1

<1

1

<1

<1

1

1

*Based on receipt of 10-week DAT-2: these counts reflect only the fraction of women enrolled in 1Qtr03 that answered DAT-2 through June 30, 2003.

**Serum or urine pregnancy test

 

3.1.2.2 Per DDRE analyses: In DDRE analyses of the Degge/SI DAT-2 responder cohort for number of pregnancy tests taken during the past two months among apparently fertile women aged 15 to 44 years women still on treatment, 81% of those reporting sexual activity reported apparently monthly pregnancy testing (Table 18).

 

Table 18. Compliance with pregnancy testing during Accutane therapy among apparently fertile* Survey participants by sexual activity status.

 

Sexually active

(N=622)

Not sexually active

(N=832)

Missing

3.2%

3.7%

Mentioned “0” test

10.9%

13.5%

One test

4.8%

7.3%

Two tests

53.4%

51.0%

More than 2 tests

27.7%

24.5%

Appropriate**

81.1%

75.5%

*excludes women reporting hysterectomy or postmenopausal

**represents the sum of the preceding 2 rows as the DAT2 pregnancy testing question applies to a >=2-month window.  Thus, patients should report >=2 pregnancy tests to indicate monthly pregnancy testing

 

3.1.3    Contraception Practices

 

Birth control practices by sexual activity status /treatment status for apparently fertile women aged 15 to 44 years is shown in Table 19. As was seen among respondents to DAT-1, most apparently fertile, sexually active women who remain on therapy with Accutane (97.6%) or completed treatment within one month (94.4%) reported use of some birth control.  Compliance with two forms, one of which is primary (“appropriate” per label) was similar to levels reported in DAT-1 (~55%).  Thus, even during therapy with Accutane, compliance with two forms of contraception as outlined in approved labeling is problematic, with compliance among the highest risk group (young, fertile women who report sexual activity) to be just above 50%.

 

 

 

Table 19. Distribution of birth control use by sexual activity status/treatment status among apparently fertile*, 15-45 year-old participants in DAT2.

 

 

Total**

Any form of birth control

Any primary form of birth control

Appropriate*** per Accutane label

Sexually active, on treatment w/Accutane

608

97.5%

91.0%

 

54.9%

Sexually active, completed treatment for > 1 month

143

94.4%

87.4%

51.1%

Not sexually active, on treatment w/Accutane

813

54.4%

44.2%

8.1%

Not sexually active, completed treatment for > 1 month

145

52.4%

47.0%

9.0%

*excludes women reporting hysterectomy or postmenopausal

**excludes 37 women with missing information on birth control

***appropriate per Accutane label requires two forms of which one must be a “primary” method (e.g., oral contraception, tubal ligation, injectable/implantable/insertable/patch contraceptives, vasectomy, IUD)

 

3.2       DAT-2 Bivariate Analyses

 

3.2.1    Relationship between Qualification Sticker and Pregnancy Testing

 

The relationship between Qualification sticker and pregnancy testing among all, apparently fertile 15-45 year Survey participants continuing on Accutane therapy is show in Tables 20 and 21.  [As in the DAT-1 Bivariate Analyses section above, these tables are included in this section and are stratified by missing=excluded (Table 20) and missing=NO (Table 21).]

 

Table 20. Relationship between pregnancy testing and Qualification sticker for all, apparently fertile*, 15-45 year-old DAT-2 participants.  [Missing=excluded]

 

Qualification sticker

Yes

Qualification sticker

No

Row total

Pregnancy test

Yes

1074

(99%)

254

(98%)

1328

Pregnancy test

No

5

(1%)

5

(2%)

10

 

1079

259

1338**

*excludes women reporting hysterectomy or postmenopausal

**restricted to those participants continuing therapy with Accutane; excludes 116 records with missing data; pregnancy testing includes any pregnancy testing performed at physician’s office or performed at home but reported to the physician; as the DAT2 pregnancy testing question applies to a >=2-month window, patients were categorized as “Yes” only if they reported >=2 pregnancy tests.

 

As was seen in analyses of DAT-1 data, any “effect” of the qualification sticker is limited as pregnancy testing is high among both groups.

 

 

 

 

 

Table 21. Relationship between pregnancy testing and qualification sticker for all, apparently fertile*, 15-45 year-old DAT-2 participants. [Missing=No]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Pregnancy test

Yes

1074

(95%)

305

(95%)

1379

Pregnancy test

No

58

(5%)

17

(5%)

75

 

1132

322

1454**

*excludes women reporting hysterectomy or postmenopausal

**restricted to those participants continuing therapy with Accutane; pregnancy testing includes any pregnancy testing performed at physician’s office or performed at home but reported to the physician; as the DAT2 pregnancy testing question applies to a >=2-month window, patients were categorized as “Yes” only if they reported >=2 pregnancy tests.

 

3.2.2    Relationship of Qualification Sticker and Birth Control

 

The impact of the Qualification Sticker on birth control among sexually active, apparently fertile 15-45 year Survey participants continuing on Accutane therapy is shown in Tables 22 and 23. These data provide little potential to examine the relationship of the Qualification sticker to birth control as these tables are dominated both by Qualification sticker use and birth control.  As noted for bivariate DAT-1 analyses of Qualification sticker and ANY birth control, it is possible that these analyses overestimate any birth control as the analysis approach accepted affirmative responses to questions which should have been skipped.

 

Table 22. Relationship between qualification sticker and ANY birth control for sexually active, apparently fertile*, 15-45 year-old DAT-2 respondents. [Missing=excluded]

 

Qualification sticker

Yes

Qualification sticker

No

Row Total

Any birth control

Yes

560

(99%)

15

(100%)

575

Any birth control

No

5

(1%)

0

(0%)

5

 

565

15

580**

* excludes women reporting hysterectomy or postmenopausal

**excludes 28 records with missing data for qualification sticker; restricted to women who are currently on treatment or had stopped the treatment within the past month.  Per approved labeling, sexually active women are to remain on birth control / refrain from conception for one month following the end of Accutane treatment; questions pertaining to birth control are different in the DAT-2 instrument in comparison to the DAT-1 instrument. However, it is still possible for women to become confused in answering the question. Thus, as in the analysis of ANY birth control use in DAT-1, ANY birth control was defined as those who reported that currently using any birth control method including tubal ligation and vasectomy.  As before, vasectomy was considered a birth control method regardless of selection of an alternative method, as required per labeling;

 

 

 

 

 

Table 23. Relationship between qualification sticker and ANY birth control for sexually active, apparently fertile*, 15-45 year-old DAT-2 respondents. [Missing=NO]

 

Qualification sticker

Yes

Qualification sticker

No

Row total

Any birth control

Yes

560

(94%)

15

(100%)

575

Any birth control

No

33

(6%)

0

(0%)

33

 

593

15

608**

* excludes women reporting hysterectomy or postmenopausal

**Restricted to women who are currently on treatment or had stopped the treatment within the past month as, per approved labeling, sexually active women are to remain on birth control / refrain from conception for one month following the end of Accutane treatment; questions pertaining to birth control are different in the DAT-2 instrument in comparison to the DAT-1 instrument. However, it is still possible for women to become confused in answering the question. Thus, as in the analysis of ANY birth control use in DAT-1, ANY birth control was defined as those who reported that currently using any birth control method including tubal ligation and vasectomy.  As before, vasectomy was considered a birth control method regardless of selection of an alternative method, as required per labeling.

 

4.         PREGNANCIES, POSSIBLE RISK FACTORS, AND PREGNANCY RATES WITHIN THE DEGGE/SI COHORT

 

In other analyses conducted by DDRE but not the Accutane sponsor, the Degge/SI cohort was examined for reports of pregnancy for calculation of an Accutane-exposed pregnancy rate.  Based on DDRE analyses, the Degge/SI dataset contains 28 reports of pregnancy among a total cohort of 5469 women as of the lock date on the dataset.  This is based entirely on information received on/within DAT-1 question 37 and DAT-2 question 17 (“Have you been pregnant at any time since you first started taking Accutane?”).  However, application of this numerator and denominator to calculate an Accutane-exposed pregnancy rate is problematic as women undergoing a repeat course of therapy (~25% of the total cohort) may correctly answer this question as “Yes” based on a un-exposed pregnancy in-between treatment courses.  Thus, it is not possible to interpret the answers reported for this segment of the cohort.  This precludes examination of pregnancy risk among women on repeat therapy.

 

After restriction to the 4277 women on their first-course of therapy, 15 participants reported the occurrence of a pregnancy since starting Accutane therapy. At the time of survey, the median age of these 15 women was 22 years (mean 24.5 years) with range 16 to 39 years.  For comparison, the median and mean of the total cohort of first-time users are 22 years and 24.4 years, respectively.  Eight women reported pregnancy on DAT1; 7 on DAT-2. So as to highlight the problems encountered in analysis of these data for risk factors, 4 out of these 15 women reported no sexual intercourse in both DAT1 and DAT2 questionnaires.  Thus, it is difficult to extract putative risk factors for an Accutane-exposed pregnancy based on the limited information captured by the Patient Survey.

 

In presentation before the FDA Dermatologic Drugs Advisory Committee Meeting (Gaithersburg, MD) on September 18, 2000, Mitchell et al reported 992 pregnancies among 339,994 women completing follow-up in the Survey (through 1999) for a rate of 2.9 per 1,000 women. Based on report of 15 pregnancies, the crude observed pregnancy rate for first-course users within the Degge/SI cohort is 15/4277 = 3.5 / 1000.  It should also be noted that the denominator used in this calculation (4277) includes some women who have not yet completed treatment and thus potentially able to report an Accutane-exposed pregnancy with further follow-up. Thus, since this rate is censored, it likely represents an underestimate of the rate to be realized when all these participants complete follow-up, as was described by Mitchell.

 

5.         SURVEY ADDENDUM (RECENT DATA)

 

FDA has received additional data, in addition to use data, since receipt of the Degge/SI dataset addendum (Oct 10, 2003) in the form of quarterly reports.  As shown in Chart 2[19] there are currently three isotretinoin patient surveys.  All three of these groups currently submit “Quarterly Reports” summarizing data accrued to the end of the quarter.  Quarterly reports from the Slone Epidemiology Group on the Accutane-brand of isotretinoin have been cited previously in this document and Slone continues to submit data (using the old survey) for patients who enrolled to Slone via Accutane materials and/or deferred transfer to Degge/SI.  Degge/SI submitted a Quarterly report for 1QTR2003 based on very limited enrollment as of March 31, 2003 and two additional (and larger) Quarterly reports for 2QTR2003 and 3Qtr2003.  The Slone Epidemiology Group has also submitted two quarterly reports for 2QTR2003 and 3Qtr2003 for generic isotretinoin.  Due to differences in the survey, survey methods, and operating procedures, it is problematic to compare across these disparate surveys.  However, for completeness, selected and generally comparable variables have compiled from the Quarterly reports for 2QTR2003 and 3Qtr2003.[20] In general, these data support results reported for the interval of April 1, 2001 through March 30, 2003. Specifically:

 

·       Report of any pregnancy testing prior to initiation of isotretinoin therapy continues at ~90%.

·       Report of two (or more) pregnancy tests prior to initiation of isotretinoin therapy continues at ~65%

·       Report of NO birth control among apparently fertile, sexually active respondents continues at ~3%.

·       Report of any pregnancy testing during therapy (DAT-2) continues at ~82%.

 

Thus, these data DO NOT show any remarkable improvements in compliance with these selected attributes.

 

DISCUSSION

 

On many points, analyses conducted and described herein by DDRE are in agreement with analyses as reported by the sponsor in their “1 Year Report.”  Some important items of agreement include:

 

·       absolute participation (22%-26% per DDRE; 28% per the sponsor)

·       > 90% report of use of the Qualification sticker

·       > 90% report of one pregnancy test prior to start of therapy

·       ~65% report of two pregnancy tests prior to start of therapy

·       use of two forms of contraception (57% per sponsor based on two forms; 46% per DDRE based on “appropriate” as per label)

 

However, bivariate analyses of the Qualification sticker have only been developed by DDRE. These analyses suggest pregnancy testing and birth control were generally similar between those reporting Qualification stickers and those not reporting Qualification stickers.  In addition, results for the respondents noting a Qualification sticker in these stratified analyses were very similar to those observed in univariate (un-stratified) analyses.

 

However interesting and potentially problematic as these results are, the generation of a pregnancy rate following new labeling could be argued as the final arbiter for success of the labeling, which was not limited to the Qualification sticker.  To this end, it is noteworthy to report the pregnancy rate of 3.5 /1000 for the initial Degge/SI cohort to be identical to the rate reported by the Slone Epidemiology Group for their Accutane Survey.  It appears, therefore, that the efforts implemented in the new labeling did little to affect the absolute pregnancy rate within the population included in the Patient Surveys. [As noted above, the denominator used in the calculation of a pregnancy rate for the Degge/SI cohort (4277) includes some women who have not yet to completed treatment and thus potentially able to report an Accutane-exposed pregnancy with further follow-up. Thus, since this rate is censored, it likely represents an underestimate of the rate to be realized when all these participants complete follow-up, as was described by Mitchell.]

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX 1GEOGRAPHIC REGIONS

 

East:               Connecticut, Delaware, Maine, Massachusetts, New Hampshire, New Jersey, New York, Pennsylvania, Rhode Island, Vermont

 

Midwest:        Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, Nebraska, North Dakota, Ohio, South Dakota, Wisconsin

 

South:             Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, Oklahoma, South Carolina, Tennessee, Texas, Virginia, Wet Virginia, District of Columbia

 

West:              Arizona, California, Colorado, Idaho, Montana, Nevada, New Mexico, Oregon, Utah, Washington, Wyoming

 


APPENDIX 2—DESCRIPTION OF ADVANCEPCS

 

AdvancePCS is one of the largest pharmacy benefit management companies in the U.S., currently covering an estimated 50 million patient lives and processing 300 million prescription claims annually.  FDA has on-line access to AdvancePCS’ database of paid claims for prescriptions filled in 55,000 pharmacies across the country.  Patients whose claims are processed by AdvancePCS include those covered under various types of insurance plans that cover prescription drugs, including some employer’s self-insured planes, selected managed care plans, Blue Cross/Blue Shield plans, and some other traditional insurers.  Demographically, these patients appear to represent all 50 states and include substantial numbers of the elder, children, and women of childbearing age.  Their representativeness of all patients receiving dispensed prescriptions in the U.S. is not known.

 

The data in AdvancePCS include date dispensed, drug name, drug strength, dosage form, quantity, days’ supply, and prescriber specialty.  All patients are assigned a unique identifier, and so can be followed through time for as long as they are in the same prescription plan.  Data are entered into the system within days of dispensing, so that real-time analysis is possible.  At any time, FDA is able to access the most recent 25 months of data online.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX 3RECOMMENDATIONS FOR PCS AND PRESCRIPTION AUDIT

 

The following recommendations cover the PCS, and audit, and also suggestions for making the results of future survey waves easier to analyze and interpret.  The survey recommendations focus on increasing the number of prescriptions captured for analysis, and consistently obtaining an adequate number of prescriptions for all of the survey waves.  The operational audit procedures need to be clearly described in detail, and may need to be altered in order to achieve the goal of data validation.

 

Survey

·       Re-examine sample size and recruiting plan to ensure adequate capture of prescriptions for analysis

·       Consider unifying the survey (sponsor and generic companies) to increase the response rate and decrease the number of inquiries to individual stores for each survey

·       Consider extending the PCS indefinitely (beyond the current 2 year limit) as an indirect measure of physician compliance, assuming that an adequate number of prescriptions can be captured for analysis.  If an indefinite extension is not possible, consider extending the survey at least until a sufficient number of prescriptions has been obtained for a long enough period of time (at least 1 year) to permit a valid analysis of compliance.

·       Consider approaching the December 2002 dropouts to discuss their possible participation, since they account for a very large percentage of the isotretinoin prescriptions.  This approach might be successful if the separate surveys are unified into a single entity

·       If the December 2002 pharmacies will not reconsider participation, consider approaching PBM's or switch companies (firms that route the prescriptions from the pharmacy to the insurance provider) as a way to get to an adequate number of prescriptions.  Again, this approach might be successful if the separate surveys are unified into a single entity

·       Re-evaluate the recruiting process every 1 to 2 surveys until the desired number of prescriptions has been captured for 4 surveys (1 year of data)

 

Audit

·       Present a detailed summary of how the current audit is being implemented, focusing on the recruiting procedures are rules.

·       Re-evaluate the current audit implementation, and make the changes necessary to ensure that the audited pharmacies are randomly chosen from the participants, and balanced across the pharmacy analysis strata

 

Analysis

·       Provide an electronic copy of the cleaned data used for the analysis and production of the tables in the PCS reports (in addition to the paper report).  The data should include information on recruited pharmacies, as well as those that responded and were audited.

·       Include a table detailing the recruited vs. responded vs. audited pharmacies across pharmacy strata in all future reports.

 

 

 

 

APPENDIX 4TABLES AND CHARTS

 

Table A.  Trends in Slone Contraceptive Status for age group 15-24 years

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

N

516

531

556

590

602

610

625

534

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

sterile*

2

1

1

1

2

1

2

1

abstinent

39

38

35

35

33

30

30

31

BCP, BC injection, IUD

 

51

 

52

 

54

 

56

 

58

 

60

 

55

 

60

Barrier, other, UK, none**

 

8

 

9

 

10

 

8

 

7

 

9

 

13

 

8

*self or partner

**includes 0% to 1% of patients per quarter who report no contraceptive use and sexually active without a history of sterility (self or partner).

BCP = birth control pills; BC injection = includes all injectable or implantable contraceptives; IUD=intrauterine device

 

Table B.  Trends in Slone Contraceptive Status for age group 25-34.

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

N

337

290

322

346

251

326

308

277

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

sterile*

24

22

20

20

20

28

23

23

abstinent

7

11

10

8

10

7

7

5

BCP, BC injection, IUD

 

58

 

59

 

59

 

62

 

61

 

56

 

60

 

60

Barrier, other, UK, none**

 

11

 

8

 

11

 

10

 

9

 

9

 

10

 

12

*self or partner

**includes 0% to 1% of patients per quarter who report no contraceptive use and sexual activity without a history of sterility (self or partner).

BCP = birth control pills; BC injection = includes all injectable or implantable contraceptives, IUD=intrauterine device

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table C.  Trends in Slone Contraceptive Status for age group 35-44 years.

era

Last four quarters before SMART

First four quarters of SMART

Qtr

2Qtr01

3Qtr01

4Qtr01

1Qtr02

2Qtr02

3Qtr02

4Qtr02

1Qtr03

N

176

158

155

139

176

158

155

106

 

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

sterile*

59

58

59

63

56

62

60

60

abstinent

6

7

8

6

6

4

4

6

BCP, BC injection, IUD

 

27

 

26

 

30

 

23

 

32

 

29

 

25

 

28

Barrier, other, UK, none**

 

8

 

9

 

3

 

8

 

6

 

5

 

11

 

6

*self or partner

**includes 0% to 1% of patients per quarter who report no contraceptive use and sexual activity without a history of sterility (self or partner).

BCP = birth control pills; BC injection = includes all injectable or implantable contraceptives, IUD=intrauterine device

 

 

Table D. Selected and generally comparable variables as reported for two calendar quarters by three disparate Isotretinoin Patient Surveys.

 

1 April 200230 June 2003

5th Quarter after SMART

1 July 200330 Sept 2003

6th Quarter after SMART

 

Slone

Accutane

Degge/SI

Accutane

Slone

generics

Slone

Accutane

Degge/SI

Accutane

Slone

generics

Any pregnancy test before Rx

 

92%

 

89%

 

92%

 

----**

 

91%

 

92%

Two + pregnancy tests

 

----*

 

64%

 

65%

 

----**

 

69%

 

62%

Sexually active but deny any birth control

 

1.1%

 

2.8%

 

3.2%

 

----**

 

2.6%

 

2.4%

§Any pregnancy test during Rx

 

87%

 

80%

 

----*

 

----**

 

82%

 

----*

* Data unavailable due to time interval or instrument

**Data unavailable due to shift to 6-month frame of reference

§ pertains to cohort establishing in the preceding quarter

 

 

 

 

 

 

 

 

 


Chart 1 Accutane / Isotretinoin Patient Survey Operating Schema.

 

Key:

 

 

 

= Accutane treatment (median)

 

 

 

 

 

 

 

 

 

= Routine questionnaires

 

 

 

 

 

 

 

 

 

= Optional questionnaires

 

 

 

 

 

 

 

 

 

= Period of interest for selected

 

 

 

compliance measures (pregnancy

 

 

 

 

testing, contraception, stickers)

 

 

 

 

10w

 

18w

 

26w

 

34w

 

 

 

 

 

 

 

 

 

D

A

T

 

 

 

 

 

 

 

 

 

 

 

 

 

-1       0             1            2            3            4            5            6            7            8             9           10          11          12  13     14          15          16          17          18

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

DAT1

 

DAT2-10

 

DAT2-18

 

DAT2-26

 

DAT3

 

DAT3

 

DAT3

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Random assignment of 1/3 of cohort to receive either DAT2-10, -18, or -26

 

Additional DAT3’s if original DAT3 was completed <6 months after stop date

 

 

Enrollment

 

 

 

 

 

 

 

 

34w

 

 

 

 

 

 

 

 

 

A

T

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

-1       0                                                                      5            6            7            8             9

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

AT1

 

AT2

 

AT2

 

 

AT2

 

 

 

 

 

 

 

 

 

 

 

 

 

Chart 2. Overlap of 3 currently active, isotretinoin patient surveys.

 

 

S

U

R

V

E

Y

S

 
                                                +(old) SLONE Accutane Enrollment       ________________________________________________________________________________________________________            

 

                                                                                     ++(new) Degge/SI Accutane

  Initial Treatment Survey                                During Treatment Survey

                                                                           DAT 1 ONLY                                                                       DAT 2

                                                                                                      

                                                        ç_____________________________ ç____________________________________________________

                                                          Nov 20, 2002March 31, 2003                    April 1, 2003---------ongoing

     

 

                                                                                                                  ++ (new)  SLONE generics enrollment

                                                                                                                                                                            DAT 1                                                  

           ç___________________________________________________                                                 

 April 1, 2003-------------ongoing                                

                                        

                        

 

 

                                             SMART 1 yEAr  Evaluation  

                   ê___________________________________________ç

                        April 1, 2002---March 31, 2003                         

 

 

                     ç                    ç                        ç                                        ê                                        ê                                                                                    

                                                                                ç                                                 ê                                                                  Ý       

               April 1st Oct           Jan                1st quarter          March        2nd quarter          June                       Aug 15                                                                                                                                                                 31                                   30                      cutoff

Text Box:                                                           2003Text Box:                    2002                                                            

 

 

 

+Survey does not include questions to determine compliance with certain elements of SMART including issuance of a qualification sticker.

++Surveys contain questions that are specifically designed to determine compliance with SMART including issuance of a qualification sticker.


APPENDIX 5EXECUTIVE SUMMARY OF THE SPONSOR’S ANALYSIS OF THE SMART PROGRAM

 

 

Note: The following excerpt is copied verbatim from pages 11-13 of the sponsor’s submission dated June 30, 2003, “1 Year Report on the SMART Program, including the patient survey, pharmacy prescription compliance survey, and global assessment.

 

This report provides a brief regulatory history of the S.M.A.R.T. program, a summary of the first year of the S.M.A.R.T. program and an evaluation of the program based on the predetermined primary and secondary assessment metrics.  The report examines pregnancy reporting for the first year of S.M.A.R.T. in an exploratory manner.  Detailed methodologies and protocols for both the Accutane Survey and the Prescription Compliance Survey are included, as well as for pregnancy reporting. 

 

The data in this report pertain to prescriptions written for Accutane and patients who initiated Accutane treatment during the period April 1, 2002March 31, 2003.  In some cases comparisons are made with historical data from the four quarters prior to initiation of the S.M.A.R.T. Program or from earlier years.

 

This evaluation of the first year of the S.M.A.R.T. Program is affected by four factors:  (1) a change in vendors for the Accutane Survey and an incomplete transition from one vendor to the other; (2) the entrance of generic isotretinoin to the US market in November, 2002; (3) decline in the use of Accutane by females of childbearing potential, and (4) the classification of spontaneous post-marketing pregnancy reports according to a discrete time period of Accutane treatment initiation, as well as according to when they were reported to Roche.  Historically pregnancy case reports have been classified solely on the basis of the date reported to Roche. 

 

All four of these factors may confound the data in this report to a lesser or greater extent.  Each factor, and its potential impact on the data, is described in the appropriate section of the report.

 

The main results of the first year S.M.A.R.T. evaluation include:

 

Primary Metrics, Accutane Survey

·       Accutane Survey enrollment rate of 28.2%, based on a denominator of new female patient starts.  This represents an approximate 10 percentage point increase in the Survey enrollment rate from the 4 quarters prior to S.M.A.R.T., and reverses a downward trend in Survey enrollment rates observed in a retrospective review.  At this time, it falls short of the anticipated 60% enrollment rate.  It is possible that some enrollees since November of 2002 may be double-counted or be taking generic isotretinoin.  At this time there is no way to conclusively identify these patients.

·       Accutane Survey respondents are generally representative of the Accutane female patient universe on age (7 groups), prescriber (dermatologist, non-dermatologist), payer (cash Medicaid, third party), and geographic region of the country.

 

 

Primary Metrics, Prescription Compliance Survey

·       97.2% of Accutane prescriptions have the Accutane Qualification Sticker attached

·       96.4% of Accutane prescriptions with Qualification Stickers have the Qualification Sticker correctly completed

·       There are no significant differences in Accutane Qualification Sticker compliance by geographic region or patient demographics.

 

The presence of a correctly completed Qualification Sticker on the Accutane prescription documents that the female patient is qualified, and includes the date of qualification (confirmatory negative pregnancy test), patient gender, cut-off date for filling the prescription and a 30 day supply limit with no refills.  In order to obtain the Qualification Stickers, prescribers must have read the S.M.A.R.T. Guide to Best Practices and signed and returned the completed S.M.A.R.T. Letter of Understanding which documents that the prescriber understands the safe and effective use of Accutane as described in the USPI, the letter itself, and in the Guide to Best Practices. 

 

Secondary Metrics, Accutane Survey

·       97% of respondents indicate recall of the Accutane Qualification Sticker attached to their prescription (consistent with primary metric of sticker use in PCS)

·       92% of patients recall at least one pregnancy test prior to starting Accutane therapy; pre-S.M.A.R.T. was  76%

·       63% of patients report two pregnancy tests prior to starting Accutane, with slightly higher compliance among women aged 16-39

·       57.5 % of women who indicate the are sexually active report using two forms of contraception; 50% of women indicate they are not sexually active

·       non-compliance with two forms of contraception is considerably higher in women over the age of 20 (20% - 39%), compared with younger women (6%)

·       Accutane Survey enrollment via prescriber office is 56%; pre-S.M.A.R.T. was 21%

 

A high level examination of invalid and absent responses to the Accutane Survey indicated relatively high proportions of invalid and absent responses to several key questions that measure S.M.A.R.T. compliance.

 

There were 97 pregnancies with Accutane therapy start date reported or estimated to be April 1, 2002 – March 31, 2003 with conception occurring prior to, during, or within 30 days after stopping Accutane, reported to Roche as of the cut-off date for this report (May 15, 2003).  At least 6 of these pregnancies occurred in a multi-source environment for isotretinoin.  Because pregnancy case report data are not yet mature for the first year S.M.A.R.T. cohort, no conclusions can be drawn about the impact of S.M.A.R.T. on pregnancy prevention.  The Accutane Survey is the source of a pregnancy rate for women taking Accutane.  Historically, since 1989, this rate is not calculated until 6 months after the last patient entering the cohort completes her Accutane treatment and completes follow-up.  This results in a reporting lag of about 18 months.

 

 

Conclusions

Although the metric of 60% enrollment in the Accutane Survey has not yet been met, the 28.2% enrollment, based on a denominator of new Accutane female patient starts, represents an approximate ten percentage point increase over the four quarters prior to S.M.A.R.T., and reverses a downward trend in enrollment rates.

 

Representativeness of Accutane Survey respondents to the universe of Accutane female patients in the United States was demonstrated for age, prescriber, payer and geographic region of the country.  While this does not ensure the absolute comparability of all Survey respondents to all Survey non-respondents, it suggests that the convenience sample is reasonably representative of the population on the selected parameters.

 

The primary assessment metrics for use of the Accutane Qualification Stickers were met and exceeded.  A third metric pertaining to distribution of the patient Medication Guide was achieved by including the medication guide with the Accutane blister pack.

 

The secondary assessment metrics indicate improvements in pre-prescription pregnancy testing and enrollment in the Accutane Survey via the prescriber’s office compared with pre-S.M.A.R.T.   Age-specific examinations of compliance pre-prescription pregnancy testing and contraception indicate a small age difference for pregnancy testing. and that older women are considerably less likely to use two forms of contraception than younger women.

 

No final conclusions about pregnancy case reports can be drawn at this time because the first year S.M.A.R.T. cohort is not yet mature for pregnancy reporting.

Thus far, the S.M.A.R.T. Program has been successful in some areas (e.g., use of the Accutane Qualification Sticker, improved enrollment in physician office, improved recall of pre-prescription pregnancy testing) and less successful in others (e.g., compliance with two baseline pregnancy tests, compliance with use of 2 forms of contraception).

 

The results of this report suggest several areas of potential improvement including the development of targeted interventions to prescribers and patients who may have difficulties with various aspects of the S.M.A.R.T. Program, and improvement in the methods used to evaluate the effectiveness of the S.M.A.R.T. Program.

 

 

 

 

 

 

 

 

 

 

 

 

 

APPENDIX 6DDRE / DSRCS CRITIQUE AND ANALYSIS OF ACCUTANE/ISOTRETINOIN PATIENT SURVEY(S)

 

Validity of patient survey questionnaire - General methodology and procedural issues with interest in validity and error

 

Isotretinoin patient surveys are currently administered by two separate groups (Slone Epi Group and Degge/SI) using three questionnaires[21].  The Degge/SI and Slone Epi Group use two similar questionnaires for women enrolling with Accutane-specific material and generic isotretinoin materials, respectively.  These generally similar instruments (see below) incorporate SMART-specific metrics (e.g., presence of a prescription qualification sticker).  This instrument was first used during 4QTR02 by Degge/SI.  Slone did not introduce this instrument until 2QTR03.  Women submitting Accutane-brand enrollment material to Slone receive the original (historic) questionnaire that does not include SMART metrics.

 

Both surveys have two arms21: a DAT (during and after treatment) arm and an AT (after treatment) arm. The DAT arm collects information on the compliance and pregnancy events during and after isotretinoin treatment. The AT questionnaire is sent six months after enrollment.  Thus, the information regarding compliance behaviors is gathered retrospectively for the AT arm.  The DAT instrument collects detailed information related to compliance behaviors at specific times during treatment: DAT-1 collects data soon after enrollment; DAT-2 at 10, 18 and 26 weeks after enrollment.  A DAT-3 questionnaire is sent to participants (who remained on therapy for DAT-2) between weeks 26 and 34 post-initiation of therapy and thereafter until treatment is completed.

 

Follow-up on women who enrolled in the first year of SMART will be complete in the third quarter of 2004 (i.e., 6 months following the completion of the traditional 6 month follow-up frame established by Slone for women who enrolled in the survey during the first quarter of 2003).  Therefore, this review covers only the information available as of October 2003. This includes all data from DAT1 and partial results from DAT2 from women who enrolled during the first 2 quarters of SMART. 

 

In an effort made to increase the response rate, the Slone Epidemiology Group conducts a vigorous follow-up telephone process. Three weeks after a second attempt to collect information from an enrollee via a mail questionnaire, the enrollee is contacted via telephone.  Once she is reached, the questionnaire is then administered by telephone. Therefore, patient surveys completed by the Slone Epidemiology Unit incorporate a mixed mode approach.  In general, telephone surveys are not as appropriate as self-administered surveys to obtain sensitive, personal information. A higher “social desirability” is expected in interactive telephone surveys than self-administered surveys. Also, multiple call backs implemented by the Slone Epidemiology Unit have potential to increase compliance behavior.

 

 

 

Measurement error

Measurement error occurs when the survey questions may not measure the construct/objective of the survey because of wording, structure, or concept of the questions. For example, one of the interest areas of DAT1 is to assess whether compliance with pregnancy testing is met at initiation of Acutane/isotretinoin treatment. The questions regarding the pregnancy testing, although sufficient to provide a general assessment about whether the pregnancy test was performed within 4 weeks of starting treatment, otherwise do not adequately address whether the test is done at proper time per the label (i.e., during the first 5 days of menstrual period or 11 days after the last act of unprotected sexual intercourse).  In addition, these questions do not adequately address whether the pregnancy test is also performed prior receiving the Accutane/isotretinoin prescription.  Thus, it is difficult to access pregnancy testing per the Accutane label within the survey.

 

The current Accutane/isotretinoin survey instrument is a follow-up, mail-based, paper-and-pencil self-administered questionnaire (paper SAQ).  Although paper SAQ’s are considered a superior mode to collect honest responses to sensitive questions in contrast to either telephone or person-to-person interviews, it still has limitations, particularly when the responders have concerns regarding the privacy of their responses.[22]  This is relevant as the Accutane/isotretinoin survey is a follow-up survey and thus the identity of the enrollee/respondent is not anonymous to the survey agency.  Under both PPP and SMART programs, female patients are to be educated about the risk of pregnancy exposure and sign a consent form to verify that she understood the risk. As a result, when patients are asked about their compliance behavior, they may report “desirable behavior” more than actual truth in order to avoid possible embarrassment.

 

Another problem with the self-administered questionnaire in this setting is a lack of probing interviewer questioning.  If a self-administered questionnaire with many complex skipping patterns does not effectively guide the respondent to fill out the questions in a appropriate order, the respondent may become frustrated and answer the questions without thought. From, our analysis of the 3rd and 4th quarters of SMART data (SI/Degge survey), it is clear that a large number of respondents did not follow the skip patterns and therefore there is concern about the reliability of the answers.

 

Another important issue about this survey methodology is reliance on the memory of participants to recall the events and experience related to compliance with the SMART program. For example, dates regarding pregnancy testing, prescription date filling, and start of therapy should be precise in order to assess the SMART program.  The closer in time between the date of event to collection of data, the lower the probability of recall bias. Unfortunately, not all women receive and fill out this survey close to the event under study. A large proportion (about one-third) of women respond to the DAT-1 questionnaire 30 or more days after enrollment and initiation of treatment.

 

 

 

Editing/processing error

Editing and processing errors arise from 1) data entry problems, 2) lack of procedures to verify that the data captured are plausible, and 3) post-hoc coding and redefinitions. Although, Degge/SI has implemented many procedures to handle invalid values, DDRE analysis of data from Degge/SI has noted inconsistent skip patterns and invalid/ inconsistent dates for pregnancy testing, treatment start dates, and receipt of drug dates.

 

Coverage Error

The enrollment or coverage rate measures the number of women who enrolled in the Isotretinoin Patient Survey (the sampling frame) in relation to the women who were eligible to participate (the inference or target population).  Coverage errors occur when all women who are eligible to participate are not given an equal chance to participate in the Survey.  Because of the volunteer nature of the Survey, potentially important subgroups of patients may choose not to participate, compromising the generalizability of the Survey to the target population.

 

The calculated enrollment rates were 21% for Slone for the 1st year of the S.M.A.R.T. program, and 14% for S.I./Degge for the period from October 1, 2002 to March 30, 2003.  The sponsor reported an overall enrollment rate of 28% for the 1st year of the S.M.A.R.T program.  Low coverage rates such as these do not automatically indicate that coverage error is present in this Survey; however, the coverage rate is generally used to assess the degree of uncertainty present when generalizing results to the target population, with lower coverage rates corresponding to higher uncertainty.  Due to the recruitment methods and voluntary nature of the survey, there is a real possibility that more compliant individuals preferentially enrolled.  This would lead to a higher likelihood of coverage error and make interpretation of any results problematic, and may limit the usefulness of any conclusions drawn.

 

Unit Non-Response Error

Unit non-response is the failure of participants to respond to the mailed questionnaire.  This type of error would arise if women who responded to the questionnaire systematically differed in some way, particularly compliance behavior, from those who did not return the questionnaire.   As with coverage rate, low response rate does not necessarily mean that any results will be biased, however, the response rate is often used as an indication for the likelihood of bias occurring (the lower the response rate, the more likely that survey results will be biased). Unfortunately, comparison of non-respondents with those who responded to the questionnaire is often not possible, since relevant variables are not routinely available for both populations. For example, information on pregnancy testing done at the physician’s office is available only for women who responded to the survey.  If respondents were more or less likely to have a pregnancy test at the physician’s office than non-respondents, this would result in incorrect survey conclusions regarding pregnancy testing by physicians.

 

For the S.I./Degge survey, the sponsor reported 8,269 enrollees, of which 6,615 (80%) were given the DAT1 questionnaire.  The estimated response rate for the S.I./Degge survey is approximately 83% of enrollees[23], or 5,489 individuals.  When this rate is combined with the 14% enrollment rate, the S.I./Degge Accutane Prescription Survey reflects information on approximately 12% of the women who were prescribed Accutane.

 

The Slone survey had a response rate of 97% for the 1st year of the S.M.A.R.T. program.  This very high rate could be the result of intensive follow-up on the part of Slone.  While the S.I./Degge follow-up consisted of a reminder phone call approximately six weeks after the DAT1 questionnaire was mailed, Slone attempted to contact women repeatedly, up to 14 attempts in a two-week period.  However, despite the high response rate, when combined with an enrollment rate of 28%, the Slone Accutane Prescription Survey reflects information on approximately 22% of women who were prescribed Accutane.

 

Item Non-Response Error

Item non-response is the failure of participants to answer individual items on the questionnaire.  For the Degge/SI Accutane Patient Survey in particular, some of the questions regarding the recall of exact dates have relatively high levels of missing data.  When examined in conjunction with the unit non-response rate, the overall non-response rate is quite high for some variables. For example; the pregnancy test date is missing for about 27% of women who are apparently fertile and are 15-45 years of age. Thus the overall non-response rate for pregnancy test date is 39%. In another words any conclusion about appropriateness of conducting pregnancy test among the enrolled population is questionable since this information is available for 61% of apparently fertile and 15-45 years of age women who enrolled in Degge/SI Accutane survey [83% (response rate) x (100 - 27%)].

 

Assessment of survey questionnaire

 

This section provides detailed comments on individual questions in the Accutane Patient Survey from both Slone and SI/Degge.

 

Overview

DAT-1 is the first questionnaire filled out after the participant enrolls in the program. Later questionnaires (DAT-2 and DAT-3) are shorter and include some questions similar to those in the DAT-1, plus a few questions that are new.  (See the end of this document for a comparison of question content among the DAT-1 and DAT-2 questionnaires.  We have not yet received data from the DAT-3 questionnaires.)

 

The Slone questionnaire was developed first, when the survey began in 1989.  The SI/Degge questionnaire was developed after the S.M.A.R.T. program began in 2002.  For the most part, the SI/Degge DAT-1 questionnaire includes all of the questions used by Slone and adds some additional questions relating primarily to aspects of the S.M.A.R.T. program.  The two DAT-2 questionnaires are similar, but do have a few questions that do not appear on both.  The comments below refer primarily to the DAT-1 questionnaires. The comments below will discuss any notable features of the DAT-2 and DAT-3 questionnaires for both studies, but will focus primarily on DAT-1 because it is more comprehensive.

 

Specific Questionnaire Comments

In general, the DAT-1 questionnaires are lengthy, particularly the SI/Degge version (40 questions, compared with 25 questions on Slone version).  The more lengthy the questionnaire, the more likely that participants will become fatigued, which can result in less accurate data for the later questions.  The questionnaires also are probably difficult for many participants to complete for reasons described below. 

 

Coverage of Important Elements of the S.M.A.R.T. Program

Some important aspects of the S.M.A.R.T. program are not addressed by the questions.   Most importantly, the questions do not completely address

·       whether patients were using two adequate forms of birth control continuously during product use

·       whether patients had sex with more than one male.  Several questions ask about the fertility of a male partner.  However, if the woman had more than one partner, there is no way to report that fact.

·       whether women with amenorrhea were tested for pregnancy at least 11 days after the last act of unprotected sex before starting to take isotretinoin.

 

Questionnaire Development

·       The questionnaires were developed using a set of objectives for the study.  There should have been sufficient questions to cover all of the objectives and every question should relate to an objective.   The following objectives reflect aspects of the S.M.A.R.T. program that were tested by the survey:

1.     to determine female patient awareness of the teratogenic risks of Accutane/Isotretinoin

2.     to measure compliance with key elements of S.M.A.R.T., including informed consent, pregnancy testing, contraception use, Medication Guide, and Accutane/Isotretinoin Qualification Stickers

3.     to calculate the rate of pregnancy among female Accutane/isotretinoin users

4.     to identify risk factors for the occurrence of pregnancy

·       It was not clear what the utility was of some of the questions.  Because the DAT-1 questionnaire, particularly the SI/Degge version, was rather long and complex, questions that are not absolutely necessary should not have been included.

 

Readability and Complexity of Questionnaires

It may have been difficult for many participants to read and respond accurately to the questionnaires due to complicated skip patterns, small print size for some questionnaires, lack of adequate instructions, complexity of some of the questions, language that could have been simplified, and other factors.  Listed below are some of the possible problems of readability and complexity that could have led to difficulties in responding:

·       The overall format is complex, with skip patterns that may be difficult to follow.

·       The print size may be too small on the Slone instruments to be read comfortably.

·       Fatigue may develop due to the length and complexity of DAT-1, particularly the SI/Degge version, making later responses less accurate.

·       For some types of questions, there are no examples provided for how to fill out the questions.  For example, in the SI/Degge questionnaire, there are boxes to fill in with numbers for dates, ages, and numbers of pregnancy tests.  Some participants may not understand how to use the boxes, particularly if there is a single digit number for the response and there are two boxes available.  On the SI/Degge form, there is no instruction on how to fill in circles.  Some participants may have checked them or “X’d” them instead of filling them in with a solid circle.

·       For one question, there is a table to fill out that may be complicated for some participants. The table lists 12 different birth control methods and asks which are currently used and for how long they were used.  For each of the 12 methods, participants have a choice of 13 different responses about the length of use.  There is no provision to indicate uncertainty.

·       For some questions, choices were not mutually exclusive, yet there were no instructions that more than one could be chosen.  For example, the SI/Degge question about whether there was a yellow sticker on the prescription gives among the choices that someone else obtained and filled the prescription and that the patient didn’t see the prescription.  Both of these could be correct if someone else filled the prescription.

·       A footnote on the bottom of the first page of the SI/Degge questionnaire lists a series of questions that should not be answered if the woman has not yet started taking Accutane/isotretinoin (Q. 2-3, 24-27, 30-32, 36-38). It is possible some women would not notice this footnote, which is linked to a small asterisk in one of the choices for the first question.  Even if the footnote had been read, it is highly possible that it would be soon forgotten as the woman worked her way through the questionnaire.  The questions that should have been skipped were not marked as such.  From the content of the questions, it may have been apparent that only those who had taken Accutane/isotretinoin should have responded.  However, it would have been better to make it more obvious which questions were not for women who had not yet started treatment.

·       The terminology used for many questions may be difficult to understand.  Question language could have been simplified.

  • The layout for the question in the SI/Degge questionnaire on whether particular materials had been received and read may be confusing.  There are two lines “pink spiral notebook” and “Self-Assessment” that are not clearly associated with response choices.  It appears that they are connected to the line either below or above them, but because there is no indentation, they are aligned the same as the other lines, and it is not clear where they belong.

 

Poor or ambiguous questions

Some questions seem to be ambiguous, making it difficult for some participants to respond accurately or for us to understand the results fully.  Examples are below:

·       One question asks if the doctor told the patient that it is important not to become pregnant.  It is not clear how a woman should answer if another health care provider told her this information, but the doctor did not.  The participant may not know whether the question is designed to find out who delivered this message to find out if anyone in the office delivered this message.

·       One question asks if certain materials about Accutane/isotretinoin were read.  At the end of the list is "other materials."  Participants may not know what "other" refers to.  We do not know if it refers to materials that are not part of the S.M.A.R.T. program, such as advertisements, something from a website, or from other sources, if it refers to materials from the sponsor or the doctor, or materials from other sources.  We do not know if the purpose of the question to find out if patients read anything besides the documents listed, or whether the question is targeting only specific types of documents, such as those from the sponsor.

·       One question asks if the woman had any other blood or urine tests in the four weeks before starting Accutane/isotretinoin.  We do not know if this question should have been limited only to pregnancy tests, or whether it includes tests for other purposes.  If only for pregnancy that should be made clear.  Women may have had tests for other purposes during this period.

·       There is no provision for responses about more than one male partner.

·       One question asks if the pharmacist provided instructions about Accutane/isotretinoin treatment.  The question does not specify whether the instructions were written or oral.  Almost all pharmacies provide computer-generated printouts of information about prescriptions dispensed, and any information pre-packaged with the product, such as the Medication Guide for Accutane/isotretinoin, would have come from the pharmacist.  Thus, 100% should have responded that they got instructions.  However, if the purpose of this question was to determine if participants received oral instructions, that purpose was not well served by this question.

·       A question on the SI/Degge form asks if the doctor’s office gave the woman a home pregnancy test kit.  It does not specify a time frame. The Slone version of this question asks if the doctor advised the woman to do a home pregnancy test in the four weeks before starting Accutane/ isotretinoin.   The source of the test is not mentioned.  If it is important to know if doctors are offering test kits to patients, the Slone question will not provide that information.

·       The question about whether the woman had been pregnant since first starting Accutane/ isotretinoin may be confusing to women who had stopped the medication before getting pregnant. Further, one choice of responses is "not sure," which is difficult to interpret.

·       The questions about the timing of any pregnancies in relation to timing of treatment are not asked directly in DAT-1.  As a result, the sponsor has had to try to figure out whether women were pregnant at inappropriate times based only on the dates given for timing of treatment and pregnancies.  As these data are frequently missing or may be inaccurate, this method is not ideal.  The DAT-2 and DAT-3 questionnaires do ask about pregnancies more directly and may provide better data regarding timing of pregnancies in relation to product use.

 

Question composition and flow

·       Some questions are leading, making them more likely to elicit a correct response.   These are questions about knowledge of contraception and the effects of isotretinoin on a fetus, as well as questions that are likely to elicit a socially desirable response.

·       There are few "false positive" questions or choices that would enable us to determine if there is a response bias operating for some sets of questions.   Many of the questions describe actions that should have been taken.  It would have been better also to include actions that would have been inappropriate, to see if participants answered them correctly as well.

·       Questions that ask if women knew of certain things, such as the toll-free information line, may result in biased responses.  Many women may be uncomfortable admitting they don't know something and would therefore be more likely to answer affirmatively.  It would have been helpful to ask these questions in another way, such as with a checklist containing actual and non-existent aspects of the program.

·       The questions used to test understanding of the material about birth defects and avoiding pregnancy did not test understanding well.  They included leading questions and simple true/false questions. They did not require application of the knowledge to hypothetical situations that would better test knowledge, and they contained almost no false positive questions.  It would have been easy to score well without much information about avoiding pregnancy.

 

Memory Issues

·