Consumer Health Information for Better Nutrition Initiative - Attachment D - Consumer Studies Research Agenda

Task Force Final Report

Improving Consumer Understanding and Product Competition of Health Consequences of Dietary Choices

Introduction

A central consideration in the development of a regulatory framework for qualified health claims is the importance of ensuring that such claims can be made in a way that is not misleading to consumers. The interim procedures for qualified health claims guidance (see attachment E) provide for three types of qualified claims corresponding to three levels of science. It is not yet certain, however, how well consumers can distinguish between such levels of science, nor is there research-based support for the most effective wording to use in conveying these differences.

In developing the consumer research agenda, FDA conferred with the Federal Trade Commission (FTC) about its ongoing research on health claim issues. FDA will continue to collaborate with FTC and other Federal agencies in developing and evaluating its consumer research agenda. FDA is also interested in private sector research, for marketing studies or other purposes, that may provide evidence relevant to the research questions outlined below. FDA encourages submission of such study results as part of specific health claim petitions. Comments from researchers who have studied these or similar questions may provide additional information to FDA.

Health Claims Experimental Study

FDA plans to conduct an experimental mall intercept study of health claims and disclaimers on conventional foods. The goals of this study are to determine whether health claims that do not meet the “SSA standard of evidence”-level of scientific support are misleading to consumers and to evaluate options for generic disclaimers to correct misleading perceptions. The initial study would focus on qualified health claims on conventional foods. This study would examine claims that meet the SSA standard and three levels of qualified claims. To help address the question of whether graphic formats would help consumers understand the level of science that supports a health claim, a graphic format would be included in the proposed experimental study. A follow-on study would investigate the same claims, qualifiers, and graphic format for dietary supplements.

The conventional foods study would use products appropriate to carry each of the health claims chosen for testing. The health claims would vary in terms of the amount of scientific evidence associated with them, such that one or two claims would be “correct” for each level of scientific support. The level of scientific support would be conveyed by specific wording, for example “…FDA believes that there is some scientific evidence to support this claim.” Certain specific variations in wording would be included in the study, including: (a) “embedded” claims, in which the qualifying language is built into the statement of the substance-disease relationship (e.g., “…Although there is some evidence that substance X reduces the risk of disease Y, that evidence is very limited and preliminary”); and (b) claims that do not include the word “may” in the statement that describes the relationship (e.g., “Consumption of lycopene reduces the risk of prostate cancer”). An additional condition (“full information”) would consist of a short summary that describes the key aspects of the scientific research support for each claim and includes any other information necessary to understand the relationship between the substance and its possible health benefit.

The research questions this study would address:

Can consumers perceive an accurate sense of the scientific evidence? Are qualified health claims, which by definition do not meet the SSA standard of evidence (i.e. Level A), misleading to consumers?

Hypotheses:

On products that display a qualified health claim that indicates strength of scientific support with a disclaimer, participants would rate (a) the strength of scientific support for the diet-disease relationship and (b) the expected health benefits of the product lower than when the claim is unqualified or does not have a disclaimer.
Can consumers distinguish multiple levels of qualified claims?

Hypotheses:

Participants would rate (a) the strength of science lower and (b) the expected health benefits lower as the disclaimers become increasingly strong.
Can the communication effectiveness and wording of disclaimers be improved?

Hypotheses:

FDA has selected wording such as, “evidence is not conclusive,” “limited,” “very limited and preliminary,” to qualify claims with varying levels of scientific support. The planned research would test consumer understanding of these and other terms.

Embedding claims would increase the likelihood participants would recognize different levels of scientific support compared to the same level of claim stated in point-counterpoint style.

Displaying a graphic format designed to help explain the strength of science concept and the relative scientific support associated with a particular health claim would increase the likelihood participants rate (a) strength of science and (b) expected health benefits accurately (lower ratings for stronger disclaimers).

Devices to improve communication effectiveness of disclaimers would bring participant ratings closer to the ratings of nutrition science experts and to participants who are better informed about the science underlying the diet-disease relationship in the full information condition.

Study Design and Protocol

The study would be implemented in seven geographically dispersed shopping malls nationwide. Each site would have available a central interviewing facility that would be responsible for recruiting participants, obtaining participants with the required background characteristics, and observing and documenting the interview experience.

Adults, aged 18 and older, would be recruited for a study about foods and food labeling and would be randomly assigned to label conditions. Participants would be presented with realistic food labels on appropriate food products for each claim and asked a series of questions for each product.

The label conditions would be executed on five food products, appropriate for five different health claims. Multiple versions at each level would be tested, including claims worded with the disclaimer first (“embedded claims”) for Levels B, C, and D. A full information control condition would present a short written summary of the scientific research that supports each claim, including information about food sources of the substance mentioned in the claim.

The selected claims represent four levels of scientific support (currently authorized SSA claim and three levels of qualified claims). All options for disclaimers are replicated for each claim.

It is of particular policy relevance to ensure that consumers correctly understand the limited evidence associated with the most highly qualified health claims (Level D). Therefore, an additional Level D health claim is being proposed, along with an additional wording option for Level D, representing a stronger attempt to convey the limited nature of Level D claims.

To test the effect of including a graphic format for conveying strength of science, the unqualified claim condition and one example of each of the qualified claim conditions would be replicated with an accompanying graphic format selected from the set tested in focus groups to be conducted this summer.

Research Methodology

Each participant would view two or three mock food packages, one at a time. Initially the participant would view only the front panel of the product. To begin, the participant would answer an open-ended question about health benefits associated with the product (“What health benefits, if any, do you think a person would get from eating this product as a regular part of his or her diet?”). Next would be a short set of questions that ask the participant to rate strength of science and health benefits. After these questions the participant would be handed the package and allowed to examine the whole label, which would include an accurate Nutrition Facts Panel (NFP) on the back. The participant would then be asked to repeat their ratings of health benefits “now that you have had a chance to look at the whole package” in order to determine whether participants’ beliefs about the food product would change in the context of a full label. This would provide a more realistic label situation where consumers are able to take the information in the NFP into account when reacting to front panel health claims on food products. In the full information condition, respondents would not see a food label. They would answer the rating questions about strength of science and expected health benefits (where the reference will be to the types of foods mentioned in the information piece).

The key measures for this study would be the rating scales that assess consumer perceptions of the strength of science underlying the claim, and the health benefits expected as a result of eating the product as a regular part of one’s diet.

Strength of science:

¬Based on the information on the label|, how confident do you think scientists are that eating this food regularly will reduce the risk of ¬disease|? On a scale from 1 to 10, where 1 means “Not at all confident” and 10 means “Very confident”

Expected Health Benefits

How much of a health benefit, if any, would eating this ¬food product| as a regular part of one’s diet have on preventing a person from ¬getting (having) disease (health condition)|? On a scale from 1 to 10, where 1 is “no benefit at all” and 10 is “a large benefit”.

Disease/health conditions would include: Having a heart attack, getting cancer, preventing a man from getting prostate cancer, having cognitive dysfunction in old age and getting osteoporosis.

To provide benchmarks for evaluating the ratings across health claim conditions, the strength of science and health benefits questions would be asked of a sample of nutrition science experts. These expert ratings would provide an estimate of the correct answer, which would help in the interpretation of the consumer responses. Similarly, the responses from participants who learn about the diet-disease relationship in the full information condition would also provide a benchmark against which the qualified claim responses can be compared.

Ratings would be classified in terms of response accuracy (correct/incorrect) in relation to the FDA evidence-based ranking system and in comparison to “expert” ratings derived in advance from nutrition science experts.

The full information control condition would consist of a one-page summary of the scientific support for the health claim written at a 6th-8th-grade reading level. The summaries would focus on the current state of science and would provide information on relevant food sources for the nutrient. Qualified health claims concern diet-disease relationships that are often unlikely to be widely known by the respondents. By providing a more extensive description of the level of scientific support for the claim than is possible in the context of a food label, one group of respondents for each claim would represent “informed consumers” for comparison purposes.

Analysis Plan

The extent to which qualified claims are misleading would be determined empirically by comparing the ratings of a claim when it is unqualified to the ratings of the various qualified wordings of the claim. The effectiveness of the generic disclaimers at helping respondents correctly understand the level of science underlying the claims would be analyzed by comparing responses at each level of disclaimer. Expert and “fully informed” consumer ratings would provide benchmarks against which the qualified claims are evaluated.

Statistical analyses (analysis of variance, t-tests) would compare question responses (coded in terms of accuracy) across label conditions. Ratings would also be compared across demographic variables (age, education, gender, health status).

Health Claims Graphic Format Focus Groups

FDA is interested in determining if consumers believe that a graphic paired with a health claim conveys to the consumer better information about level of scientific evidence than does a claim by itself. The Agency would conduct eight (8) focus groups at four research sites across the country to examine consumers’ reactions to a variety of graphic formats designed to signal to them the level of scientific support for health claims (e.g., commensurate with the B, C, and D levels explained previously in this report). Consumers’ reactions to the formats tested in these groups would be used to help determine a graphic format that would be evaluated empirically in the experimental mall intercept study.

A professional market research company would recruit the participants for the eight groups. There would be 8-10 attendees for per group. During four of the groups, consumers would react to claims on conventional foods and in the other four they would respond to claims on dietary supplements. Participants in the dietary supplement groups must have consumed a supplement within the prior two weeks. Attendees would be recruited to include a mix of men and women and a racial/ethnic mix representative of the region. The groups would be segmented by educational level. Respondents could be paid a $50 stipend for their participation in the two-hour session.

A professional moderator would facilitate the groups. The moderator would use a discussion guide to ensure that all topics are addressed in the session. The participants would share their thoughts and feelings about various health claims found on dietary supplement and food labels. They would also be given some brief information about levels of scientific evidence supporting claims, and then would provide reactions to five different graphic formats designed to provide consumers with a visual display of the level of science. They would share their personal opinions about which graphic formats seem to best convey this information, and which they view as least effective. They would also have the opportunity to participate in a brainstorming exercise to develop an even better graphic to accompany and clarify health claim statements. During each focus group, the respondents would offer ideas for the optimal number of levels of qualified claims. They also would review examples of qualified claim language and discuss the words and phrases that best describe the level of scientific evidence supporting a claim. For example, for a qualified claim backed by stronger science, the respondents would react to and compare the signal value of qualifiers such as “the scientific evidence is not conclusive” and “the scientific evidence is very promising, but more research is needed.” The participants would provide reactions to embedded qualified claims and asked to compare such claims to similar point-counterpoint claims. Finally, they would discuss the relative pros and cons of mentioning FDA in the qualifying language, for example, “FDA concludes there is little scientific evidence to support this claim.”

Following each focus group, the moderator would review the videotape and would read the transcript, and then would develop a topline summary. Each topline would include a summary of key findings supported by the respondents’ actual words and phrases. After the eight focus groups are completed, the moderator would develop a final report that would synthesize the information from the toplines. The topline would include an Executive Summary, Key Findings, and Recommendations.

Health Claims Internet Panel Experimental Studies

FDA intends to conduct between three and five experimental studies with Internet panel participants. These studies would empirically evaluate petitions for health claims in order to determine if claim and disclaimer language accurately communicates to a “reasonable consumer” the level of scientific evidence behind the claim.

Internet panels have limitations in survey research, but provide an excellent venue for conducting experimental studies designed to demonstrate cause-and-effect. It is possible to include several questions pertaining to health claims as part of a much larger Internet omnibus panel. This is an economical way to quickly investigate a specific research question. It is also possible to conduct a detailed heath claims research study with an Internet panel that is convened to only respond to this topic. This type of study is more economical to conduct than is a similar-sized mall intercept study.

Internet panel studies are relatively new to the field of consumer research. Depending upon how reliable and sound they prove to be, they may provide an efficient compliment or alternative to mall intercept studies to gather information about consumer understanding and behavior. The agency also believes it is essential to gather data to evaluate consumer behavior, and in turn its impact on actual public health outcomes, in response to changes in FDA health claims policies.

Additional Topics for Consumer Research (Resources Permitting)

The Task Force foresees the utility of conducting additional consumer research in areas such as the following: a) whether consumers perceive health claim messages differently when on foods referred to generically, versus on specific foods, versus on specific brand name products (e.g., fruit juice versus orange juice versus brand name orange juice); b) how consumers perceive comparative risk reduction claims on foods, e.g., “substitute low fat food X for normal fat food Y to reduce the risk of disease Z.”

**Timeline for Health Claims Study**
June 16 – July 25, 2003	Select Peer Reviewers
Sept. 8 – Oct. 10, 2003	Submit Research Plan to OMB for Approval
Nov. 24 – Dec. 19, 2003	Pretesting
Jan. 12 – Mar. 12, 2004	Conduct Study
Mar. 15 – Apr. 30, 2004	Draft Final Report
June 1, 2004	Final Report Completed