Archived Content

The content on this page is provided for reference purposes only. This content has not been altered or updated since it was archived.

Food

Report of Quantitative Risk and Benefit Assessment of Consumption of Commercial Fish, Focusing on Fetal Neurodevelopment Effects (Measured by Verbal Development in Children) and on Coronary Heart Disease and Stroke in the General Population : Peer Review Report

January 15, 2009

Introduction and Summary of Contents:

In early August, 2008, FDA sent two draft documents, now entitled “Report of Quantitative Risk and Benefit Assessment of Consumption of Commercial Fish, Focusing on Fetal Neurodevelopment (Measured by Verbal Development in Children) and on Coronary Heart Disease and Stroke,” and “Summary of Published Research on the Beneficial Effects of Fish Consumption and Omega-3 Fatty Acids for Certain Neurodevelopmental and Cardiovascular Endpoints,” to seven individuals who are expert in a range of scientific disciplines, identified below. The peer reviewers were asked to provide individual, written comments in response to a specific set of questions and to provide other comments on the draft. At the time of review, the two documents identified above were contained in a single document.

We greatly appreciate the peer reviewers’ comments and suggestions, as well as their willingness to provide them relatively quickly despite the length and complexity of the document. The charge questions are set forth below. The reviewers’ full responses to the specific charge questions, and other comments, are provided below without attribution to the specific reviewer.

In the last section of this document we have provided a table that identifies issues raised by the peer reviewers and sets forth whether FDA agrees or disagrees with the comments and describes the actions the agency has taken in response. While we have carefully considered and made several changes in response to the peer reviewers’ comments, this peer review report is being posted in draft form. Further edits to the document in response to these peer reviewers’ comments and to other review comments are likely.

The Peer Reviewers:

Elaine M. Faustman, Ph.D.
School of Public Health and Community Medicine
University of Washington
Seattle, WA 89105

Dr. Faustman graduated cum laude with a dual major in Chemistry and Zoology from Hope College, Holland, MI and received a Doctorate in Pharmacology and Toxicology from Michigan State University. In addition to serving as a Professor in the Department of Environmental and Occupational Health Sciences at the University of Washington, Dr. Faustman is also the Director of the Institute for Risk Analysis and Risk Communication in the School of Public Health and Community Medicine, Director of the Center for Child Environmental Health Risks Research, Director of the Pacific Northwest Center for Human Health and Ocean Studies, and Director of the Reproductive and Developmental Toxicology Core, NIEHS Center for Ecogenetics and Environmental Health, and chairs the University of Washington Chemical Hazards Advisory Committee. She also serves as an Adjunct Professor in the Evans School of Public Affairs at the University of Washington and as an Adjust Professor in the Department of Engineering and Public Policy at Carnegie-Mellon University in Pittsburgh, PA. Dr. Faustman is a Fellow of the American Association for the Advancement of Science, a Diplomate of the American Board of Toxicology and serves as a panel member on the National Advisory Panel for the National Oceanic and Atmospheric Administration's (NOAA) Oceans and Human Health Initiative. Dr. Faustman's primary areas of study are the mechanistic investigation of reproductive and developmental toxicants; molecular mechanisms of action of metals and pesticides; quantitative risk assessment; development of biologically based dose response models for non-cancer risk assessment; in vitro toxicology; molecular epidemiology; toxicogenomics; and public policy. She has served as a presenter and/or chair at several international seminars and symposia, including the International Society of Exposure Analysis (ISEA), International Society for Environmental Epidemiology, the 5th Congress of Toxicology in Developing Countries, and the South Africa Toxicology Society. Dr. Faustman serves as a reviewer for a variety of scientific journals, including: Applied Occupational and Environmental Hygiene Journal; Teratogenesis, Carcinogensis and Mutagenesis; the American Journal of Epidemiology; and Aquatic Toxicology. She has published extensively on toxicity and teratogenicity of metals.

Herman J. Gibb, Ph.D.
Sciences International
Alexandria, VA 22314

Dr. Gibb received a Masters in Public Health (Environmental Health) from the University of Pittsburg and a Doctorate in Epidemiology from Johns Hopkins University. He is President of Sciences International. Dr. Gibb is an Adjunct Professor at the George Washington University School of Public Health and belongs to the International Society of Environmental Epidemiology. Dr. Gibb chairs the World Health Organization's Foodborne Epidemiology Reference Group (FERG) task force on foodborne chemicals and is a member of the FERG source attribution task force to evaluate methods to determine the disease risks from food versus those from water. Before joining Sciences, Dr. Gibb held positions as the Associate Director for Health and Assistant Center Director at the National Center for Environmental Assessment of the U.S. Environmental Protection Agency (EPA). As the Associate Director for Health, Dr. Gibb was responsible for the Integrated Risk Information System, EPA's on-line system of health risk assessments. He was the Project Officer for EPA's cooperative agreements with the World Health Organization. He directed EPA's assessment of inhalation exposures and potential health risks to the general population that resulted from the collapse of the World Trade Center towers. He was the recipient of the EPA's Scientific and Technological Achievement Award for his study of lung cancer mortality and clinical irritation among chromate production workers and the recipient of the EPA's Gold Medal for Exceptional Service for his work on the drinking water standard for arsenic. Dr. Gibb was a member of the White House Interagency Committees on Mercury and Risk Assessment. He was the lead author of EPA's Mercury Research Strategy.

Dariush Mozaffarian, MD, Dr.PH, FACC, FAHA
Harvard School of Public Health
Boston, MA 02115

Dr. Mozaffarian received a medical degree from Columbia University, a Masters in Public Health (Epidemiology) from the University of Washington, and a Doctorate in Public Health (Epidemiology) from Harvard University. He is an Assistant Professor, Department of Medicine, Harvard Medical School and Assistant Professor, Department of Epidemiology, Harvard School of Public Health. He is founder and co-director of the Program in Cardiovascular Epidemiology at the Harvard School of Public Health. Dr. Mozaffarian's teaching has included Cardiovascular Epidemiology at the Harvard School of Public Health and Tufts School of Medicine. Dr. Mozaffarian's primary area of study has been the effects of lifestyle factors on multiple endpoints, including coronary disease, sudden death, stroke, heart failure, and atrial fibrillation. Examples of specific research projects include investigation of relationships of different fish meals with arrhythmic and non-arrhythmic coronary events; relationships of dietary fiber from fruit, vegetable, and cereal sources with stroke and coronary event; relationship of trans fatty acid intake with systemic inflammation; relationship of fish intake with heart failure and atrial fibrillation; effects of fish oil on heart rate in randomized trials, and mercury, selenium, and risk of cardiovascular disease in women and men. The aim of the latter research is to investigate prospectively the relationships of mercury and selenium levels and fish and omega-3 fatty acid intake with incidence of coronary heart disease and stroke. Dr. Mozaffarian has participated on a FAO/WHO Expert Consultation on Fats and Fatty Acids in Human Nutrition and on a United States Department of Agriculture Seafood Education Project Advisory Group. He has published extensively on the effect of fish and omega-3 fatty acid consumption on risk of coronary heart disease and stroke.

Gregory M. Paoli, M.A.Sc.
Risk Sciences International, Inc.
Ottawa, ON, Canada, K1N 6Z4

Mr. Paoli has a Master of Applied Science in Systems Design Engineering from the University of Waterloo. He is President of Decisionanalysis Risk Consultants, Inc., specializing in risk assessment and risk management in the field of public heath and safety. Within Canada, Mr. Paoli has served on Expert Committees of the National Roundtable on the Environment and the Economy and is a member of Health Canada's Expert Advisory Committee on Antimicrobial Resistance Risk Assessment. He has provided guest lectures at the Queen's University's Public Sector Executive Programme and School of Public Policy, the University of Calgary's Faculty of Management and the University of Ottawa's Institute of Population Health. In the United States, Mr. Paoli has served on an Institute of Medicine Committee tasked to Review the United States Department of Agriculture's E. coli 0157:H7 Farm-to-Table Process Risk Assessment. He was appointed to a NRC Committee entitled "Improving Risk Analysis Approaches Used by the Environmental Protection Agency." Mr. Paoli served for several years on an Expert Panel to develop a risk ranking framework for the FDA and was on the peer review panel for the Harvard BSE risk assessment. He has served on several international expert panels including Expert Consultations as part of the Joint Food and Agriculture Organization and World Health Organization (FAO/WHO) Activities on Microbial Risk Assessment. Mr. Paoli has provided training in risk assessment approaches in North America, Japan, and South America. He also provides lectures as part of the Harvard School of Public Health continuing education course in Probabilistic Risk Assessment.

Barbara Petersen, Ph.D., M.P.H.
Exponent
Washington, DC 20036

Dr. Peterson received a Masters in Public Health in Nutrition from the University of California at Los Angeles and a Doctorate in Biochemistry from George Washington University. She is employed by Exponent, an engineering and scientific consulting firm, where she serves as Principle Scientist in Exponent's Health Sciences Center for Chemical Regulation and Food Safety. Dr. Peterson's areas of expertise include exposure assessment methodology, functional food safety and efficacy evaluations, food consumption profile modeling, and applications of Mote Carlo techniques to risk assessments for chemicals, including contaminants, pesticides and nutrients. Dr. Peterson has directed the design and conduct of numerous statistically based market basket studies, including acute and chronic assessments for pesticides, compliance assessments under proposition 65, and market research. Dr. Peterson served on the EPA Science Advisory Board's Integrated Exposure Committee and as an Expert Advisor to the FAO/WHO for several sessions of its Joint Expert Committee on Food Additives and Contaminants and for numerous consultations on risk assessment. She also served as Principal Investigator for the National Cancer Institute's International FOODBASE project, a major effort to collect and computerize descriptive and summary information on food consumption surveys conducted in more than 40 countries. Dr. Peterson has provided statistical support to FDA's Center for Food Safety and Applied Nutrition, including developing criteria for evaluating nutrition databases, and to EPA's Office of Research and Development. She has been a faculty member in risk assessment training programs for government scientists in the European Union, Thailand, the United States, and China. She has published extensively on methods for estimating dietary exposure.

Kimberly M. Thompson, Sc.D.
Harvard School of Public Health
Boston, MA 02115

Dr. Thompson has a Master of Science degree in Chemical Engineering Practice from the Massachusetts Institute of Technology and a Doctor of Science degree in Environmental Health from Harvard University. She is an Associate Professor of Risk Analysis and Decision Science at the Harvard School of Public Health Department of Health Policy and Management and the Department of Society, Human Development, and Health. She is the creator and Director of the Kids Risk Project. Dr. Thompson is also Associate Professor of Risk Analysis and Decision Science (Pediatrics), Children's Hospital, Harvard Medical School, where she is co-founder of the Center on Media and Child Health. Prior academic appointments include Visiting Associate Professor, MIT Sloan School of Management. Dr. Thompson's research interests and teaching focus on issues related to developing and applying quantitative methods for risk assessment and risk management, and consideration of the public policy implications associated with including uncertainty and variability in risk characterization. She is particularly interested in issues related to variability in risk for sensitive sub-populations, particularly children, and the potential risk tradeoffs associated with policies designed to protect them. The work includes research on a range of children's risks including injury, environmental, medical, and product-related risks, as well as perception of children's risks and the portrayal of risky behaviors in popular entertainment media. Her publications includes the book entitled "Risk in Perspective: Insight and Humor in the Age of Risk Management," a guide to help consumers take charge of health information.

Renee C. Wachtel, MD
Department of Developmental and Behavioral Pediatrics
Children's Hospital and Research Center at Oakland
Oakland, CA 94609

Dr. Wachtel received a medical degree from the State University of New York, Downstate Medical Center. She is Director of the Division of Developmental and Behavioral Pediatrics, Children's Hospital and Research Center at Oakland, Oakland, California. Previously, she was Professor of Pediatrics, University of Maryland School of Medicine, and the Director of the Division of Behavioral and Developmental Pediatrics, University of Maryland School of Medicine. She was also an Associate Professor of Pediatrics at The Johns Hopkins School of Medicine. Dr. Peterson serves as Co-Chairperson, Committee on Developmental and Behavioral Pediatrics, Northern California Chapter, American Academy of Pediatrics; Co-Chairperson, Committee on School Health, Northern California Chapter, American Academy of Pediatrics; and Chair, Autism Task Force, Children's First Medical Group. She has conducted research and published extensively on a range of neurodevelopmental issues.

The Charge to the Peer Reviewers

Each expert peer reviewer was provided with a written “charge” concerning the document, as follows:

When the FDA generates a scientific assessment, it is presenting its scientific evaluation about the accumulated evidence. The peer review should provide input on the reasonableness of judgments made from the scientific evidence. The result should be an independent determination by each peer reviewer as to the appropriateness of (a) the assumptions made and hypotheses postulated, (b) the methodology utilized, (c) the quality and relevance of the data and information, (d) the accuracy of the analytic results, and (e) whether the conclusions reached are supported.

Charge Questions:

  1. Is the document logical and clear?
  2. Were scientific assumptions explained and are they appropriate?
  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included?
  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?
  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?
    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions?
    3. (5c) Are scientific assumptions explained and are they appropriate?
  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used?
  7. Are the intervention scenarios appropriate?

The Peer Reviewer Comments (provided in random order and without attribution)

Peer Reviewer Number 1:

I. GENERAL IMPRESSIONS

This well written and well researched report seeks to provide updated scientific evidence regarding the health risks and benefits of eating commercial fish products. The purpose is to assist the FDA in developing consumer advisory information about the health consequences of exposure to methylmercury (MeHg) from fish products in the diet of American consumers. Three "health endpoints" are considered: effects upon neurodevelopment, fatal coronary heart disease and fatal stroke. In addition to reviewing the available scientific literature, quantitative risk modeling is included, to provide a quantitative assessment of potential health effects of different potential health advisories. Using statistical methods, the document seeks to separate the potential health consequences of the unintentional ingestion of MeHg found in some fish products from the potential health consequences of ingesting commercial fish products, some of which contain negligible amounts of MeHg.

The results of the report are well supported and indicate that for the general public, there are substantial advantages to fish consumption, and small, if any, risk, if low MeHg fish are consumed. Neither the thumbnail summary nor the executive summary has a section at the end that provides these conclusions.

This topic is complicated by a number of factors:

  1. The variation in commercial fish species, MeHg concentration between and within fish species, portion size, difficulty of reliable dietary recall, especially over prolonged periods of time.
  2. Variations between studies in methods of assessing MeHg exposure (e.g. blood, hair, cord blood).
  3. Differences in timing (e.g. at one year of age, or later in life) and techniques of measuring outcome variables (e.g. IQ, age of walking).
  4. Many of the studies on neurodevelopment have small sample sizes, which limit the ability to generalize from the data.

All of these potential confounding issues are discussed in the report, including potential explanations for why individual studies have different results regarding health effects. The extensive bibliography is helpful in providing documentation about the scientific basis of the report's discussions and findings. A few suggested additions are noted below. Adding the two references about the documented beneficial effects of fish oil would add to the premise of the report that there are clinical studies documenting the beneficial effects of fish components upon neurodevelopment.

As a medical professional with specialized expertise in neurodevelopment, and as an advocate in the developmental disability community, I have several additional concerns about what is missing from the document, rather than what is in the document:

  1. There is an extremely limited discussion of ethylmercury exposure, with most risk assessment assuming that all organic mercury is MeHg. Given the large public concern about exposure to ethylmercury related to immunizations, ignoring this issue, instead of putting it into the proper context of mercury exposure will be problematic to parents and pregnant women concerned about neurodevelopment. A clear discussion about mercury levels after exposure to immunizations containing thimerisol would help the evaluation of fish consumption exposure tremendously. While thimerisol is largely out of current immunizations, most the of studies cited were conducted when thimerisol was used, and most consumers reading the report will not understand that ethylmercury exposure has been greatly reduced without clarification in the report.
  2. Much of the concern about the effects of mercury upon neurodevelopment has been raised since the autism epidemic gained public attention in the past 10 years or so. There is no mention of autism in this document, and none of the studies reviewed seemed to have had any children with autism in the cohorts. Given the current prevalence of autism as 1 in about 150 children, this topic (the relationship of mercury exposure to autism) should be included in the document, if only to discuss the limited data available and fact that there are current studies seeking to answer this question (see below).
  3. The major concern in the disability community is not whether eating fish during pregnancy or early childhood will change the age of talking or walking by a day or two, or the IQ by a point or two. The concern is whether there are some individuals (either the fetus/child or pregnant women/fetus) who are particularly susceptible to the deleterious effects of MeHg exposure. Thus studies that look at outcome measures of average age of walking/talking or average IQ do not address whether some individual children might have been harmed. Since autism is not diagnosed generally until after the age of 2 years, and 25-30% of children with autism show normal early development and then regression in talking, the risk analysis performed would not address this concern.
  4. Also, looking at average exposure does not address the public concern about whether spikes in MeHg ingestion from commercial fish products and timing of effect during fetal or infant development might affect neurodevelopment . While this area is briefly discussed in the report, and the limitations of individual studies to address these issues, it is not sufficiently highlighted and explored.
II. RESPONSE TO CHARGE QUESTIONS
  1. Is the document logical and clear?

    The report is very well written, well organized, and well supported. While the neurodevelopmental effects research was well characterized from a MeHg exposure point of view, there was limited discussion of the limitations/strengths of the different developmental outcomes measures used and their generalization ability. Some of the studies were more credible than others, based upon outcome measures used and ages of testing. But in the discussion and analysis, all neurodevelopmental outcome measures were considered equally credible. For example, in the Hibbeln 2007a study, early developmental outcome was measured by the DDST, a screening test administered by the parent, with a subset actually tested using a standardized measure (the Griffith) at 18 months of age. The accuracy of developmental measures is well known to be much better within the normal range at later ages. Much more compelling data in this paper is the standardized IQ data from age 8 years, administered to more than 5000 children, with generally better p values as well. An advantage of this study is the effort to control for a number of environmental variables. Unfortunately, this study did not differentiate species of fish consumption, portions or assessment of MeHg levels. Similar results were found by the US study by Oken et al., which was able to correlate MeHg levels, maternal pregnancy fish consumption, and standardized child outcome measures (but at age 3) in a prospective cohort of 341 maternal-child pairs.

    A table comparing the different studies would be very helpful to allow the reader to compare the studies and their results more easily.

  2. Were scientific assumptions explained and were they appropriate?

    In general, the modeling presented data variables and assumptions in a comprehensive fashion, although at times hard to follow for a non-statistician. A clearer summary would be helpful.

    Some modeling assumptions seem questionable, such as subtracting 3 months from the Iraq data since the ages of walking and talking were recorded in 6 month increments, which seems arbitrary and probably not supportable.

    However, the "what if" modeling of neurodevelopmental effects was not sufficiently explained. For example, what is the definition of "age of talking?" Was it first word including names (mama or dada), first word not including names (not including mama and dada), age of having a specific number of words in his/her vocabulary, age of speaking in phrases or sentences? These normal milestones range from 10 months to three years. Standardized developmental tests are much more predictive, especially if "age of talking" is obtained retrospectively. The modeling based upon amounts and types of fish consumption were much more clearly delineated.

  3. Has the appropriate literature been cited? Are there publically available, peer reviewed papers that should be included?

    Some additional references are:

    Jacobson JL and Jacobson SW. Risks to Child Health from Methylmercury Exposure in Immigrant Populations. Journal of Pediatrics 2006; 148:716-718.

    Innis et al. Increased levels of mercury associated the high fish intakes among children from Vancouver Canada. Journal of Pediatrics 2006; 148:759-763

    Clifton JC. Mercury Exposure and Public Health. Pediatric Clinics of North America 2007; 54: 237-269.

    Sato et al. Antepartum seafood consumption and mercury levels in newborn cord blood. Am. J. Ob Gyn. 2006; 194:1683-8.

    Holland et al. Maternal supplementation with very long chain n-3 fatty acids during pregnancy and lactation augments children's IQ at 4 years of age. Pediatrics 2003:111, e39-44.

    Dunstan et al. Cognitive assessment of children at age 2.5 year after maternal fish oil supplementation Arch. Dis. Child. Fetal. Neonatal Ed. 2008; 93:F45-50.

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    The conclusions presented in the report are well supported by the various methods of analysis and literature review.

  5. Specifically in regards to the quantitative risk assessment:

    See above section.

    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?
    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions?
    3. (5c) Are scientific assumptions explained and are they appropriate?
  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used?

    See my comments in Section I. I am not sure that enough data exists to perform modeling about these issues, but the areas need to be addressed.

  7. Are the intervention scenarios appropriate?

    Yes, but the intervention strategies will need to be crafted better than current strategies to be effective. We do not want to have a complex message that will confuse consumers, or precipitate changes in dietary behaviors that are the opposite of the message.

III. SPECIFIC OBSERVATIONS:

In the thumbnail summary, it would be more understandable if the literature review was cited first, and that it included both prenatal and postnatal exposure upon neurodevelopment studies. The risk assessment then performed as part of this report would have a clearer context when discussed thereafter.

The executive summary is extremely well written and clear. The discussion of the different approaches to neurotoxicity and risk/safety assessments is helpful background information for the reader of the report. Dividing the literature review to different levels of Average US Exposure is a very useful method to help the reader keep the level of exposure in different studies in context.

PEER REVIEWER NUMBER 2:

I. GENERAL IMPRESSIONS

The document takes on a difficult question – a quantitative risk-benefit analysis of fish consumption contaminated by methylmercury. The document does a good job of laying out the assumptions, methods, limitations, and conclusions in its evaluation of risks. In general, it provides sufficient detail for the reader to understand how the document's conclusions were reached. Additional detail is needed in some areas, however (see discussion below). The document considered several models and "what if" scenarios in its evaluation which allowed the authors to explore various approaches and ways to look at the issue and made the evaluation more enlightening. The document describes some of the limitations in knowledge (e.g., the constituents of fish that could contribute to neurological development, the relationship between omega-3 fatty acids in fish and methylmercury, whether a diet relatively high in selenium can neutralize methylmercury toxicity in humans) as important areas for further research. The document also points out the differences in the levels of methylmercury in the studies where effects have been observed with the levels consumed in the U.S. (e.g., Iraq and Minamata 100X; Seychelles, Faroe Islands, and New Zealand 10X, etc.). This is helpful to the reader in gaining a perspective of the studies where effects have been observed vis-á-vis the U.S. experience. While the overall impression for the reader is that consumption of fish is beneficial, methylmercury could present a problem for women of child-bearing age under some scenarios. The document is written for a U.S. audience, but the document will be used worldwide and that consideration should be made in developing the final message, particularly with respect to what the document refers to as the methylmercury-to-fish ratio. For example, for localized populations in the Amazon living near gold mining operations or populations in the Orient where shark or tuna may represent a significant food source, unrestricted consumption of fish could present a problem. The same may be true for some populations in the U.S. Overall, however, unrestricted fish consumption would appear to be a benefit in most populations.

II. RESPONSE TO CHARGE QUESTIONS
  1. Is the document logical and clear?

    In general the document was logical, and the presentation was clear.

    The executive summary, however, is too detailed and needs to make the summary points more concisely. Particularly confusing in the executive summary is the discussion on Quantitative Risk Assessment for Fetal Neurodevelopment. Perhaps the Thumbnail Summary or a slightly expanded Thumbnail Summary could be made the Executive Summary?

    My understanding of Table IV-11 is that the first percentile of the population could experience negative effects, presumably because they did not eat enough fish. It took me several readings to understand that, however. I think the confusion is caused by the title of the first column, "Population Percentile." (Population percentile of what?) The text which discusses the table could also be made more explicit.

    The first time that the word, "baseline" is used in the document is on page 19. Discussion on baseline continues throughout the text, but it is not until the bottom of page 132 that there is a good definition of what is meant by "baseline."

  2. Were scientific assumptions explained and are they appropriate?

    One assumption that seems questionable is the assumption that the combination of the Iraq (Marsh et al. 1987) and Seychelles (Myers et al. 1995) data "was sufficiently close to a methylmercury effect minus fish to give us a reasonable approximation of it." (page 107, middle of page). There were 680 mother-infant pairs from the Seychelles (where fish were consumed) and 81 mother-infant pairs in Iraq. It is difficult to understand how fish could not have had an effect in the model at the lower dose (Seychelles) given that there were so many more mother-infant pairs in the Seychelles study than in the Iraq study yet the document states in several places that the models were dominated by the Iraq data (e.g., page 107, second bullet; page 114, first para). It is also unclear how the data can be combined to estimate age of first talking or age of first walking when the ages of the Iraqi children are unknown.

    A second assumption that is questionable is the combining of the UK data (Daniels et al.) study with the Iraq and Seychelles data in the combination net effect model given that the Daniels et al. study did not examine age of talking. The authors argue (page 108) that the Daniels et al. study did include tests of verbal comprehension at young ages and therefore the results are comparable to the Iraq and Seychelles data, but verbal comprehension occurs at an earlier age than talking. Since the Daniels et al. study did not observe an adverse effect of methylmercury, the inclusion of the study with the Iraq and Seychelles data could therefore bias the model (underestimate the effect of the methylmercury) since age of comprehension is younger than the age of talking.

    Finally, it is not intuitively obvious how one combines data from a study where there was no fish consumption (Iraq) with two studies that examine methylmercury exposure via fish consumption to arrive at a net evaluation of fish and methylmercury consumption.

  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included?

    The appropriate literature has been cited.

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    The conclusions appear reasonable given the results of the quantitative risk assessment; however, some of the assumptions used to develop the assessment may be questionable.

  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?

      There was sufficient information on how the data were identified for the quantitative risk assessment, and the criteria used to determine the suitability of the data for the quantitative assessment were adequately described (pp 110-111, 141, 152-153). A primary criterion for choosing the data for the assessment was the access to individual, as opposed to group data. This resulted in the selection of three studies – the Iraq study, the Seychelles study, and the UK study (Daniels et al.). The selection of these three studies may open the analysis to criticism in that it identified two large studies (Daniels et al. and Myers et al.), both of which found no effect of methylmercury, to combine with a much smaller study (Iraq) where effects from methylmercury were observed. Individual data provide a much better basis for model development, however, and thus provide a defensible position for the selection of these three studies. Interpretation of the data from the three studies in an overall statement on risk-benefit may be complicated by the assumptions which are discussed above, however.

    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions? 

      It is unclear what is meant by "two-dimensional population models" (page 132). There is some discussion of it in an appendix, but more description of what is meant by the term should be provided in the body of the text.

      The document states that in the "absence of methylmercury," the "model" was able to predict the age of first walking and the age of first talking. The "model" predicted a "central estimate" for first walking of 10.4 months (page 117) and a "central estimate" for first talking of 15.1 months (page 114). These "central estimates" are early in comparison with the literature on child development. Most textbooks suggest a central estimate for walking of 12 months. It depends what is meant by "talking," but the first word other than mama/dada is usually spoken by 9-14 months, but 4-6 words are not said until 11-20 months and two word combinations until 14-21 months. An explanation needs to be provided for the discrepancy between the model estimates and the data in the literature on ages of walking and talking.

    3. (5c) Are scientific assumptions explained and are they appropriate?

      See the discussion of assumptions in my response to charge question 2 above.

  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used?

    I am not aware of any.

  7. Are the intervention scenarios appropriate?

    Yes, given the current FDA/EPA guidance to limit their consumption of fish to 12 ounces per week during pregnancy.

III. SPECIFIC OBSERVATIONS

p. 54, paragraph beginning "Two years later," - Change "Gandjean" to "Grandjean."

p. 120, paragraph beginning "Cohen et al…" - Change "Harvand" to "Harvard."

p. 126, first paragraph - Change 63 grams to 28.6 grams.

p. 132, paragraph beginning "The age-of-talking …"- Change "Given the evidence in support a beneficial…" to "Given the evidence in support of a beneficial…"

p. 142, paragraph beginning "Of the thirteen…"- Delete "of" in the first line.

PEER REVIEWER NUMBER 3:

Primary review comments on the MeHg RA:

From an analytical perspective, the overall approach to risk assessment for methylmercury represents a relatively thorough treatment of information from the literature (e.g., gaining access to raw data for intensive meta-analysis with respect to dose-response relationships). While the scientific-level reasoning is thoroughly described, the nature of the computational approach taken and the details of the implementation are not adequately described. As a result, it is not possible to determine the validity of the computational implementation.

Documentation and Overall Computational Structure:

The overall computational structure of the model is poorly described, with a vague description spanning only a few pages in Appendix A. A crude representation of the flow of information is provided in two high-level diagrams (Figures AA-18 and AA-19). While these figures are appropriate and simple enough to be understood within a general summary, they are not sufficient to describe the computational approach. Relative to the details of the computation they are superficial and do not provide nearly the amount of information that would be required to fully understand what is being computed, and how all of the many computational model elements work together. The relative amount of documentation given to interpretation of the scientific literature as compared to the computational implementation of the model is quite imbalanced given that the computational implementation is just as much part of the results as the many scientific assumptions and the raw data.

This level of report documentation may have been more tolerable if the Excel implementation had been adequately documented separately, within the Excel worksheets or within the VBA code itself. As a simple example, the document (Appendix A) indicates that the model is implemented in Microsoft Excel. While true, the full implementation spans multiple worksheets in multiple workbooks in multiple folders, with a Word document entitled "What's on this disc?" providing less than a quarter of one page as the documentation of the computational structure of the system and the relationships between the various files. As a result, when reviewing the document, it is not clear in which of the many files (and within each file, which macro or worksheet) the corresponding technical implementation is performed. The inter-relationship between these many files is not possible to construe (except as can be inferred in a time-consuming and quite uncertain process from scrutiny of the raw VBA code). The VBA implementation appears to use a custom VBA library called MC2D but it appears to have no separate source of documentation (it is not referenced in the report). The principal VBA macros are not identified, requiring scrutiny of the raw VBA code before determining which macro should be run first.

The macros themselves are generally not documented. Even when some documentation is provided, it tends to be minimally helpful or unhelpful. At one point, for example, there is the following line:

"GammaTrap = alpha * beta ' use mean value if gamma function craps out"

The purpose of this function must be inferred from the VBA help documentation on the built-in function gammainv, which indicates that this is an algorithm which will not always produce a numerical solution. This code proceeds to replace the intended value with the mean of the distribution when Excel produces an error. It is not clear if this function will be used as a frequent part of the simulation, or if it is a piece of code that captures an extremely rare exception in the application of the gammainv function that can be safely ignored. It is also not clear whether the replacement of error-handled estimates with the mean of the distribution will provide a bias toward reduced or elevated exposure estimates, or will simply narrow the apparent variability. It would be a simple matter to determine how often this 'Trap' was required in the development of the estimates.

In the same module, there are the following lines:

GSD = 0.37 + 1.4 * CurrentModel(SpeciesIndex, 2)
beta = 0.11 + 0.34 * (1 - CurrentModel(SpeciesIndex, 2))

These lines come with no documentation. Through scrutiny of the code and by inference, the reviewer might guess that these lines in the code represent the use of the uniform distribution referred to in the paragraph below (p. 174 in electronic copy provided). However, it is not clear where the constants come from. It is only by determining (from elsewhere in the VBA code) that CurrentModel(SpeciesIndex,2) is a random number between 0 and 1 that it is possible to make this inference. This type of guesswork must be repeated every few minutes by a reviewer of this model.

"Since raw data were unavailable for some species, distributions were generated with modeled distributions that reflected reported arithmetic mean values published from a National Marine Fisheries Service survey (NMFS 1978) for each group and a range analogous to those obtained from tuna, shark, and swordfish. Lognormal and Gamma distributions were used to represent the data, with each model assigned a probability of 0.5 to represent model uncertainty. The magnitude of the shape parameters (the geometric standard deviation of the lognormal distribution and the beta parameter of the gamma distribution) were represented as uniform distributions that encompassed the range of values resulting from fitting the shark, swordfish, and tuna data. The scale parameters (the geometric mean of the lognormal distribution and the alpha parameter of the gamma distribution) were calculated from the arithmetic mean in the NMFS survey and the shape parameter (Evans et al., 2000)."

This situation is unfortunate given the minimal effort that would be required to facilitate understanding: a simple line of documentation within the code stating that this was the implementation of a uniform distribution and that CurrentModel was a random number distributed as Uniform [0,1] would have been immediately apparent and would save considerable effort in scrutinizing the code. Similarly, a simple footnote in the report indicating that the paragraph above was implemented in a particular macro and/or a particular spreadsheet would avoid having to match the report description with the code implementation in a haphazard, time-consuming fashion.

As another example of the guesswork required, the document does not indicate how long it might take to run the model (and, for example, if it takes a long time as one might assume, whether it could be run in stages). There is also no indication as to what uncertainty and variability iterations would be minimally required to produce reliable results. The number of iterations that are applied in providing the report's results is never justified as providing a reasonably stable estimate of the mean, median or of any percentile of the resulting distributions. This may be particularly relevant given that there is a delicate balancing of risks and benefits which may be prone to being affected by numerical stability of statistical estimates. If this sensitivity is known not to be critical, it would be a simple matter to provide a summary of the stability of estimates as a function of simulation size (both uncertainty and variability iterations). Given the number of uncertainties applied in the model and the repeated use of probability trees, it is not possible to hazard a guess as to what level of simulation might be required to reach a given level of stability. In addition, there appear to be multiple places in the code where the actual 2-D simulation might be governed. There is an uncertainty loop within the MeHg intake model, as well as the apparent role of the MC2D model in conducting a 2-D simulation model for the risk estimates. It appears as though the MC2D model may provide some form of resampling from the uncertainty iterations in the MeHg intake model, but this again is largely a guess pieced together from a minimal description provided in Appendix A and the VBA code.

The repeated requirement for 'guesswork' both at the micro level (e.g., what individual lines of code are intended to achieve) as well as at the macro level (e.g., the general lack of documentation as to how the many spreadsheet files interact to form a complete simulation) forbids a formal characterization of the model as being sound. In the end, while no concrete 'fatal' errors were found in the computational implementation, it is simply not adequately documented to allow sufficient scrutiny to support or refute an opinion that the model and corresponding results are technically sound. Any opinion would simply be an opinion on what the reviewer has come to believe is being computed rather than being based in knowledge of precisely what is being computed. Determining the soundness of the model would require either considerably more documentation both within and outside of the model software, or extensive interaction with the author(s) of the code to determine exactly what is being computed, where, and in what order.

The inability to render an opinion given this level of documentation is unfortunate, particularly given this reviewer's general impression of the approach is that it is generally sound and might even be considered exceptional in the extent of the attempt to generate a faithful representation of the uncertainty that manifests from the raw data in both exposure assessment and dose-response assessment. This effort has been undermined, at least to this point in its development, by the inadequacy of the computational documentation. Had the computational implementation been better described it would have been possible to provide more substantive comment on the approach taken.

The reviewer recommends that the computational implementation be thoroughly described both within the VBA code and in a separate formal appendix and then subject to further review, particularly given a favorable overall impression of the underlying theory being applied in structuring the model. The overall structure of the simulation should be described as a separate matter from the treatment of any specific piece of evidence in the model. This level of documentation would allow for a peer review of the model and its results without the distraction of uncertainty as to how the results are ultimately computed.

PEER REVIEWER NUMBER 4:

I. GENERAL IMPRESSIONS

The FDA assessment is comprehensive. The assessment addresses both the risks (due to methylmercury in fish) and some of the benefits (of fish consumption). In conducting this assessment FDA has explicitly acknowledged the body of literature demonstrating that substantial health benefits accrue from eating fish while at the same time acknowledging and equally substantial literature demonstrating the toxic effects of methylmercury. The assessment was designed to quantify the dose response for several effects including neurological developmental effects, coronary heart disease and fatal stroke in a way that allows a determination of a net benefit (or if appropriate a net benefit).

FDA has done a thorough and balanced job of assessing both the risks and benefits.

Key attributes of the assessment include the detailed in-depth analysis of all of the available literature using techniques that weigh the benefits versus the risks (and on the same scale). Because my personal expertise is in dietary exposure assessment, I have focused especially on the analyses of exposure – the methods used to estimate short and long term consumption and to conduct the assessment.

The study acknowledges the potential benefits of eating fish (due to factors other than methylmercury) and conducts an in-depth assessment of the "net benefits."…Essentially the benefits are measured with the same endpoints as "adverse" effects but without any attempt to identify what the factors are or to measure the presence or absence of the factors. For example, it may be that benefits are conveyed due to consumption of fish that contain omega 3 fatty acids and/or selenium and these nutrients offset some or all of the adverse effects of methylmercury. The FDA assessment does not attempt to identify these factors or to estimate their presence in the diets (other than a brief analysis of fish vs. marine mammals).

The FDA model recognizes that there many amounts of fish consumption that provide a net benefits while other amounts of fish (either very low or very high) that provide net risks. Therefore, the models incorporate very sophisticated dose response modeling. In doing so, it appears that all of the available data have been used – in one analysis or another.

The study notes that very high intakes of methylmercury have been demonstrated to cause net negative effects and as a result of contamination events in Japan and Iraq have convincingly demonstrated the adverse effects of methylmercury at high doses. Subsequent studies in which populations were exposed to much lower levels have looked for the same effects – with somewhat conflicting results. FDA has conducted in-depth evaluations using the individual data from many of these studies. The results are presented in a clear, understandable format and the order of those presentations is helpful. In particular, FDA has made it clear that there are very likely thresholds for different effects and simulated those thresholds by identifying the studies where consumers were eating similar amounts to Americans, and those where consumers were eating 10 times versus 100 times as much fish (and by extrapolation methylmercury) as Americans.

Given the available data, I cannot suggest any further analyses that would be of value to the users of this assessment. I have made a few suggestions below that would assist the reader in the understanding the implications of the findings.

The complexity of these analyses makes it imperative that there be a rigorous QA/QC of each set of data and analyses. I attempted to independently derive some of the estimates. I did not find any significant differences but could not begin to conduct a true QA/QC of the components of the assessments. It would be useful to include summaries of the QA/QC in an appendix to the report as a way of documenting the conclusions.

OVERALL CONCLUSIONS:

In summary, I cannot suggest any additional analyses that I think would make a meaningful difference in the report. I think additional discussion of some of the analyses would help most readers understand the significance. The uncertainty analyses and "what if" scenarios provide useful measures for understanding the impact of potential policy guidance. The difficulty in identifying a "threshold" type value where the adverse effects exceed benefits is clearly demonstrated in this assessment. In fact, I am persuaded by these analyses that such a threshold must be well above the levels consumed by even the high fish consuming populations that have been studied to date except where in the 2 studies where there was extremely high levels of methylmercury in the food (Iraq and Japan).

II. RESPONSE TO CHARGE QUESTIONS
  1. Is the document logical and clear? 

    Yes. As I noted above, the FDA document is thoughtfully organized and provides a clear description of the methods, assumptions and even the ways in which each of the datasets were utilized. In my opinion, the evaluations that the FDA assessment considered are as comprehensive as the data would allow.

    On p. 180 there is a discussion of diet-blood relationships and results that could be noted in more detail in the text. This is particularly important given the ongoing biomonitoring form NHANES and other sources.

    It does not appear to me that there is any realistic scenario (e.g. based on what is known about the American diet) where American consumers would consume so much commercial fish that they would have a net risk from eating fish. In other assessments I have seen discussion of subsistence fisherman consuming large amounts of fish from a single source that is highly contaminated and it is possible to assume that a pregnant female consumes large amounts of high methylmercury fish on an on-going basis. These diets aren't reported in the national surveys – nonetheless FDA has shown prudence is identifying "cut off" points of the amount of such fish that might result in net negative effects. It appears to me that these cut-off points are more restrictive than necessary based on the risk assessment (in other words I didn't see a clear threshold around 12 ounces of fish).

  2. Were scientific assumptions explained and are they appropriate?

    Yes. I reviewed the literature on fish consumption and cardiovascular disease and am convinced that there is clear evidence of a positive benefit in consuming fish and that this is the consensus of a wide array of scientists. The components of the fish that provide these benefits have not been conclusively determined although omega 3 fatty acids and selenium have been suggested. The lack of knowledge concerning the factors and the almost certain variability in the levels in different types of fish makes the analyses attempted by FDA difficult. Nonetheless, the results are in agreement with those of other scientists and should be highlighted. In several places the possibility of a net adverse effect was noted in the report but it was not clear to be whether this was a remote possibility (e.g. the extreme bounds of a confidence interval based on conservative assumptions) or a plausible possibility. This was highlighted in the thumbnail on p. 8 where there is a statement "this combination net effect" model estimates results that are beneficial most of the time but that can occasionally be adverse…" However, I could not determine a realistic scenario where the net effect would be adverse …on even a short-term basis (and this appears to be confirmed in the "what if" analyses). If the "adverse" scenarios occur only rarely I think this sentence should be deleted or further qualified. At a minimum there should be additional discussion in the text.

    I have assumed that QA/QC have been conducted to ensure that the modeling is as described as it was impossible in the time frame and with my level of expertise to repeat most of the analyses.

    The Finnish study of fish consumers appears to suggest adverse effects due to fish consumption but given other potential differences (genetic differences, alcohol and smoking, etc.), it is possible that other conclusions could be drawn based on these findings. Differences in test subjects should be noted in the text. This study highlights many of the difficulties in looking at differences in different populations. There are probably population differences that are larger than any likely adverse impact of methylmercury in the fish of these populations and it would be helpful to make some note of this in the report.

  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included? 

    I believe all of the appropriate literature has been cited (and more importantly considered in designing the risk assessment).

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    The analyses incorporate all of the important variables. The methods used to integrate the consumption data and the methylmercury concentrations in food are appropriate and are widely used. The approach to correcting for long term consumption is the best available methodology (although new (2003-2006) NHANES data will allow this approach to be validated since short term (2 day) and longer term (30 day) are now available for the same individuals. (Based on preliminary work with the new day, I believe the findings will be essentially the same as though obtained through the methodology in this report).

    More specifically, I considered the variables that should be included are listed below along with my assessment of FDA's handling of these variables:

    • sources of methylmercury in the diet: FDA concluded that fish are the largest source of methylmercury which is in agreement with all the literature I could locate and the data used in the analysis are the most comprehensive that are available and allowed a variety of different types of analyses.
    • variations in levels by types of foods and within the fish categories by fish species - the data were most limited for these analyses but FDA did take this into account; the fish grouping on p 174-177 seems appropriate.
    • amount of each food consumed at an eating occasion and the frequency of consumption – both portion size and frequency were incorporated into the analysis; the LTSTCR seems important but I couldn't understand the specific method that was used (a more complete discussion would be helpful); on p. 73 there is a portion size adjustment – I assume this is a per capita assessment but it isn't labeled.
    • impact of cooking on methylmercury – the impact of cooking was considered (cooking doesn't affect methylmercury except in that water/fat is lost/gained from the food).
    • methylmercury versus total mercury – handled as well as possible given the data available.
    • time period of interest (acute versus chronic exposures) – the exposures of primary interest – e.g. to the US population on an ongoing basis – are chronic exposures and these were estimated.
    • population of interest (entire US population, adults, children, women of child-bearing age, etc.) – the important populations were evaluated.

    For the exposure modeling exercises I considered the following questions:

    • Were appropriate assumptions made in conducting the basic analyses and the "what if" analyses? The analyses as summarized used appropriate assumptions and
    • Were the impacts of the assumptions tested? the "what if" analyses allow the reader to understand the impact of many of the key assumptions. In my opinion the discussion of the impacts are "timid," e.g. the upper bounds of the uncertainty are given more weight than seems appropriate in many cases (e.g. the reader isn't given enough guidance as to what is plausible and what is not).
    • Were the models standard (re. accepted methodology)? The models are both standard and unique (e.g. this level of complexity is ordinarily not possible because of a lack of data and resources). If I had been conducting the analyses I would have attempted to conduct the same analyses and I believe the application of the data and models are appropriate (again it was impossible to test all of the components).
    • Do the results make sense? The results make sense in terms of the range of exposure and the likely impact of different scenarios. A discussion of the likelihood of any of the "what if" scenarios actually occurring would be helpful in understanding how best to apply the results. The correlations of exposure based on blood, hair and diet are helpful and appear to have been done correctly. I concur that is was appropriate to exclude the subject with 849 ppm. Figure AA-5 suggests to me that a log scale for the hair values might be more appropriate (was this tested)? The Poland Study (Jedryschowski et al) – analyses in Appendix A don't seem to be discussed in the report. Comment: Polish study (p. 58) – the source of the methylmercury is not discussed.
  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used? 

      Yes. I have a few suggestions that I think would improve the report (in terms of providing those of us with less familiarity of the risk assessment models more background information so that we can better understand the significance of the results).

      187 – top of page, "the estimate is employed in our analysis as a normal distribution..." It would be helpful to know why this was assumed to be a normal distribution and whether a different assumption would make any difference (if so how much). I believe this could be done without actually doing additional modeling based on the statisticians understanding of the impact of this assumption versus a different assumption.

    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions? 

      Comments on the assessment of the role of methylmercury/fish consumption on neuro-development:

      Yes. The results seems to suggest that the threshold for the delays in the age of talking is above any of the exposures that were seen in the long terms epidemiological studies (e.g. Saychelles, Faroe Islands, NZ) except for the poisoning episodes in Iraq/Japan. That is, the delay in the age of talking: consumption over 12 oz doesn't seem to delay talking (in a clinically significant way unless the fish are highly contaminated)…I don't believe that it is possible to model a situation where you could eat enough fish with typical levels of methylmercury to cause a delay beyond 1-2 days (the investigators in the Iraq study used 6 month intervals).. I reached similar conclusions regarding IQ decrements. It would be helpful to the reader is the actual changes (in numerical terms) were inserted in the bullets on p. 165. On p. 193 the table (Net IQ decreases) has both IQ and delayed walking – are the units the same? In my opinion the results of the UK study (Daniels et al 2004) should be further discussed in the text. The levels of exposure to methylmercury are likely to relevant to US populations and the findings show net positive benefits.

      Comments on the assessment of the role of methylmercury/fish consumption on CHD death rates:

      Tables IV-16-20 needs additional explanation of the assumptions and findings (and even the units).

      The findings of these analyses highlight the importance of more people consuming fish on the incidence of CHD death. I don't believe the results apply to fish consuming individuals who simply consume more/less fish. Am I correct? This section would benefit for an expanded discussion of the assumptions and findings.

      Comments on the assessment of the role of methylmercury/fish consumption in fatal stroke:

      On p.158 the difference in the size of the confidence intervals between the Carrington and Bouzan based model is highlighted. I believe these differences are likely due to the differences in data that were included in the two models. If that is correct, it would be helpful to mention that.

      At the top of p 159 the documents notes "...47 additional stroke deaths may be caused by fish consumption." I think it should be clear that the model attributes these deaths to fish consumption…and the net benefit should be expressly calculated an included.

    3. (5c) Are scientific assumptions explained and are they appropriate? 

      Yes. It is important to note that the risks are from a different component than the benefits (My understanding as result of my review of this report and of the published literature is that there is no benefit of methylmercury itself so the benefits are due to other components of the fish (ALA and other long chain fatty acids most likely).

      For most audiences of the report, the assessments would benefit from some additional general background information about the range of variability in the types of endpoints that are measured. For example, changes in onset of talking and walking are highly variable endpoints that are likely to be affected by many factors. Although the FDA analysis provides indications of the variability, there is little discussion about the precision of these types of measurements (inter and intra individual variability in measurements) and effects of IQ, socioeconomic status, etc. on the measurements. Background information will be particularly helpful since the studies were done in different populations (with different baselines) and different researchers used different surrogate measures for the endpoint. Also the large battery of tests that were run in some studies make it even more likely that one or more of the endpoints would show statistical significance just by chance (again, comparison to clinically important value would assist the reader). The end points involving behavioral and clinical assessments may not be able to be detected or to be clinically meaningful. For example, can changes in IQ of less than one point be meaningfully measured or would not be judged to be different? If not, then predictions of less than 1 point should be defined as a "no effect?" Similarly is a delay of < 1 day in talking/walking of any clinical significance?

  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used?

    There are probably other endpoints that could be assessed but based on the epidemiological studies they would be very minor and of no public health consequence. FDA has identified those with the largest effects.

  7. Are the intervention scenarios appropriate? 

    The "what if" scenarios raise one of the most important regulatory questions and the discussion of the assumptions that went into these analyses should be expanded along with additional discussion of the impact of assumptions on the outcome and the likely significance to those making policy decisions, e.g. what guidance is appropriate?

    Based on actual poisoning episodes (Iraq and Japanese populations), very high levels of methylmercury intake must be avoided. The impact of the intake of lower levels of methylmercury does not appear to cause any significant adverse effects in any of the populations studied (despite decades long studies). Thus the challenge is to determine the "threshold" where the net benefits of fish consumption cross to net adverse effects due to methylmercury (and most likely other substances).

    Additional comments on some of the scenarios:

    p. 19 "what if scenarios" that exclude benefits – should be clearly identified and not used for policy decisions. Fish is not consumed without benefits.

    p. 20 "…other sources of nutrients found in fish" – based on a review of foods – this is not a likely scenario in my opinion

    p. 21 – "…could produce a small neurodevelopmental decrement (up to 4 %) in some people." My review of the data does not suggest that this arise in any realistically meaningful situation. Additional discussion should further highlight the assumptions and their impact on the outcome.

    The discussion of the results of the "what if" guidelines should specifically note that methylmercury exposures are a function of the amount of the food consumed and the level of methylmercury in the fish. Therefore, the "what if" scenarios must address both variables as should any resulting "guidance," e.g. should guidance focus on amounts of food consumed, types of food (e.g. high methylmercury containing fish) or both?

    These results indicate that the difference in risk of adverse effects above and below 12 oz per week is not large (and would not be clinically meaningful unless the concentrations in the fish are as high as those seen in frank poisoning episodes or are at the highest levels seen in the TDC and those species are consumed all of the time by high fish consumers). While it is possible to imagine such a scenario, none of the available data identify individuals in the US population who are consuming such species on a long-term basis and that should be noted in the discussion. On p. 164 in the first paragraph under (g) Summary and Interpretation, the statement is made "the risk assessment predicts, that on a population basis, capping maternal fish consumption at 12 ounces would reduce the national average level of neurodevelopment slightly even when pregnant women eat only 'low methylmercury' fish." This is an important finding and should be further explained since current guidance includes the 12-ounce figure. Would this same effect be seen if it were capped at 16 ounces/week? Or is it the type of fish that should be capped? Or is it the need for more variability in the type of fish?

    On page 65:the report states "the risk assessment indicates that encouraging consumption up to, but not beyond 12 ounces per week essentially accepts some risks in order to obtain nutritional benefits from fish consumption, while also avoiding slightly higher risks and benefits associated with higher consumption..." In my opinion, this is a premise which, when tested in the risk assessment, wasn't confirmed. The risk assessment didn't show that limiting consumption to12 ounces produced any clinically meaningful difference in outcomes. There was a net benefit at much higher levels of consumption – even considering the high methylmercury fish.

III. SPECIFIC OBSERVATIONS

Editorial note: The figures in the various chapters are incorrectly referenced in the text in several places including in Appendix A…Figure AA-4 on are incorrectly referenced in the text.

PEER REVIEWER NUMBER 5:

I. GENERAL IMPRESSIONS

First, the U.S. FDA should be lauded, in no uncertain terms, for undertaking this crucial and critically important analysis. Public health recommendations must have – must – as their principal intent the improvement of health. The traditional principles of toxicology (risk)-based assessments were developed to consider toxin-containing exposures that were otherwise generally health-neutral, e.g., paint, plastics, water (and most foods), etc. Such principles can lead to bizarrely illogical results when applied to exposures that otherwise have important health benefits. This problem is particularly patent for fish consumption, for which the health outcomes that may be worsened by toxins in fish are the same as those that may be benefited by nutrients, especially omega-3 fatty acids (but also selenium and vitamin D), in fish. Thus, risk-assessment of both risks and benefits is essential to enable sound conclusions, and consequently appropriate recommendations, for health effects of fish consumption.

Many great strengths are evident. The report focuses on the endpoints of greatest relevance: neurodevelopment and cardiovascular disease. Studies of both risk and benefit are considered. Both qualitative and quantitative analyses are included. Generally, the conclusions are sound and reasonable. However, careful attention to revisions to address a few crucial limitations would greatly improve the accuracy of the information provided and, especially, the clarity of presentation and soundness of conclusions. In summary, these include:

  1. The rationales, goals, and descriptions of the MeHg-only analyses are dangerously muddled. Performing this analysis and stating that it is an estimate of the effect of MeHg "if ingested in fish that are lacking in nutrients that could have a beneficial effect" is wholly invalid. This description and interpretation have little scientific justification and defeat the purpose of the Report – to evaluate the risk to U.S. consumers from MeHg in commercial fish products. There are certainly no commercial fish products in which MeHg is present without any other beneficial nutrients, and indeed there are no fish products of any kind in the world in which MeHg is present without any other beneficial nutrients. In no uncertain terms, this interpretation of this analysis, and any and all references to "fish consumption" in the descriptions of the modeling results, must be removed.

    Performing this analysis as an estimate of the effect of MeHg "if ingested in food other than fish" is clearer, although not very meaningful – where such levels of MeHg would come from, absent another grain disaster, is unknown. Such theoretical exposure is not meaningful for the U.S. public. Further, this is not a Report to investigate the potential effects of MeHg in isolation (just as it does not quantify the potential effects of EPA+DHA alone, selenium alone, or vitamin D alone). This modeling should be deleted altogether, or at the very least removed from the Summaries and placed in an appendix, as it is not based in any real scientific scenario and presents highly misleading and confusing "findings."

  2. Comparability of risk assessment methods is essential to allow valid results. The lack of comparability of methods for determining dose-responses of MeHg and neurodevelopment endpoints, fish and neurodevelopment endpoints, and CHD and neurodevelopment endpoints is striking. The end result is that, paradoxically, endpoints with relatively little data (MeHg and neurodevelopment, fish and neurodevelopment) have relatively small uncertainty, whereas endpoints with enormous amounts of data (fish and CHD) have spuriously large (and incomparably calculated) uncertainty. In particular, as modeled, the assumption regarding an independent (different) effect in each individual study of fish and CHD is unwarranted, and such a drastic assumption is not made for MeHg and neurodevelopment or fish and neurodevelopment.
  3. Whereas a principal strength of this report is the quantitative risk-assessment, these quantitative results are lost in much vague and imprecise language of the Thumbnail and Executive Summaries (arguably the most important sections). The precision and detail of the descriptions of the quantitative results in these Summaries require close attention.
  4. There is a strong but apparently unrecognized disconnect in the report between methods for consideration of MeHg (assessed by biomarkers) vs. fish consumption (in this report, assessed only by questionnaires), and the direct relevance of these differences for the accuracy of the information, clarity of presentation, and soundness of conclusions.
  5. A more formal consideration (and description) of hierarchies of evidence is essential. Based on emphasis in the report (e.g., page content alone), one would be led to believe that the best evidence comes from small retrospective case-control studies and poorly controlled case series; then somewhat less robust evidence from modestly sized retrospective case-control studies and small prospective cohorts; then somewhat less robust evidence from short-term randomized controlled trials of intermediate phenotypes, and then the least robust evidence from large prospective cohort studies and randomized trials of disease outcomes. Clearly, this order should be exactly reversed, together with reversal of the report's greater attention to relatively small studies with many limitations vs. little attention to large studies with greater strengths.
  6. Compared with the details presented for neurodevelopment in the Thumbnail and Executive Summaries, the content for CVD is strikingly cursory. While the stronger interest of the panel for neurodevelopment is evident, as well as the historical importance of this outcome for prior FDA recommendations, CVD is clearly at least as important for public health. Much more attention to the CVD data and conclusions in the Summaries is required.
II. RESPONSE TO CHARGE QUESTIONS

The Report has many more strengths than limitations, and once again the panel must be lauded for their efforts. My comments below focus largely on the few – but very important – limitations that are present in the Report.

  1. Is the document logical and clear? 

    Generally yes, with three important exceptions:

    1. (A) The "MeHg-only" analyses are neither logical nor clear.

      Performing this analysis as an estimate of the effect of MeHg "if ingested in fish that are lacking in nutrients that could have a beneficial effect" is wholly invalid. This description and interpretation have little scientific justification and defeat the purpose of the Report – to evaluate the risk to U.S. consumers from MeHg in commercial fish products. There are certainly no commercial fish products in which MeHg is present without any other beneficial nutrients, and indeed there are no fish products of any kind in the world in which MeHg is present without any other beneficial nutrients. In no uncertain terms, this interpretation of this analysis, and any and all references to "fish consumption" in the descriptions of the modeling results, must be removed. Descriptions could remain that discuss "maternal consumption of MeHg up to xx amounts per day in foods other than fish" etc., but not fish.

      Importantly, even the first interpretation (MeHg from foods other than fish) is not meaningful for exposures in the U.S. public, and further this is not a document to investigate the potential effects of MeHg in isolation. Thus, this modeling should be deleted altogether as it is not based in any real scientific scenario and presents highly misleading and confusing "findings" in a Report of quantitative risk-benefit assessment of fish consumption. Neither EPA+DHA-only, selenium-only, nor vitamin D-only analyses are performed – why should MeHg-only analyses be performed and highlighted so strongly? The MeHg-only analysis could be included in an appendix (but certainly not in the Executive or other Summary!), but with clear definitions that this is an analysis of MeHg exposure not from fish, and with removals of all references to fish consumption as a metric, as described above.

    2. (B) Whereas a principal strength of this report is the quantitative risk-assessment, these quantitative results are lost in much vague and imprecise language in the Thumbnail and Executive Summaries (arguably the most important sections). The precision and detail of the descriptions of the quantitative results in these Summaries require considerable attention.

      A good example of clear and quantitative presentation of results is seen on page 4, paragraphs 2-3, e.g., that includes specific percentiles of consumption, gains in IQ points, and oz's per week of fish.

      A converse example is seen on page 4, paragraph 4, in which vague and imprecise wording –"relatively high," "could," "reduce the benefits," "unusual cases," and "a fraction" – leaves interpretation to any reader's imagination of what these words mean. This imprecision should be replaced with quantitative language: i.e., at what levels of mercury in fish and what levels of consumption such effects could be seen; the CI or some other measure of the word "could;" by what extent, quantitatively, the benefits could be reduced (which should also be re-emphasized as being very different from net harm); what percentile of the population is likely represented by "unusual;" what precise fraction of an IQ point could be lost; etc. Development of clear and quantitative answers to these questions is obviously challenging, but this indeed is the charge of this Report, and this single paragraph in many ways is the core of the neurodevelopment conclusions.

      Similar vague and confusing imprecisions are seen in the summary of the effects of the models (page 19, page 21). The description of the Combination Net Effect model is unclear and very difficult to follow in these instances. For each of the major predicted effects, the description of the findings should be explicitly quantitative, and a summary Table or preferably Figure should be included. Page 21, Combination Net Effect model, what is the likely quantitative benefit? To what metric does "four percent probability" refer? Who are "some people," and how large is this group? How big is the "larger net benefit?" etc.

      Another important example: Page 20, paragraph 3, last sentence. "In this case…" How plausible is this scenario? What proportion of commercial fish, if any, contain high MeHg and low nutrients? Do what proportion of fish consumers does it apply? How likely, quantitatively, is a "small adverse effect," and what does "small" mean? Again, sentences like these are the heart of the Report, and should be clear and quantitative.

      Similar quantitative details should replace the indistinct wording describing findings for CVD (page 5, paragraphs 2-3; page 26, paragraph 2). For example, page 5, paragraph 3, states "a small possibility of increased risk." Several points should be made clear. First, two estimates were performed, and the similarity of the central estimates should be emphasized (and the explicit numbers of prevented CHD deaths included). Second, only one estimate had a CI that included zero, and the percentile at which this was seen should be specified (e.g., was it at the 92% upper limit?). Also, if the Summary mentions results for the outer bound in one direction, it should include the results for the outer bound in the other direction. Thus, rather than the vague statement "a small possibility of increased risk," a clear and quantitative statement should be provided, e.g., "Using two techniques, both central estimates predicted ~35,000-40,000 fewer CHD deaths due to current fish consumption. The 95% CI's of one estimate only included benefits of ~5500 to 60,000 fewer deaths. The 95% CI's of the second estimate, which were much broader due to differences in assumptions relating to uncertainty, included an ~8% probability of no benefit and, at the upper limit, an ~5% probability of up to 50,000 greater CHD deaths, but also included at the other extreme the possibility of up to 250,000 fewer deaths due to current fish consumption." [Much of the confusion surrounding these impossibly broad CI's will be mitigated by re-assessment of the fish and CHD effects using more reasonable, appropriate, and comparable assumptions – see comment 5bA].

      The above are only examples – the Thumbnail and Executive Summaries (and the body of the report) should be carefully reviewed to minimize imprecise statements and maximize quantitative conclusions.

    3. (C) Compared with the details presented for neurodevelopment in the Thumbnail and Executive Summaries, the content for CVD is strikingly cursory. The two paragraphs on page 5 should be expanded significantly to include quantitative assessments of levels of intake, numbers of deaths prevented and CI's, etc. Also, vague and imprecise language should be removed (e.g., "most likely," "small possibility," "increased risk," etc.) and replaced with quantitative estimates. Similarly, 14 pages in the Executive Summary describe neurodevelopment results; only 3.5 pages are given to CVD results. Indeed, as described below (see 2A), based on emphasis in the report (e.g., page content alone), one would be led to believe that the best evidence comes from small studies rather than large prospective cohorts and RCTs of disease outcomes.

      The quantitative results for the "what if" scenarios for CHD should also be included in both Summaries.

      The body of the report also gives short thrift to CVD. Following Section II-A's 30 pages of careful discussion and evaluation of neurodevelopment literature, including substantial (and appropriate) attention to studies of benefits of fish consumption, Section II-B gives 8 pages to the MeHg papers (a remarkably porous and limited set of data, if judged using an objective set of hierarchies of evidence quality [see 2A, below]), and then nothing at all on the numerous large prospective cohort studies of CHD death totaling >300,000 individuals, the many biomarker studies of CHD death, the many large prospective cohort studies of nonfatal CHD or total CHD outcomes, the many large prospective cohort studies of stroke, and the several large RCTs (>35,000 subjects) of fish or fish oil for both primary and secondary prevention of CHD. Placing the full account of this embarrassingly rich data in the Appendix is okay, but at least several pages of summary of this rich data must be added to Section II-B, explicitly detailing the relative sizes, strengths, and robustness of these studies compared with the MeHg and CHD data (and even compared with all the neurodevelopment data).

  2. Were scientific assumptions explained and are they appropriate? 

    Generally yes, with two important exceptions:

    1. (A) The assumptions (if any) regarding hierarchies of evidence quality were not specified. The reader is left with a general feeling that RCTs are preferred to observational studies, but no other guidance is given, as if all observational studies (or even all RCTs) were equivalent.

      For example, factors affecting evidence quality include study design (RCT of disease outcomes > prospective cohort of disease outcomes >> RCT of intermediate risk factor >> retrospective case-control study of disease outcomes >>> observational study of intermediate risk factor >>> case-series); power (driven largely by number of events); accuracy of assessment of exposure, outcome, and covariates; avoidance of multiple hypothesis testing (often too frequent in neurodevelopment studies); control for confounding; statistical power; generalizability; etc. Some of these issues are occasionally mentioned for specific studies, but a clear description of hierarchies of evidence quality is needed.

      Based on emphasis in the report (e.g., page content alone), one would be led to believe that the best evidence comes from small retrospective case-control studies and poorly controlled case series; then somewhat less robust evidence from modestly sized retrospective case-control studies and small prospective cohorts; then somewhat less robust evidence from short-term randomized controlled trials of intermediate phenotypes, and then the least robust evidence from large prospective cohort studies and randomized trials of disease outcomes. Clearly, this order should be exactly reversed, together with reversal of the report's greater attention to relatively small studies with many limitations vs. little attention to large studies with greater strengths.

      A good place to formally address the issues of hierarchies of evidence is page 12, paragraph 3, by adding 2-3 paragraphs of explicit details for strengths and limitations of different study designs, especially within the category of observational studies. A summary figure or table of evidence hierarchies would be particularly useful. In the detailed descriptions and interpretations of individual studies in the body of the report, the panel appears to have taken some of these issues into consideration, but the Report lacks an explicit summary of criteria for how these different studies were judged. Of particular relevance are (a) the RCTs of DHA for neurodevelopment (largely postnatal, but some prenatal), that demonstrate biologic benefits of increasing DHA above current background intakes; (b) the RCTs of fish or fish oil for CHD, performed in both primary prevention populations (JELIS) and patients with established heart disease (DART, DART2, GISSI-P, GISSI-HF), that demonstrate reductions in CHD for increasing EPA+DHA above background intakes; (c) meta-analyses of fish oil RCTs for total mortality, arguably the most important outcome; and (d) the numerous large prospective cohort studies of fish intake and CHD death. Indeed, the RCTs of fish oil (both for neurodevelopment and CHD) are relatively underemphasized in the report (certainly in the Summaries), whereas they provide some of the strongest evidence and should be especially emphasized. The idea that RCTS of fish oil somehow do not inform us about effects of fish consumption is disingenuous – the corollary for such a fallacious argument would be that the Iraq poisoning tells us nothing about health effects of MeHg in fish simply because the exposure was not fish.

    2. (B) There is a strong but apparently unrecognized disconnect in the report between assumptions for consideration of MeHg (assessed by biomarkers) vs. fish consumption (in this report, assessed only by questionnaires), and the direct relevance of these differences for the accuracy of the information, clarity of presentation, and soundness of conclusions.

      One of the recurrent tenets of the Report is that nutrient-specific benefits of fish could not be assessed. This is incorrect. First, many RCTs have addressed effects of EPA+DHA for both neurodevelopment and CHD, with results largely quite congruent with observational studies of fish consumption. That these RCTs often tested a variation of the hypothesis compared with the observational studies should be highlighted as a strength, not a limitation, of the overall evidence. For example, neurodevelopment RCTs of DHA were often (though not exclusively) postnatal, whereas observational studies of fish consumption were often prenatal; and CHD RCTs of fish or fish oil were often (though not exclusively) in patients with established heart disease, whereas observational studies of fish consumption were often in patients without known CHD. The correspondence of the results of these related but not identical hypotheses tested is powerful. As described above, it is disingenuous to state that the similar results of several large prospective cohort studies of maternal (prenatal) fish consumption and neurodevelopment in many countries and several RCTs of DHA supplementation (mostly postnatal, one prenatal) and neurodevelopment are not strongly co-confirmatory simply because one group of studies were observational and assessed prenatal fish while the other group were RCTs and assessed postnatal DHA – rather, the similar results using different populations, study designs, and exposures (fish vs. fish oil) provides even stronger confirmatory evidence than if the studies were more similar. It is also disingenuous to state that the similar results of many large prospective cohort studies of fish consumption and CHD death in many countries totaling >300,000 individuals and now several RCTs of fish or fish oil and CHD death in many countries totaling >35,000 individuals are not strongly co-confirmatory simply because one group of studies were assessing fish intake in individuals without known disease and the other largely (but not exclusively) fish oil intake in individuals with known disease – rather, the similar results using different populations, study designs, and exposures (fish vs. fish oil) provides even stronger confirmatory evidence than if the studies were more similar. Based on these large bodies of evidence, EPA+DHA are clearly major active ingredients for both neurodevelopmental and CHD benefits of fish intake. This does not exclude the possibility that other factors in fish, particularly selenium and vitamin D, may also have benefits, but EPA+DHA are clearly the major players.

      Second, the Report explicitly uses a biomarker of dietary exposure for one constituent in fish, i.e., MeHg levels, to derive quantitative models and conclusions about the specific effects of this factor on health. This is done even though other factors are present in fish; it is possible that effects of this constituent are confounded by other constituents (both beneficial and harmful); and the levels may not perfectly reflect diet due to metabolism and changes in exposure over time. Indeed, this type of assessment is the rule rather than the exception in toxicologic analyses.

      Given this treatment of MeHg, there is no justifiable reason for not performing comparable and objective analyses for a different biomarker of dietary exposure to another important constituent in fish, i.e., long-chain omega-3 (EPA+DHA) levels. In all respects, use of tissue levels of MeHg as a biomarker for assessing health effects is entirely analogous to use of tissue levels of EPA+DHA as a biomarker for assessing health effects.

      Paucity of data is certainly not the limitation; the extent of observational literature on tissue EPA+DHA and CHD is at least comparable (if not superior) to that for MeHg and neurodevelopment, and certainly far exceeds the literature for MeHg and CHD. Arguments for inability to extricate effects of EPA+DHA vs. other nutrients cannot be made. First, if so, identical arguments could be made for MeHg, that is certainly present with other toxins and nutrients yet is evaluated for its independent effects. Second, EPA+DHA is clearly a major active nutrient for both neurodevelopmental and CHD benefits, with RCT data supporting, in consistent doses, directions, and magnitudes, the observational benefits of EPA+DHA for both neurodevelopment and CHD benefits (whereas no such trial data exist for MeHg – the closest corollary is the accidental MeHg exposures that are clearly limited in dose-comparability).

      Thus, the Report's assumptions in assessing effects of MeHg (evaluated as a biomarker, concluded to be causal based on observational studies only) are radically different from those for EPA+DHA (evaluated only indirectly as fish intake in dietary questionnaires, always discussed with major caveats as to causality given the observational nature of the evidence). This cannot be justified on any biologic, epidemiologic, or scientific grounds – the only reasons for these discrepancies are the historical traditions of toxicology assessments vs. nutritional assessments.

      To provide an objective and comparable assessment of benefits and risks, and to communicate these appropriately, several issues must be addressed. First, the biomarker studies of health effects of EPA+DHA (many for CHD, a few for neurodevelopment) should be discussed, including in the Summaries. Second, the difference in exposure (mis)classification between biomarkers and estimated dietary intake, and the implications of these differences, should be considered and discussed. Exposure misclassification that is largely random (e.g., in prospective studies, although not always so in retrospective studies) can strongly bias results toward the null, and such misclassification is much larger for estimated dietary intake than for measured tissue biomarker levels. Consequently, estimates of health benefits based on estimated fish consumption will be much more biased toward the null, compared with estimates of potential harms based on tissue MeHg levels. This bias toward the null can be quantified by comparing studies of estimated fish intake and sudden death vs. tissue EPA+DHA and sudden death: in the former, the RR estimates indicate a 3-fold (~33%) lowering of risk, whereas in the latter, the RR estimates indicate a 10-fold (90%) lowering of risk. Consideration of these critical issues, and clear description of them, would lead to appropriately stronger conclusions about benefits vs. risks.

      Finally, a quantitative assessment, or review of prior assessments, of quantitative benefits of EPA+DHA should be included: this should be done for both neurodevelopment and CHD, that each have both observational and RCT evidence on effects of EPA+DHA. It is understandable (although unfortunate) that the panel may not wish to carry out such new quantitative assessments for this Report, that would require substantial additional time and work. However, in this case, this omission should be made explicit and emphasized, not because "we were not able to assess these" (page 5) or "we lack the data necessary" (page 144), but because "we chose not to do so, even though such analyses are possible, scientifically justified, and have been done by other groups."

  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included? 

    Generally an excellent job of selection and inclusion of papers. Two RCTS should be added: JELIS, a large RCT demonstrating benefit of EPA for CHD in a predominantly primary prevention population, GISSI-HF, a large RCT demonstrating benefit of EPA+DHA for total mortality in a patients with established heart disease.

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    Yes, with the exception of the limitations detailed above. Attention to these limitations would improve the appropriateness and clarity of the conclusions about relative risks and benefits, as well as relative strengths of evidence for these different effects.

  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?

      Somewhat. Important limitations that require attention:

      1. (A) As noted above, a formal description of hierarchies of evidence and criteria for judging quality is needed. For example, what are the criteria for comparing strength of evidence for a small cohort study such as in the Faroes, with exposure largely from mammalian blubber, strong confounding by parental socioeconomic status and lifestyle (rural and traditional vs. urban and educated), multiple hypothesis testing, and unclear generalizability to Western populations vs. large prospective cohort studies with exposure of fish consumption, a few carefully chosen outcomes, careful control for confounding, and done in Western populations, such as in ALSPAC and Oken et al.?
      2. (B) RCT data on fish or fish oil consumption and neurodevelopment or CHD risk should be added to the quantitative risk assessment. If not added, this omission should be made explicit and emphasized, as noted elsewhere, and the tremendous qualitative strength of evidence afforded by these studies made clear in the qualitative Summaries.
      3. (C) The methods state that CHD effects were to be investigated by considering CHD death, but the dated (2002) Hooper meta-analysis is included, that focused largely on combined (fatal and nonfatal) CHD events, that are unlikely to be affected comparably (see 5bC below). Further, the Hooper report excluded most observational studies of fish consumption and CHD death (the focus of this Report). Thus, the Hooper report largely fails to meet most criteria, and its major limitations should be further emphasized.
    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions?
      1. (A) Comparability of risk assessment methods is essential to allow valid results. The lack of comparability of methods for determining dose-responses of MeHg and neurodevelopment endpoints, fish and neurodevelopment endpoints, and CHD and neurodevelopment endpoints is striking. For MeHg and neurodevelopment, milestones at two years were evaluated using pooled data from the Iraqi poisoning (not from fish!) and Seychelles; IQ at seven years using a Bayesian analysis of three prospective studies; IQ at seven years based on a pooled analysis of a few studies; and Bayley scales at 12 months from a single small study. For fish and neurodevelopment, effects were estimated from one single study, although many other large studies exist. For fish and CHD, effects were estimated by fitting data from individual studies rather than pooling, with additional sampling error estimated by not assuming a common variance, and with additional uncertainty introduced by not modeling a common effect across studies. This approach is clearly very different from those used for MeHg and neurodevelopment and fish and neurodevelopment! The end result is that, paradoxically, endpoints with relatively little data (MeHg and neurodevelopment, fish and neurodevelopment) have relatively small uncertainty, whereas endpoints with enormous amounts of data (fish and CHD) have spuriously large (and incomparably calculated) uncertainty.

        The Report must either re-model the effects for MeHg and neurodevelopment and fish and neurodevelopment using similar methods as for fish and CHD (i.e., using all available estimates from all reports, fitting data from individual studies rather than pooling, adding sampling error by not assuming a common variance, and adding further tremendous uncertainty by not modeling a common effect across studies), or re-model the fish and CHD effects using more similar methods as for MeHg and neurodevelopment and fish and neurodevelopment.

        The Carrington model for CHD death gives outrageously large CI's for an effect based on such enormous amounts of data, compared with the modeling and CIs for MeHg and neurodevelopment and fish and neurodevelopment that are based, in this Report, on such relatively little data. The resulting uncertainties in the Carrington CHD estimate are so large – ranging from -208,000 to +29,000 CHD deaths in men 46+ due to current fish consumption – as to render the estimate nearly meaningless from a practical standpoint. In particular, as modeled, the assumption regarding an independent (different) effect in each individual study of fish and CHD is unwarranted, and such a drastic assumption is not made for MeHg and neurodevelopment or fish and neurodevelopment. Removal of this drastic assumption would provide much more valid estimates for fish and CHD and importantly, render the methods for this estimate at least partly more similar to methods used for other endpoints.

        Finally, data from biomarker studies of EPA+DHA should also be included in the modeling of fish and CHD effects – the data for MeHg is entirely, after all, from biomarkers. (Data from RCTs of fish and CHD could also be included, although this is less essential.)

        Similarly, for fish and neurodevelopment, the additional data from other studies should be included in the quantitative assessment, at least in sensitivity analyses. The absence of availability of individual-level data does not mean that this other data should be ignored – sensitivity analyses should be done to include and account for these several other studies (e.g., using pooling or other similar methods as were used to combine the studies of MeHg and neurodevelopment).

      2. (B) The much greater misclassification of estimated fish intake compared with tissue MeHg should be addressed, that will lead (at least when random as in prospective studies) to substantial underestimation of health benefits of the former.
      3. (C) The assumption that benefits of fish consumption for nonfatal CHD events will be proportional to benefits for CHD death is not justified. The great body of literature – observational, RCT, and experimental – indicates that fish or fish oil reduces nonfatal CHD to a meaningfully smaller extent than fatal CHD. The effect seen in JELIS – approx. 19% reduction – is consistent with modest (10-20%) benefits seen in some (though not all) observational studies. Thus, current evidence suggests that benefits for nonfatal CHD are likely to be approximately one-half to one-third the magnitude of benefits for fatal CHD. (Of note, this also indicates that studies evaluating combined CVD events, such as the Hooper report, will see smaller effects).
    3. (5c) Are scientific assumptions explained and are they appropriate? 

      See comments elsewhere.

  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used? 

    No – these endpoints are the only ones for which sufficient evidence exists to make such calculations.

  7. Are the intervention scenarios appropriate? 

    Yes, they are reasonable.

III. SPECIFIC OBSERVATIONS

Page 4, paragraph 4: The concept "reduce the benefits" should be emphasized as being quite different than "net harm."

Page 11: A nice summary of safety assessment issues for MeHg. Discussion here is crucial regarding the traditional lack of "safety" assessments for low consumption of fish, that could be as least as harmful (and likely more, based on the conclusions of this Report and other studies) for both neurodevelopment and CHD as MeHg. Toxicologists often communicate in terms of safety and risks, whereas nutritionists often communicate in terms of benefits – these are of course identical concepts, requiring only a change of the reference group. If the reference group were made to be adequate fish consumption, what are the safety limits and risks for the population of having lower consumption?

Page 13, paragraph 4: "The researchers anticipated that effects would appear as subtle differences…" The subtlety of the neurodevelopment effects in question, throughout this Report, should be expanded and emphasized further here and elsewhere.

Page 21, last paragraph: As differences in risk of adverse effects above and below 12 oz/week are not large, and consumption >12 oz/week includes a possibility of larger net benefit, what the do the final conclusions indicate about the utility of the 12 oz/week limit? This is also at the heart of the purpose of the Report. The data appear to suggest that, at the current state of knowledge, the 12 oz/week limit should be discarded and that women should eat a variety of fish and (since commercial fish are at the low end of the spectrum on average) more so than 12 oz/week. An explicit conclusion should be drawn here about this.

Page 22: Summary of RCTs of DHA should be included here.

Page 35: It is a shame that "the primary practical function of the consumer advisory is to encourage 4.65% of women to increase the margin of safety for their fetuses." The primary practical function should be to improve the net health of U.S. women and neurodevelopment of their babies.

Page 37: Title of this Section should be "Scientific Basis For Risk and Benefit Assessment." Also, this section did not review "studies that looked for associations between the consumption of fish and health endpoints." Many health endpoints were not evaluated, including fish and cognitive decline, fish and atrial fibrillation, fish and depression, etc.

Page 40: Two other major categories of studies should be added and reviewed:

  1. 3. Studies that evaluated biomarkers of omega-3 consumption from fish (EPA+DHA) and CHD or stroke. (No justification to exclude – biomarker studies of MeHg are evaluated.)
  2. 4. RCTs that evaluated fish or omega-3 consumption and CHD or stroke.

Page 73: The much higher levels of MeHg and the absence of selenium in the Finland study, compared with the U.S., should also be discussed in the Executive Summary.

page 74: The Kuopio authors' own conclusions that mercury in fish "could attenuate the protective effects" of fish consumption, and the very important difference between (a) attenuation of protection and (b) net harm, should be included and emphasized in the Executive Summary. (This is also the case for neurodevelopment outcomes).

Page 75: Guallar et al. The retrospective case-control design (subject to strong selection bias) and the inclusion of only nonfatal CHD (not fatal CHD, the focus of this report that is much more strongly benefited by fish consumption than nonfatal CHD) should be emphasized. This design is very weak to investigate the effects of exposure on fatal CHD, given the strong possibility of selection bias of controls in retrospective studies and the inclusion of only survivors. These issues should also be discussed in detail when defining the general "hierarchies of evidence" that form the background for this report (see comments in 2A). Results for DHA in this study – showing net benefit with higher consumption, and only an attenuation of benefit, but not net harm, before adjusting for mercury – should be included.

Page 77: Limited power to detect associations is not a strong argument here, as the number of cases in this prospective nested case-control study was much larger than in the Finland study.

Page 139: "The collective size of these studies… compares favorably [emphasis added] to the… studies that measured mercury exposure." The understatement of the Report!! The numbers should be included here: the numerous large prospective cohort studies of CHD death totaling >300,000 individuals, the many biomarker studies of CHD death, the many large prospective cohort studies of nonfatal CHD or total CHD outcomes, the many large prospective cohort studies of stroke, and the several large RCTs (>35,000 subjects) of fish or fish oil for both primary and secondary prevention of CHD – probably in total nearly 1,000,000 individuals studied.

Page 144, 147: Table IV-14, fourth row heading, and Table IV-16, each row heading, clarify that these estimates are for current fish consumed, e.g., "Median change in CHD death rate due to current fish consumption."

Page 163: Cannot make similar assumptions for nonfatal CHD, as effects are likely smaller – see comments elsewhere. No evidence, however, that fish consumption has different proportional effects for fatal vs. nonfatal stroke.

Page 241: Line 11: "Interestingly," should be replaced with "As would be expected." "Substitution" arguments hold little water in assessing potential mechanisms for benefits of fish consumption. First, the average amounts of "harmful" macronutrients (saturated fat, trans fat, and dietary cholesterol) in, for example, meat that would be replaced by 8 oz per week of fish is remarkably small and would have little impact on overall daily dietary intakes of these dietary factors. Second, for "substitution" effects to have any meaningful mathematical impact on risk, predominantly one type of food or foods would have to be replaced nearly uniformly by the entire population consuming fish, not only in the U.S., but in all countries in which similar effects of fish consumption have been seen. The plausibility of this is very close to zero. Third, the similar effects of small amounts of fish oil (1 g/d = 9 calories/day!) on CHD in RCTs, compared with observational studies, indicates that the benefit is due to what is present in fish, not what is not present. A discussion of these issues, including in the Executive Summary, would be helpful to put this "substitution" notion to rest.

PEER REVIEWER NUMBER 6:

I. GENERAL IMPRESSIONS

Overall, the report presents a very thorough analysis of the risks and benefits of fish consumption in the U.S., and I commend the authors on this important work. The report provides both updated reviews of critical literature and helpful links back to prior risk assessments on this topic that I found very helpful in sifting through the weight of the evidence. I particularly appreciated that the authors were very careful to highlight critical data, including both old and new studies, and that they made their assumptions very explicit throughout the document. The information appears accurate, although I did not investigate the underlying models exhaustively. I had access to some of the spreadsheets used to support the analyses in the report and I found no errors in my review of them, but these were not the primary focus of my review. If they are released publicly, the spreadsheets would benefit from much better documentation for purposes of review, particularly given the relative complexity of two-dimensional Monte Carlo analyses.

With respect to the report, I think that it would benefit from substantial editing and that this will significantly improve the clarity of the presentation. The report reads as if it was written by multiple authors, with writing in passive third-person in some places and active first-person in others. (I strongly encourage use of active first-person throughout, which provides a much friendlier and accessible tone to the reader). I found it striking that the main body of the report contains almost no figures. Although the report includes many tables, I found numerous places in which I would have personally preferred to see a figure. For example, many of the data and results are presented as percentiles in tables, but I would prefer to see the cumulative distribution functions presented with the means clearly noted on them (and then being able to look up the actual numbers associated with various percentiles in an appendix). I appreciated very much that the authors used appendices, which kept the text in the main chapters contained. However, I very much missed having a roadmap to what I would find and where I would find it within the document (beyond looking at the list in the Table of Contents). I would suggest changing the title of the "Thumbnail Summary" to "Abstract" and adding a section at the end of the "Abstract" or "Executive Summary" that is a "Roadmap" for the report (brief descriptions of the subsequent sections that would then lead to the Table of Contents and then the Executive Summary). I am not clear about the anticipated audience for the Executive Summary, but I would suggest that while the content of the Executive Summary is very good, the writing itself probably needs some work with respect to writing it for the appropriate audience. In the attached I suggested heavy edits to the "Thumbnail Summary" and the beginning of the "Executive Summary," but I suggest that a professional editor be hired to edit the entire document.

Overall, the conclusions of the report appear sound. I commend the authors for conducting a rigorous analysis and I found the versions of the model that they developed (the "Carrington" models for CHD and stroke) to be very interesting and persuasive with respect to the robustness of the results. If anything, the authors appear to be understating the significance of their findings. The message that increasing fish consumption by 50% would decrease risks of CHD and stroke and that in general most Americans need to eat more fish to improve their health could easily be lost, but this is the most critical insight and message from all of the analysis in my opinion. The fact that women of childbearing age and their children would most likely benefit from eating more fish low in MeHg is also very important, particularly given the common perception created at least in part by the prior EPA/FDA advisory that pregnant mothers who eat fish are endangering their babies. Similar to the prior advisory, the message from this report is nuanced, but it is clear that eating fish provides significant health benefits to nearly all (if not all) members of the population, with fish low in MeHg and high in n-3 fatty acids appearing to offer the greatest benefits. I think that the conclusions are correct, but as noted above they may not be clear enough for all audiences.

II. RESPONSE TO CHARGE QUESTIONS
  1. Is the document logical and clear? 

    The report is generally well-written, but the organization could use some improvement and it would help if the report specified the target audience(s) (at least making clear who should understand the Abstract, Executive Summary, parts of the main report, and appendices). As noted above, I believe that the document would be significantly improved if it had a roadmap at the beginning that made clear what the reader should expect in each section. Some sections also feel repetitive to some degree (e.g., going through the analysis for CHD and then similarly for stroke), but this is a necessity. However, I think that the authors should do as much as they can to make it feel less repetitive. I also think it might make sense to reorganize the report to put the CHD and stroke results before the fetal neurodevelopment results since readers might find it easier to understand these. This would mean that large sections of the report and subsections within would need to be reorganized, but I think it would help the report to better emphasize the results that are relevant for most US consumers.

    I think the text needs a lot of tightening up and that the authors should make the report more engaging by adding some figures and graphics that can supplement or replace some text or tables in places (see my suggestions in the track changes version). I also think that the authors will run into challenges with some of the concepts that they use. As a scientist with some training in statistics I found their use of the Z-scores helpful, but I do not think that this will work with a broader audience. I hate to say it, but I think that they should keep the Z-scores in the background and technical parts of the document and convert all of the results to the natural interpretation (e.g., increased by X%). In most (if not all) places that mention the Z-score the interpretation is also provided, so I am mainly suggesting that the Z-score aspects be relegated to an appendix or reported in parentheses instead of as the main result.

  2. Were scientific assumptions explained and are they appropriate? 

    I was very impressed by the thorough explanations that the authors provided for their assumptions throughout the document. The authors took the time to review the critical evidence, and to explain why they made particular choices in their modeling (e.g., in the descriptions of the "Carrington" CHD and stroke model choices). I did not find any assumptions that seemed inadequately justified or inappropriate, and in fact, I commend the authors for their resourcefulness in obtaining, analyzing, and combining data using rigorous statistical models. The authors also carefully specified appropriate input distributions for their models with consideration of both uncertainty and variability.

  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included? 

    The authors have done an excellent job summarizing the existing literature. Given that studies continue to come out related to this study topic, it is not surprising that the report might be missing some recent relevant papers (e.g., Oken et al., "Associations of maternal fish intake during pregnancy and breastfeeding duration with attainment of developmental milestones in early childhood: A study from the Danish National Birth Cohort." American Journal of Clinical Nutrition, 2008; 88(3):789-796).

    Another paper that I think would be useful to include is: Hicks, Doris, Pivarnik, L, McDermott, R. "Consumer perceptions about seafood - an Internet survey." Journal of Foodservice 2008; 19(4):213-226. (DOI: 10.1111/j.1748-0159.2008.00107.x), which surveyed U.S. consumers about "seafood consumption frequency, sources of information about seafood and preferred formats, knowledge of key seafood issues, and barriers to seafood consumption," and "if they had heard positive or negative information about seafood and where they heard this information." I believe that this study and prior literature that it cites might provide some additional useful context about seafood consumption (and therefore exposure), consumer perceptions of risks and benefits, and the potential impacts of advisories.

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    The conclusions are appropriate and they follow very logically from the quantitative risk assessment. With the multiple what-if scenarios that the authors considered and their careful and thorough discussion of all of the assumptions, I think that the main challenge that readers will face will be sorting through the numerous results to get the major messages.

  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?

      The report provides sufficient information and explanations throughout. I was particularly impressed with the detailed discussion of the specific inclusion criteria for the data used, and that the authors developed their own models (i.e., the "Carrington" versions) based on their best judgment. I believe that the methodology they used was appropriate and that it represents the state-of-the-art in the field. The other data that the authors might include (mentioned above) relates to analysis of consumers' perceptions of risks from eating seafood and how these might impact food choices and exposure. I think that this might be a useful addition because it may suggest that one message from the report should directly address the impact of the prior EPA/FDA advisory on health.

      One issue that the authors need to make sure that they check carefully throughout the report is that they always refer to methylmercury when that is what they mean. In some cases they just say mercury, and this leaves the reader wondering if they mean total mercury or methylmercury. The authors also assume that methylmercury and organic mercury are completely synonymous, and while methylmercury most likely represents the main type of organic mercury, other forms are also possible. I think that the assumptions made are appropriate, but the existence of other forms should probably be mentioned somewhere.

    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions?

      The uncertainty analysis represents a state-of-the-art analysis for which I commend the authors. I believe that the uncertainty analysis adds a large degree of confidence that the main insights are robust, in spite of wide ranges for some of the confidence intervals. The ranges presented encompass the range of possibilities, but more importantly they provide insights about the likely probabilities. The authors are cautious to note that their main conclusions apply to most people, but may not apply to all, and they provide analyses that explain why prior studies provided inadequate information. For example, I found the report's description of the most recent Faroe Island results that separate the fish and marine mammal (pilot whale) data particularly helpful in explaining what previously has been puzzling to me with respect to why the Faroe Island and Seychelles Island studies appeared to yield different results.

    3. (5c) Are scientific assumptions explained and are they appropriate? 

      The scientific assumptions are explained in detail and they are appropriate. The appendices provide a wealth of useful details about what the authors did and they provide the support needed to understand the science underlying the assumptions.

  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used? 

    The authors have included the significant endpoints and I do not believe that they should be modeling any others. The report is very dense as it is, and I would not suggest that the key findings be diluted by discussion of other possible endpoints. The one main omission that the authors explicitly state is beyond the scope of their report relates to the substitution of non-fish foods that might pose larger health risks if people reduce their fish consumption and/or additional benefits that might result from consumers substituting fish for other foods in their diets that might lead to benefits beyond the CHD and stroke benefits. I believe that some studies exist that explore these potential risk-risk tradeoffs in some depth and that the authors should probably cite that literature and further discuss the challenges of characterizing any additional benefits that might exist.

  7. Are the intervention scenarios appropriate? 

    The intervention scenarios support a very useful set of analyses that appropriately represent the possibilities. I found it particularly helpful that the authors explored the impacts of both increases and decreases in fish consumption and discussed the uncertainty around the estimates. The what-if scenarios for fetal neurodevelopment results are more difficult to understand, even if one understands Z-scores, and they are also complicated by the different results obtained with the different models used (i.e., Combined Net Effect, Age-of-Talking, and Age-of-Walking). This is really the main reason for why I suggested above that it might make sense to put the CHD and stroke results before the fetal neurodevelopment results.

III. SPECIFIC OBSERVATIONS

A mark-up of the document with track changes has been provided to the FDA/CFSAN authors.

PEER REVIEWER NUMBER 7:

I. GENERAL IMPRESSIONS

This reviewer was very interested in reading this report and this reviewer applauds the authors for delving into the extensive literature on methylmercury and fish consumption to accomplish this analysis. The ideas and approaches for modeling are interesting and exciting. However, this reviewer has major areas of concern and the focus of this reviewer's extensive comments are present in several areas including: the choice of the data to model and implications of what was not modeled, interpretation of the modeling results and how this interpretation is placed in FDA's role in establishing allowable levels of MeHg in commercial fish. Many assumptions are made and these are clearly delineated in almost all cases, however many of the initial assumptions, in this reviewer's opinion, would severely limit the interpretation and application of the results. These limitations are not designated as clearly as needed and the recommendations on how to use the model findings imply that these would not limit application. In that regard, this reviewer's conclusions frequently differed from the authors. In regards to the question of whether major revisions are needed, yes, lots of clarification and discussion and suggested changes will be needed. This is because two of the key areas noted above on which this reviewer's comments will focus, are ones that have or could have significant impact on the final conclusions and extrapolation of FDA's modeling for risk assessment. This reviewer feels that these questions and requests for additional considerations are significant as the ideas proposed in this report would alter considerably how we currently are conducting risk assessments on this topic.

II. RESPONSE TO CHARGE QUESTIONS
  1. Is the document logical and clear?

    The document is logical and clear but very detailed. In order to fully understand the document it is this reviewer's opinion that the appendixes must also be read. Unfortunately this reviewer's comments are in most cases making suggested modifications and additions rather than providing ways to shorten the document. In fact, in several places, this reviewer suggested moving information from the footnotes back into the report. As mentioned in my summary it is when assumptions are made in order to accomplish modeling comparisons that this reviewer had most difficulties with the report as the limitations in the assumptions were not always carried forward to limit the conclusions or applications. It is in this regard that this reviewer feels that the report is not always "logical."

    Please also refer to the numerous specific comments below that specify where clarifications are needed.

  2. Were scientific assumptions explained and are they appropriate?

    Many scientific assumptions were made in the document and this reviewer feels that most assumptions were explained. In several very important places this reviewer feels that the assumptions made are not then used to qualify the results and at least one of these situations is with a major conclusion of the report. Please see the detailed comments below as well as reviewer comments for question (5c).

    Specific response number 49 also lists another example of assumptions that are made and which are not well analyzed and are in conflict with other (NAS) reviews.

  3. Has the appropriate literature been cited? Are there publicly available, peer-reviewed papers that should be included?

    The document does an excellent job of citing highly relevant and current literature. In only a few places did this reviewer have some additional suggestions. First, please refer to specific comments 46 and 47 below regarding fish consumers and non-consumers and appropriate literature. See also references that are listed as part of comment number 35 on regional and cultural differences in fish consumption and relevant reports. There is also a new, in press report evaluating national fish advisories that would be good to cite in this report. It is Scherer, et al. 2008, Environmental Health Perspectives 116: 1-9.

  4. Do the conclusions follow from both the analysis of the studies that are reviewed from the peer-reviewed literature and from the results of the quantitative risk assessment?

    Not always and these inconsistencies are the most significant of this reviewer's comments. In many cases the initial assumptions made in order to allow for modeling of the varied endpoints would result in significant qualifications of the model output. Please refer to specific review comments that are listed below. For example, in specific comment number 80 for page 159, second paragraph: Authors make important points about model estimates. Yet this reviewer was left wondering what to do with these "problems" in model estimates. This paragraph states that "For this same subpopulation, the "Carrington" stroke model's central estimate is that 127 stroke deaths are being averted for this subpopulation from fish consumption but at the fifth percentile confidence interval it also estimates that fish consumption could be averting up to 6,476 deaths." For this latter estimate to be correct, fish consumption would have to be averting over four times more stroke deaths than occur in this subpopulation on an annual basis. At the other end of the spectrum – the 95th percentile of the confidence interval – the "Carrington" stroke model predicts that fish consumption may be causing 434 stroke deaths, i.e., roughly one-third of all stroke deaths for women aged 16-45. Such an outcome seems implausible given the nature of the underlying data (see previous footnote). The results from the fifth and 95th percentile confidence intervals raise similar issues for other subpopulations." This reviewer was left wondering if such "implausible" observations at both ends of the dose response model outputs negate these models. Authors need to provide more in terms of what these observations mean for their overall risk assessment.

  5. Specifically in regards to the quantitative risk assessment:
    1. (5a) Were sufficient information and explanations given that describe how the data were identified and what criteria were used to determine the suitability of the data? Were these criteria adequate? Was the methodology appropriate? If there are other data that should be included in the quantitative analysis, how should it be used?

      No, not always. For example, this reviewer asks the FDA modelers to re-examine how they define consumption rates in their models. Please see specific comments number 31 through 37. Also this reviewer does not feel that variability in cultural and regional differences in fish consumption was adequately modeled. There are just two examples of numerous examples detailed in (5c).

    2. (5b) Is the model clearly described and is it supported by existing data? Are uncertainties in the model identified and characterized? In particular, does the uncertainty analysis encompass the range of plausible assumptions?

      No, again, not always. For example in specific comment number 54, very important neurodevelopmental studies were excluded from the model. See also reviewer response number 65 for CHD.

    3. (5c) Are scientific assumptions explained and are they appropriate?

      No, not always. An example of where inappropriate assumptions were made in order to facilitate modeling and which was done with minimal analysis following is seen in the example given for specific comment number 54 regarding what was modeled and what studies (in particular studies with positive findings) were not modeled. This is only one example but an important one which has important implications for neurodevelopmental endpoints. Another example is found in specific response number 78 where this reviewer is asking for explanations for the biological assumptions in the CHD versus fatal stroke models. Exactly what studies are relevant and which are not is an important decision that was not clear in the text or in the application.

  6. Are there additional endpoints of risks associated with methylmercury or of benefits of fish consumption that were not modeled and should have been? If so, what are they and what data should be used?

    Because of the limitations in the literature, the report correctly notes that some endpoints could not easily be modeled and when they were, they required assumptions to be made. Hence the answer to this question is yes, but the human literature is limited and cannot be used to fully answer many of the relevant endpoints. As I note in my specific comment number 54, these limitations resulted in "leaving out" some studies because they were not in a form that could be modeled in the current state of the models. In specific comment numbers 30 and 50, this reviewer asked whether the human data could be supplemented by available primate data. This would be the largest area of available research that was not modeled but which could still be relevant.

    My reviewer comments also document many limitations in modeling the benefits of fish consumption where fish intake was measured but that methyl mercury levels had to be calculated based on that intake and not on measured values. This highlights another area where additional data is needed but not everything is available in every study so the report does a heroic effort towards trying to model what is available.

    See specific comments numbers 44 and 45 that suggest that true separation of individual versus population modeling was not achieved. See also specific comments number 41 through 47 that discuss the need to ensure that modeling is done for consumers not consumers and non-consumers as an average of the population.

    In specific comment number 44, this reviewer also noted that peak exposures during critical time points during development should not be forgotten nor averaged over total developmental period without regard for modeling peak exposure which could occur during these critical time points of development.

    An important aspect of these risk questions from an overall public health view that was not modeled was the "big picture cumulative methyl mercury model." For example, the report largely dismissed the importance of modeling the 20 percent commercial fish consumption that the FDA regulatory jurisdiction would be concerned with within the broader context of potential contribution of methyl mercury exposure for the other 80 percent fish consumption that was non-commercial and not within FDA control. For many other pollutants, an approach is used that is based on a relative source contribution model where each source is modeled within the total contribution. In this case when recommendations are made for 12 ounces of fish per week for pregnant women and that these calculations are very close to a fine balance between benefits and risks without considering the variability and quantitative contribution of the other 80 percent, individuals making these decisions and public health officials outside of the FDA regulations find it very difficult to provide the "broader context" view. This lack of broader model was a serious deficiency in this report and limits its applicability for overall risk assessment applications. The lead IEUBK models have shown us how important these source contribution evaluations can be for overall risk evaluations. A better kinetics model for total methylmercury is needed to match the sophistication of the endpoint and benefits modeling.

  7. Are the intervention scenarios appropriate?

    This reviewer was a bit confused by this question and asks: Appropriate for what? The document specifically lists "Intervention Scenarios" from pages 224-230, however throughout the report various advisory scenarios are described and modeled in great detail. This reviewer will respond to the broader discussion. Yes, the advisory scenarios are ones that hinge upon the FDA advice for commercial fish and they are discussed in appropriate detail; however, the public and consumer is trying to answer a broader question. In fact, the consumer's question is: Is it safe to eat this fish? And how much can I eat? And they expect a synthesized response that deals with risks from multiple pollutants and toxins that may affect neurodevelopment and that may be present as a mixture in fish tissue. However, because of jurisdictional responsibilities, as risk assessors, we have to answer from our appropriate contexts. Thus, are the intervention scenarios appropriate, yes, looking at what happens above and below the 12 ounce guidance makes sense for this report. The larger question that this reviewer is struggling with is whether these are the appropriate ones for the consumer who needs to make informed decisions about eating fish or red meat, or soy or fish, etc. In these cases, the scenarios developed only for methylmercury and for only commercial sources are limited and do not match consumer needs.

III. SPECIFIC OBSERVATIONS
  1. Page 3. If this historical perspective is included in this summary, why not give a more quantitative perspective than "extreme exposures."
  2. Page 3, 3rd paragraph. Were there cases where the fetus was not more sensitive?
  3. Page 4, part (b). It is interesting that an additional scenario is not given.
  4. Page 4, 2nd paragraph. What about situations where other contaminants in fish affect the same neurodevelopmental endpoints as methylmercury?
  5. Page 5, after 1st paragraph. I would add a paragraph here that there is a need to consider other contaminants and the impact of their presence on these calculations.
  6. Page 10, 2nd paragraph, last sentence, editorial comment. Change use of "recent times."
  7. Page 12, editorial comment. Add footnote 3 into text as a significant ultimate focus of this report is about calculating such benefits.
  8. Page 13. Studies are listed here but no indication of size or power of detection can be determined for this section of the report, yet on the next two pages the size of population for those studies are given. Please be consistent. This reviewer would suggest adding population size for all.
  9. Page 15. Please clarify for reader any population overlap between the two Oken at al. studies. Add population size.
  10. Page 16, 1st paragraph. Please quantitate what is meant here by high methylmercury to fish ratio.
  11. Page 20, 2nd paragraph, last 3 lines. These lines list an important omission from these analyses…" In our model the most likely cause of an adverse effect would be a diet of fish that are relatively high in methylmercury. The model does not take into account how variations in amounts and types of nutrients in fish could mitigate a net adverse effect." What is the relationship of low nutrient fish and other contaminants such as PCB's? What about high nutrient fish?
  12. Page 22, 1st paragraph, 3rd line, editorial comment. Replace "neurotoxins" with "neurotoxicant."
  13. Page 22, 3rd paragraph, final summary. The third paragraph states that methylmercury levels in Children were not measured. Inclusion of this study in analysis and then saying that there is no evidence of an association between postnatal exposures to methylmercury is very speculative. This reviewer would modify concluding statements to match data. Isn't this also another case where foreign studies could be included? See this reviewer's later comments about not being consistent in including and excluding foreign data.
  14. Page 23, 2nd paragraph. This paragraph states that there are no reports of neurological effects in the mothers in either the Seychelles or Faroe Island studies. Please explain what if any neurological effects were examined in mothers in Seychelles or Faroe Islands studies. Modify statement if no research was done.
  15. Page 24, 2nd paragraph. Please add clarification and additional details on statement "The improvements varied depending on the category of fish or fish product that was most consumed." Also please explain why these findings were not modeled even if positive.
  16. Page 26, 2nd paragraph, last 3 lines. Please add quantitative details evenly through this page. For example, this line states "For now, the issue of methylmercury as a risk factor for some people in the United States cannot be dismissed, although the likelihood of adverse effects from methylmercury appears to be very low." How low is this?
  17. Page 27. Please quantitate these statements on each endpoint.
  18. Page 28, 2nd paragraph. Please explain what a "threshold question" is. This reviewer thinks authors may mean critical or key question.
  19. Page 29, 5th paragraph, line 5. Please be specific about "other components" here as this reviewer assumes this is not PCB's or other contaminants.
  20. Page 30, 4th paragraph. It is unclear how the wording of final conclusion and executive summary are consistent with this focus on only fish in interstate commerce. If FDA's approach in this document it to understand potential risks and benefits of fish consumption level of 12 ounces per day then the ways the conclusion and executive summary are worded is extremely confusing. Rather, authors should make an attempt at discussing total contribution of commercial fish to overall fish consumption as this would be required in order to make such broad statements as currently worded.
  21. Page 31, footnote 8. Provide reference for assessment of Molluscan shellfish and finding of lack of public health significance.
  22. Page 32, 2nd paragraph, 1st line, editorial comment. Change "neurotoxin" to "neurotoxicant."
  23. Page 35, line 2, editorial comment. Change "pf" to "of."
  24. Page 37, part (b), editorial comment. Replace "neurotoxin" with "neurotoxicant." Please change in each subsequent location of the document.
  25. Page 38 and 39. Incorporate footnote 19 and 20 into text on page 38 between 4th and 5th paragraph.
  26. Page 40, section II-A, 1st paragraph, lines 7-10. Specify what "threshold effects" are referred to in these lines.
  27. Page 43, end of chapter. It is very important to specify for each of the studies listed what the power of detection would be for the endpoints evaluated. For example, on page 43 the case study had only 247 individuals. How much power did it have to detect changes in the Denver Developmental Scale? The same question would be relevant for the other studies reviewed.
  28. Page 49, last 2 paragraphs. This reviewer found the comments on power of detection of the studies to be compelling and felt that the document's response is a bit "flippant." This section should possibly emphasize the 50% chance of seeing an effect. If subsequent modeling uses these insensitive studies please ensure that they are reviewed with this strong caveat.
  29. Page 52, 3rd paragraph. The second caveat raised in this paragraph is that the beneficial endpoint and toxicity mechanisms are raised and dismissed as not being relevant for the public health question at hand. This reviewer would be more cautious in dismissing this caveat. Yes, this doesn't matter in one part of the public health question but it does in another part - that considering the impact of other food components or contaminants. I would add conditional language at this point in the document that refers to additional discussion of this caveat for FDA's report (refer to section where later discussion occurs).
  30. Page 70, next to last paragraph and final footnote. Again tone between what this report says versus footnote in JECFA conclusion is evident. Authors should make statements more neural – use JECFA language… [JECFA] based on the view, expressed in 2006, that "the data do not allow firm conclusions to be drawn regarding [children's] sensitivity compared to that of adults. While it is clear that they are not more sensitive than the embryo or fetus, they may be more sensitive than adults because significant development of the brain continues in infancy and childhood" (FAO/WHO, 2006). As well, in absence of human data, one could look to animal study results that evaluate a broader developmental period. Could this help in interpretation of data?
  31. Page 82, 2nd paragraph. What about ethnic groups or Native Americans? Please add.
  32. Page 82, 3rd paragraph. Please provide more explanation about why these limitations do not significantly affect the data utility. Please reiterate purpose. Also please clarify that these individuals in Table 11-1 are only consumers. Obviously the inclusion of non consumers would weaken this premise.
  33. Page 83, 4th paragraph. It is important to do these back calculations.
  34. Page 84. This page provides some very interesting details about types of commercial fish and their mercury concentration. What is lacking is any regional sensitivity to these averages. Is tilefish eaten equally across the US? How large are regional differences? Could this change any of your calculations?
  35. Page 83-84 Additional references should be added in this section. For example, authors should review the literature for more recent data on variability of fish consumption rates in various populations residing in the US. The two following references have shown mean mercury exposure levels in U.S. women of childbearing age of Japanese descent living in Washington state are higher than the 90th percentile of mercury exposure among the general U.S. population estimated by NHANES (Tsuchiya et al., 2008). These observations were found to be true for Japanese but not Korean women living in the US. Please see Table 8 below.

    References to add include:

    1. Tsuchiya, A., T. A. Hinners, et al. (2008). "Mercury exposure from fish consumption within the Japanese and Korean communities" Journal of Toxicology and Environmental Health-Part a-Current Issues 71(15): 1019-1031.
    2. Tsuchiya, A., J. Hardy, et al. (2008). "Fish intake guidelines: incorporating n-3 fatty acid intake and contaminant exposure in the Korean and Japanese communities." American Journal of Clinical Nutrition 87(6): 1867-1875.
    Table 8 Comparison of estimated Hg intake and Hg-Hair levels between 1999-2000 NHANES study results and the Korean and Japanese cohorts.
    Parameter Group n GM Mean Percentiles
    10th 25th 50th 75th 90th 95th
    Estimated
    Hg intake
    (µg/kg/d)
    Japanese Cohort 106 0.09 0.14 0.02 0.05 0.09 0.18 0.25 0.37
    Korean Cohort 108 0.05 0.07 0.01 0.02 0.05 0.09 0.15 0.19
    NHANES 1999-2000a 1,727 0.02 NA NA NA NA 0 0.04 0.13
    Hair Hg (ppm) Japanese Cohort 106 1.23 1.57 0.5 0.78 1.37 1.96 2.68 3.52
    Korean Cohort 108 0.61 0.75 0.29 0.39 0.67 1.02 1.29 1.52
    NHANES 1999-2000b 17256 0.2 0.47 0.04 0.09 0.19 0.42 1.11 1.73

    a: Mahaffey et al.2004.
    b: McDowell et al. 2004

  36. Page 85, 86. The temporal trend data for mercury concentration in fish are of limited use in the current assessment as the authors explain. However, what this reviewer found to be of more potential interest and what was missing were changes in types and amount of fish consumed over time. For example has the amount of sword fish consumed by the US population decreased while sushi grade tuna increased? How much has consumption been affected by consumer advisories? Authors should include this in these comparisons.
  37. Page 89, last paragraph. Authors should also specify that this "what if" modeling also doesn't include the health effects of eating contaminants in fish other than methylmercury.
  38. Page 90, 1st paragraph. This reviewer was a bit surprised to read that "Because the fetus has not been directly studied for this endpoint, our modeling incorporates neurodevelopmental data from children who were prenatally exposed to methylmercury as a result of their mothers' exposures while pregnant." Please note the extensive data base from animals, including prenatal data, that could be used here for modeling.
  39. Page 91. This reviewer feels that major limitations of the modeling results from these omissions. Were the authors of these studies approached for such modeling? This would seem to be at the very heart of this modeling exercise.
  40. Page 93-94, exposure assumptions. Please specify that only consumers were evaluated in the risk assessments as it is essential that these models at this point do not include non-consumers. Explain how these numbers were obtained for consumers.
  41. Page 95, last paragraph. Authors should note that the survey discussed in this section addresses several of the concerns raised in the FDA document and comes to different conclusions than Paustenbach 2000. For example, note that the issue of seasonal differences is addressed in the survey hence authors of the FDA document are encouraged to cite and use the original discussion on these issues not the secondary literature.
  42. Page 95, last paragraph and page 96 first paragraph. This reviewer disagreed with the emphasis on the 30 day survey over the shorter surveys. Recall memory in other studies have been shown to drop off rapidly with longer duration from survey time. Given the authors modeling expertise it would not seem appropriate to test this choice to see if this would make a difference in the modeling conclusions.
  43. Page 96, footnote 37. This foot note says that changes in commercial fishery patterns are known and less methylmercury is consumed. Is this due to dietary advisories we have produced or reflective of true changes in fish consumption? Please reconcile with page 86 and 86 Section C.
  44. Page 95-99. Estimating Species and amounts of fish that people eat. This reviewer requires clarification in this important section. If the risk assessment questions in Box IV-1 page 89 are truly the risk assessment questions addressed in this report then this reviewer had difficulty understanding how individual fish consumption rates were obtained. This section appears to discuss population bases where both rates of consumers and non-consumers are determined as population averages. Please assure this reviewer that the modeling actually addressed individual risk. For example, what is the risk to the developing fetus if the mother consumes fish that day? The report refers several times to the importance of peak as well as average concentration of toxicants for making such estimates and not population averaged values between consumers and non-consumers is considered. Was this information really used in this modeling?

    This reviewer is greatly concerned by several points in the exposure calculations. First that average fish consumption estimates dominate the exposure scenario. As a person in public health we are interested in protecting fish consumers not an average of fish and non fish consumers. In developmental toxicity we assume that both peak exposures as well as area under the curve (integrated exposure) are important for outcome. If average exposure is estimated by averaging fish consumers and non consumers the dynamics of peak exposures on critically important life stage sensitive days during development is lost. Authors must address these concerns in their modeling context.

  45. Page 96-97. This section describes variation in species that people consume however the emphasis again appears to be on population based estimates versus individual estimates. Please discuss how both estimates were determined.
  46. Page 97, 1st paragraph, last points. This reviewer would question the use of the "repetition ratio" from the 30 day survey that is based on only surveying women of child bearing age to extend the "repetition ratio" to the entire population. Women of child bearing age would already be decreasing their intake especially of high mercury containing fish due to our repeated advisories to this effect. See Scherer et al 2007 that documents the number of fish advisories directed at women of child bearing age. Such an extrapolation as done here could under estimate the amount of mercury intake.
  47. Page 97-99, Amounts of fish that people consume. A newly published report by State of Oregon Department of Environmental Quality; Oregon Fish and Shellfish Consumption Rate Report, Human Health Focus Group, June 2008 discusses extensively the issue of calculating daily fish consumption averages for individuals who consume fish and not averaged fish consumption rates across populations of consumers and non-consumers. Although this review primarily focuses on high consumers of fish and focuses on fish and shellfish of regional origins (commercial and non-commercial), their philosophy of protecting consumers is an important one for these authors to consider (Oregon Department of Environmental Quality and USEPA).

    This reviewer asks that for the individual calculations that this view be examined and incorporated in a way so that the individual consumers are protected. Is FDA's regulation designed to protect consumers or only consumers averaged across consuming and non-consuming populations?

  48. Page 98, 3rd paragraph. Authors provide important points about differences in the percentage of women eating fish between the 2 surveys 34% in the 3 day survey versus 85% in the 30 day survey. The authors note that the 85% in the 30 day survey may underestimate the number of consumers for a large period yet no additional comments are made as to what actions or corrections should be made about this fact. This reviewer would suggest a sensitivity analysis on this point and on the earlier point on use of a population that is already actually reducing fish consumption. Both of these factors would again underestimate the frequency and types of fish that consumers truly consume (see page 97 comments).
  49. Page 103-104, Section C Fetal Neurodevelopment. This reviewer was very surprised to see the extraordinarily brief discussion of the choice of adverse neurodevelopmental endpoints to model that was presented in this section. This choice is extraordinarily important for the modeling outcome as described in great detail in the NAS report on Methylmercury (NCR, 2000) and it is also extraordinarily important to ensure that these same adversely affected endpoints are the same endpoints that would benefit from increased fish consumption. If this section is analyzed in detail very contradictory choices are made without sufficient robust discussion of other choices made by other groups. See especially comments by the NAS group regarding the increased importance of looking at outcomes at 5 to 7 years of age due to decreased ability to detect neurodevelopmental effects present at earlier ages (see pages 282-283, NAS, 2000). This is in direct contrast to statements in the FDA report, page 104, fourth paragraph, where such choices are made and two key areas of uncertainty are identified but not evaluated when assumptions are made. Why not use these other important endpoints in this sophisticated modeling effort by FDA or at a minimum use sensitivity analysis to analyze or set-up an integrated analysis such as was done in the NAS report (pages 289-293). These are important decisions and have significant implications for these analyses but also for directing future studies, i.e. implications beyond this report, but are also critical for the acceptance of this report.

    Of particular concern is the fact that the NAS, 2000 report specifically states after reviewing the neurobiology of the outcomes that "The committee concluded that it would be inappropriate to pick the Seychelles study as the basis for risk assessment, given the available evidence for positive effects in New Zealand and Faroe Islands studies as well as in the Seychelles pilot study" (page 286, NAS, 2000) The committee went on to conclude that there was a "good argument" for choosing the Faroe Islands study because the Faroe Islands was large (over 900 children), had been extensively analyzed and reanalyzed, and had two biomarkers of exposure. This same study does a detailed analysis of multiple outcomes across the studies even joining two types of analyses – that of specific outcomes as well as suggestions and analyses as an integrated model. Given such a robust bases start it is surprising that the FDA study chose to not use so many studies and outcomes in their analysis. It was as if the need to model benefits outweighed the scientific judgments on outcomes – throwing out the impacts of studies and outcomes as shown in Table 10-4. Choosing the SeyChelles study which although it had over 700 children, had such a low power to detect effects (only approximately 50%) for some of the outcomes from the Faroe Island and New Zealand (see page 278, NAS, 2000) seems to counter the desire to understand potential risks and ultimately a true understanding of potential benefits of eating fish. It is important to model these potential benefits but not at the "expense" of throwing out significant findings of positive adverse health outcome.

  50. Page 105, 2nd paragraph. Why not use primate data to look at this shift?
  51. Page 105, 3rd paragraph. Please clarify statements in this paragraph "We regarded the availability of individual subject data to be a prerequisite for including those data in the FDA modeling (although not for the modeling on IQ that was performed outside FDA)." Please explain this difference and why it is also not important for understanding exposure estimates.
  52. Page 105-106, last paragraph (105) and first paragraph (106). This reviewer applauds the authors in their efforts to examine the significance of looking at the same outcome but what if this excludes the most sensitive endpoints. Risk assessments don't often have this option if the purpose is to protect public health. Reviewer's comments on this section suggest these choices limit the applicability of the overall analysis.
  53. Page 105-106, Methylmercury Effect Independent of Fish. This reviewer is concerned about these approaches. It appears that lots of "second choice" endpoints/studies are made due to availability of data not that the questions are answered by data that is needed – Is this really an adequate solution to this missing data?
  54. Page 107-109 and Table IV-4. Importance of not knowing what component(s) in fish are important is significant here. On one hand the methylmercury effects incorporated into the model only include Iraq data with high dose effects and Seychelles Islands data that was largely negative for MeHg effects. Is it any wonder that the ultimate modeling of this data set will show only effects at high doses? If the Faroe Island and New Zealand data are excluded then this reviewer probably could have saved these authors a lot of time by predicting their modeling results without any additional effort. This reviewer feels that the FDA modelers must take a different approach, more inclusion of endpoints and studies to ensure the robustness of their approach for a broader public health context.
  55. Page 110-11, table IV-4. Please provide power of detection values for all the outcomes measured, whether they are included in the subsequent analysis or not.
  56. Page 115, table IV-5. Please specify for the report what percentage of the NHANES survey sample were due to non-consumers of fish i.e. non-fish methylmercury in maternal hair.
  57. Page 120, line 8 typo. Please correct spelling of Harvard.
  58. Page 121, last paragraph. This reviewer does not feel that the similarity in outcomes "….obviate concerns that our model has too narrow a focus…".
  59. Page 124, line 11. Please explain more fully item (1) about genetic variability for the methylmercury outcomes of interest.
  60. Page 128-130. Because of the many limitations in the basic reports on the benefits model this reviewer had difficulty in determining what and how this modeling exercise was informing the risk assessment.
  61. Page 138, Table IV-12. The understanding of these findings would be enhanced by a graphic or figure.
  62. Page 140, lines 1-2. Reviewers should examine these samples about MeHg being the only risk for CHD. Are there any toxins, Seafood allergens or other pollutants present that could confound this statement? Please add if other information is available or state what was considered and not used. Would ciguatera toxin or other natural toxins contribute?
  63. Page 140, 3rd paragraph. This reviewer appreciates that the authors include a review of Omega 3 FA in appendix C however the simple statements on assumptions in this section of the report really highlight limitations in the interpretation of that data. For example, authors state that they will not address whether supplements or natural Omega 3 FA are of relevance but also have just stated that likewise they will not look at types and varieties of fish consumption data – this is of concern as so many uncertainties can result from such large generalizations.
  64. Page 140, 3rd paragraph, footnote 46. This reviewer was confused by this statement and also by appendix F. It is difficult for this reviewer to understand rationale to include and not include the same types of information at whim.
  65. Starting Page 139, Section (d) Fatal Coronary Heart Disease. Due to data limitations, the modeling strategy for fatal CHD differed from the strategy for neurodevelopment because most of the studies looked at fish and CHD but did not look at MeHg exposure therefore this report assumed that all fish contained MeHg and exposure to MeHg correlated with fish ingestion. So this reviewer asks what the implications of these assumptions? When MeHg was measured and CHD was assessed adverse associations were seen in 2 out of 5 study populations (including 1 in the US), 4 of the 5 studies used exposure estimates for biomarkers other than blood. The FDA reviewers choose not to use these studies and this reviewer questions this decision and would suggest doing a sensitivity analysis to see if inclusion of these studies would result in a different conclusion.
  66. Page 139, last paragraph. This paragraph states assumptions for the exposure estimates for the CHD studies saying that they have "no data to suggest methylmercury levels in fish in the US market place are higher than elsewhere." Please provide references which are available for supporting such statements or not. References are available.
  67. Page 141, lines 4-7. This reviewer understands why the CHD data was only modeled on a population basis and not as individual severity as the fact that CHD either occurs and is fatal or it does not occur. However this reviewer does not feel that the population estimates for fish consumption in the general population reflect individual situations – see earlier concerns regarding use of averages of consuming and non consuming as averages and only looking at populations and not individuals.
  68. Page 141, first bullet point. Please specify if the use of "biomarkers" in this paragraph is only for biomarkers of effect or if this was also for exposure. Please specify.
  69. Page 142, paragraphs 2 and 3. This reviewer would be more convinved by the authors statements about the similarities of the studies (6) from US populations with overall He et al results if a sensitivity analysis was done showing the impact of including non-US populations. Just show variability in relationships (adjusted by N value contributions).
  70. Page 142, 2nd paragraph. Typo. Remove extra word "of" in first line.
  71. Page 143, 1st paragraph. Please cite appropriate references for making the assumption that sex differences in CHD rates from 45-50 ages are relevant for calculating sex differences for ages 15-44. This reviewer is not convinced. How much of a difference could this make?
  72. Page 144, 1st paragraph, lines 6-8. Reviewer feels this is a major assumption and feels uncomfortable without including additional evaluation in the report for such assumptions.
  73. Page 144, last paragraph. This reviewer appreciates the lack of specific fish data but is concerned that the model done is this section will be used with specific fish type data for final advisories when this overall report is implemented. How can these conclusions be made when it is missing fish variety data and the endpoints modeled and yet are used for benefits calculated for other end points?
  74. Page 147. Table IV-15. Please provide a figure for the data in this table. Comparing the cdfs for the model results would be helpful. Are there really any significant differences between these two predictions?
  75. Page 145, table IV-15. This reviewer suggests discussing the number of significant figures displayed in this table.
  76. Page 148-151, "What if" scenarios. Please see this reviewer's earlier questions regarding assumptions made for the CHD modeling. Please do these sensitivity calculations. It is the reviewer's option that those assumptions would out weight the sensitivity assessments performed in this section.

    Authors should add some acknowledgement in this section that because most women of child bearing ages do not know that they are pregnant until later in first trimester the assumptions that women of child bearing age would extend their lifestage where they would be consuming 12 oz a week is highly acceptable. Tie modeling observations back to the biology.

  77. Page 152, fifth paragraph. Please indicate in the main body of text whether the assumption of the ratio of non-fatal to fatal stroke occurrence is age adjusted. If not, please include in "what if" scenario.
  78. Page 152-153. Authors need to explain differences in the biological assumptions for their CHD and fatal stroke models. For example, with the CHD data, lots of discussion is given to the "non-relevance" of the Finish studies and the non-US studies yet for fatal stroke it appears that numerous non-US studies are considered in the various models presented. More discussion is needed and in fact if this reviewer is to believe what the authors say is true for CHD then similar exclusion of data is needed. (Personally, I don't know whether exclusion of non-US data is justified but at least these endpoint approaches should be similar or each evaluated for sensitivity of assumptions in model predictions).

    Likewise for fatal stroke it appears that 7 out of 10 studies are for males only. How was this handled? Why are these limitations in data not described? Rather emphasis seems to be on stroke death rate by age for gender and the fact that so few studies relevant for women are available. Again the excitement of the modeling is diminished by the lack of a thorough biological discussion of the limitations. This reviewer was disappointed as this group of authors is uniquely poised to conduct these assessments but in their excitement about the modeling forgot their own earlier caveats.

  79. Page 158, footnote 55 is essential to the discussion and should be moved into the text. Relate this discussion to the studies chosen for the modeling.
  80. Page 159, 2nd paragraph. Authors make important points about model estimates. Yet this reviewer was left wondering what to do with these "problems" in model estimates. This paragraph states that "For this same subpopulation, the "Carrington" stroke model's central estimate is that 127 stroke deaths are being averted for this subpopulation from fish consumption but at the fifth percentile confidence interval it also estimates that fish consumption could be averting up to 6,476 deaths." For this latter estimate to be correct, fish consumption would have to be averting over four times more stroke deaths than occur in this subpopulation on an annual basis. At the other end of the spectrum – the 95th percentile of the confidence interval – the "Carrington" stroke model predicts that fish consumption may be causing 434 stroke deaths, i.e., roughly one-third of all stroke deaths for women aged 16-45. Such an outcome seems implausible given the nature of the underlying data (see previous footnote). The results from the fifth and 95th percentile confidence intervals raise similar issues for other subpopulations." This reviewer was left wondering if such "implausible" observations at both ends of the dose response model outputs negate these models. Authors need to provide more in terms of what these observations mean for their overall RA.
  81. Page 160-163. Authors should refer to earlier comments on the "what if" scenarios given for CHD for these additional endpoints here and for the rest of this section.
  82. Page 163, section F. This section, like the previous sections, should discuss limitations in the assumptions made for non-fatal events. If in the previous section the authors suggest that the estimates seem implausible (see page 159). What are the author's conclusions about these assumptions? To this reviewer these predicted non fatal events averted per year due to fish consumption seem extraordinarily high. Authors should comment. Does this discredit the reliability of their predictions?

    Page 164, (g) Summary of Interpretation. Fetal Neurodevelopmental risk assessment. This author looked forward to this section yet the bottom line from the modeling exercises was rather disappointing (see page 165, paragraph 4 and 5. Page 166 paragraph 1). If one combines these statements with this reviewers earlier concerns about what models are modeled then the conclusions that at 12oz fish benefits outweigh adverse healthy benefits for unborn offspring are significantly weakened from the summary statements found earlier in this document (see thumbnail summary and executive summary). Based on these observations this reviewer feels that the author should more closely reflect the limitations (qualifiers) stated in this section in the earlier conclusions on pages 4-5, 18-19 and 20-21. For example statements on page 20, second paragraph that says on line 2 "The size of an adverse effect is typically smaller, and can be much smaller, than the size of a beneficial effect." VS page 165, 3rd and 4th paragraph. "The risk assessment indicates that encouraging consumption up to, but not beyond, 12 ounces per week essentially accepts some risks in order to obtain nutritional benefits from fish consumption, while also avoiding slightly higher risks and benefits associated with higher consumption, as described below:" "Consumption of up to 12 ounces of fish per week is most likely to produce a small neurodevelopmental benefit over no fish consumption for most people but it could also produce a small neurodevelopmental decrement (up to four percent possibility) for some people. Consumption over 12 ounces per week includes a small possibility of a larger net benefit but it also includes a slightly increased possibility (five percent rather than four percent) of a net adverse effect." This section continues to say that for "age of talking," "maternal consumption of up to 12oz of fish per week would likely result in delays in age of talking of less than a day for most offspring. Consumptions over 12 ounces of fish per week could indicate the possibility of a somewhat longer delay, although the likelihood of larger delay is low." This section continues and says that "IQ: Based on the results from this model, maternal consumption up to 12 ounces of fish per week would likely result in IQ decrements for most people of less than one IQ point while consumption over 12 ounces a week would most likely result in slightly larger IQ decrements – although still less than one IQ point. For both the age-of-talking and IQ models, avoiding shark, swordfish, king mackerel and tilefish could help reduce the decrement for the offspring of women who eat these species." However the authors state two things earlier that 1) they where unable to model all of the epidemiology results and thus used a selection of studies (see this reviewers comments at #49) and 2) the authors also state that they were unable to attribute benefits to specific fish types and that methylmercury was modeled as an estimate of fish ingestions. Several of these specific factors makes this author question the wording and implied certainty both of the magnitude of adverse effect as well as attribution to specific fish beneficial factors in the overall report conclusions.

  83. Page 166, Fatal Coronary Heart Disease Risk Assessment, 1st paragraph. Include in these statistics the low estimate of deaths averted per year for the Carrington model as well as the He model.
  84. Page 167, Fatal Stroke Risk Assessment. Include in the calculations for this section the limitations noted by the reviewer in the preceding sections.
  85. Page 385, 2nd paragraph, lines 1-2. Add references for all statements like this.
  86. Pages 384-387, Appendix E. This is a very useful summary. It would be helpful to the readers of this document to add the details on safety factor for each of the levels discussed. This reviewer suggests adding a short explanation on how each of the safety or uncertainty factors was derived. For example, how was a 6.4 fold safety factor determined for JECFA's PTWI on page 386 and a 5 fold safety factor for Canada's pTDI. The final paragraphs in section C on pages 386 and 387 does not provide enough specificity.
  87. Appendix F, Page 390, 2nd paragraph. As written this paragraph appears to only discuss one side of this issue. Couldn't it be just as likely that another contaminant could increase the risk, i.e. PCB versus other risk factors? Both sides of this possibility should be included.
  88. Page 360. This reviewer does believe it is very important to expand the consideration of higher fish consumption patterns as new studies show how much higher these ingestion rates can be and authors should include considerations of what this does to the estimated risk/benefit ratios.
  89. Page 392, 2nd paragraph. This reviewer would encourage the agency to expand and further develop these ideas on consumer information and consumer advice.
  90. Page 393, last paragraph. This reviewer would suggest that not only would "The net effect over time of eating fish containing methylmercury can be different from the effect of eating the same amount of methylmercury over time in some other food, e.g., pilot whale," but that this review should acknowledge additional consideration between types of fish and seafood that should be considered in this section.
  91. Appendix F. This reviewer would also request that some discussion or consideration of the larger impact of using benefits of food in this section on FDA's role. Adoption of such an approach could impact our consistency in applying "action" levels for international food products. This approach proposed in this FDA document would seem to have very large implications for other action levels and for food decisions made via world trade organization efforts across countries. Please add/consider.
  92. Appendix G, Page 397, 2nd paragraph. Please provide context for the use of 75th and 90th percentile. Why not 95 or 99 percentile?
  93. Page 397, 3rd paragraph. For context, please provide statement about whether it is the overall decreased use of action levels or is it just decreased for methylmercury action levels.
  94. Page 398, 4th paragraph. Surely there is FDA guidance on issue of "representative" aspects of sampling foods. Cite FDA guidance here.
  95. Appendix H, Page 401, 4th paragraph, line 6, editorial. Extra period.
  96. Page 402. Fish consumption issues, Appendix H. This appendix cites several fish consumption surveys, data and refers to the issue that many population based studies include both fish consumers and non-fish consumers in determining (page 401, footnote 81) per capita fish consumption. Use of such data should be reworded as the risk question facing FDA are risk to fish consumers. (Please see this reviewer's earlier concerns as how consumers were defined).
  97. Pages 403-405. Appendix Hg. Since multiple sections of the advisory tables on pages 403-405 refer to the WHO guidelines. Please add as footnote.
  98. Page 406-407, Appendix I. It is the option of this reviewer that this appendix should be removed as is appears to be an "add on" and is not done in as carefully detailed manner as the rest of the report. In fact it detracts from the earlier sections. This reviewer had great difficulty with these two pages and actually did not agree with these discussions. It appears to "fixate" on action level and demonstrates very little understanding of how reference doses are set.
  99. Page 407, 3rd paragraph, line 5. Check spelling of "substantial."
  100. Page 164-168 (g). This reviewer was very pleased to read the brief but "to the point" final section. It hit key points and observations for the model but also was rather modest in claiming what could be concluded from the available data. For perspective at this point, should such ending paragraphs go back to USFDA context – for example by only including closing commercial sources of fish. What do these conclusions mean in that sense? This report should include a section that builds from the models to show the value of missing data for impacting decisions that FDA needs to make. This reviewer feels that the "true utility" in the model presented in this report is to show how critical missing data has lead to assumptions that ultimately limit our ability to understand both the risks and benefits of fish consumption. It should not be too difficult to prepare a table of data that was available that which if available would resolve significant risk assessment undertaking. Without this I fear that we will be using the same population data 10 years from now and missing similar critical data. This would really help identify priorities for research finding.
Summary and Interpretation

The sophistication of the modeling is overshadowed by inconsistent assumptions across endpoints. For example for one endpoint only US studies are taken (CHD) for another (fatal stroke) a mixture are taken. But these inconsistencies more significant as although there are inconsistencies in the databases available, arguments are given for each of the end points that are counter to the choices made in the next assessment. This reviewer would suggest less ambitious modeling of everything and instead use consistent assumptions for key end points and include other less robust or less relevant data points for appendix discussions. A good example of the shift in emphasis that is needed is the request by this reviewer to move footnote 55 into the main text. This is just one example of the misplaced emphasis.

FDA RESPONSE TO THE PEER REVIEW COMMENTS

The following responses are to peer review comments that requested changes or clarifications. The responses are provided in either or both of the two right side columns in the table, below. It should be noted that the peer reviewers received a single document to review that contained (1) the risk and benefit assessment with accompanying and explanatory materials; and (2) two appendices devoted that inventoried research and analyses on beneficial health effects from fish and omega-3 fatty acids. Partly in response to peer review comments FDA has divided the document into two documents that track (1) and (2) above. The first table, below, is devoted to comments and agency responses involving the paper containing the risk and benefit assessment and accompanying and explanatory materials ("Report of Quantitative Risk and Benefit Assessment of Consumption of Commercial Fish, Focusing on Fetal Neurodevelopment (Measured by Verbal Development in Children) and on Coronary Heart Disease and Stroke"). The second table addresses comments and agency responses involving the paper containing the inventory of research and analyses on potential beneficial effects from fish and omega-3 fatty acids ("Summary of Published Research on the Beneficial Effects of Fish Consumption and Omega-3 Fatty Acids for Certain Neurodevelopmental and Cardiovascular Endpoints.")

PEER REVIEW CONCERNS and FDA RESPONSES: RISK and BENEFIT ASSESSMENT REPORT

Concern
Response
Exposure-related issues:
1. Variability in cultural and regional differences in fish consumption were not adequately modeled.
The assessment is intended to be nationally representative of the U.S. population. It does not address risk to segments of the population whose exposure to methylmercury or patterns of fish consumption may be substantially different from that of the population as a whole, as a result, for example, of their subsistence or sport fishing in localized bodies of water that might be subject to unusual conditions. Separate assessments would be needed to predict effects in such sub-populations.
2. The report lacks regional sensitivity to the data on average concentrations of methylmercury in fish. How large are regional differences? Could this change any of your calculations?
There are no data available on regional consumption of commercial fish. The national surveys of fish consumers would incorporate regional consumption since fish consumption tends to be higher in those areas along the coasts where fishery products are more readily available. Since the consumption surveys FDA used in its exposure modeling already include upper percentile consumers, it is unlikely that such information would significantly alter the results of the assessment.
3. Would the NHANES survey of exposure to mercury miss ethnic groups? If so, please say so.
As we explain, NHANES is likely to miss subgroups of high fish consumers such as sport and subsistence fishers. To the extent that ethnic groups fit those categories, NHANES would likely miss them. The important point to remember is that FDA’s assessment is intended to be nationally representative. More focused assessments directed toward localized situations, including specialized consumption patterns by ethnic groups, would be useful future research.
4. Modeling should include peak exposures during critical time points. What is the risk to the fetus if the mother consumes fish that day?
We could not do that for two reasons. First, there are no data by which estimates of peak exposures could be developed. Second, it is unknown what peak exposures represent in terms of potential risk from methylmercury. Data available for use in dose-response modeling are from long term, chronic exposures. Put another way, the models (including models created outside of FDA) all presume that average (or "area under the curve") exposure is the relevant dose-response metric. Therefore, this metric must be used for the exposure assessment. The question of peak exposures is a good one, but simply beyond our ability to model at this time.
5. The report largely dismissed the importance of modeling the 20 percent commercial fish consumption that FDA regulatory jurisdiction would be concerned with within the broader context of potential contribution of methylmercury exposure from the other 80 percent of fish consumption that was non-commercial and not within FDA control.
The assessment accounts for the vast majority of commercial fish. Consumption of non-commercial species was not included because non-commercial fish are outside of FDA's jurisdiction and because data are lacking on the consumption of these species. On an overall per capita basis, the National Marine Fisheries Service estimates that recreational catch would add about 10 percent to its estimate of per capita non-recreational fish consumption in the U.S (NMFS 2005).
6. The authors should make an attempt at discussing total contribution of commercial fish to overall consumption.
See response to item 4, above.
7. Exposure modeling should have used an approach based on a relative source contribution model such as the lead IEUBK models.
The lead IEUBK models are not relevant to this assessment because lead exposures occur through a number of plausible sources and pathways (e.g., soil, duct air, food water, etc.). This is not the case with methylmercury. The source and pathway is almost exclusively fish consumption.
8. Provide a reference for the assessment that exposure to methylmercury in molluscan shellfish is not of public health significance.
The FDA database of concentrations of mercury in fish, including molluscan shellfish (http://cfsan.fda.gov/!frf/sea-mehg.html), shows average concentrations in clams that are nondetectable by current means of detection. In oysters, average concentrations are 0.013 ppm and in scallops they are 0.05 ppm. These concentrations are very small.
9. A better explanation is needed for why the limitations in the NHANES survey described in Section II of the report do not significantly affect the data utility. Also, clarify that the individuals in Table II-1 (results of NHANES blood sampling) are only consumers of fish.
The limitations described in Section II refer to the hair and blood sampling that have been conducted as part of the survey. FDA did not use these results in its exposure modeling except for purposes of comparison. Table II-1 (results of NHANES blood sampling) includes fish consumers and non-consumers because it represents all sampling conducted by NHANES. That survey is of the general population and is not limited to fish consumers.
10. In the discussion of whether methylmercury concentrations are increasing in commercial fish species, what was missing was a discussion of changes in types and amounts of fish consumed over time.
This question of how exposure is changing as a result of changing patterns of consumption and availability of species is potentially very important and we are aware that it is being studied by some scientists. It is beyond the scope of this assessment, however, because the assessment attempts to model current exposure and not those that may have occurred in the past. Any updating of the FDA exposure modeling that occurs in the future will attempt to take into account how consumption patterns may have changed since the last exposure modeling. We have attempted to make our estimates as current as possible by using the most recent marketing data.
11. FDA should reconsider use of the two day survey consumption survey that it did not use because it takes seasonal differences into account.
From our standpoint, the two day survey does not adequately characterize frequency of consumption over a month or a year. We do not regard seasonal differences as being the issue here.
12. The reviewer disagrees with the emphasis placed on the 30-day survey over the shorter surveys. Recall memory drops off with longer survey duration.
The 30-day survey was used in conjunction with the short term surveys and was not emphasized. The report explains how the longer and shorter duration surveys were both used to take advantage of the strengths of each and to mitigate the described weaknesses of each.
13. The emphasis of the consumption estimates appears to be on population based estimates versus individual estimates. Discuss how both estimates were obtained.
The FDA population estimates were created by generating estimates for multiple individuals.
14. The "repetition ratio" that FDA devised was developed from the 30 day survey of women of childbearing age but the ratio was extended to the entire population. Women of childbearing age would be decreasing their fish intake, especially of high methylmercury fish, due to the presence of consumption advisories. This could cause an underestimation of methylmercury exposure.
The 30-day survey was conducted in 1999-2000, before the current 2004 consumption advisory. It remains the only source of long-term consumption frequency data available.
15. The report notes that 85 percent of women reported eating fish in the 30 day survey, but that this percentage might underestimate the number of consumers yet no additional comments are made as to what actions or corrections should be made.
The exposure assessment includes corrections for this potential underestimation, which are described in Appendix A.
Fetal Neurodevelopment Modeling – Clarity:
1. The explanation for the choices and rationales for the fetal neurodevelopmental modeling are difficult to follow due to complexity. Move the discussion of the fetal neurodevelopment modeling back, behind the discussion of the cardiovascular and stroke modeling, since the latter two are easier to understand.
The explanation of the fetal neurodevelopmental modeling in Section IV has been substantially redrafted for purposes of clarification. We believe that this clarification should solve the problem better than moving the order of the models in the text. We prefer to lead with the fetal neurodevelopmental modeling because this issue has been central to questions of risk and benefit surrounding methylmercury in fish.
2. The rationales and goals for the modeling of the methylmercury contribution to the net effect analysis are muddled. You shouldn't say that this would be the effect if the fish had no benefits. All fish have benefits.
The explanation for the methylmercury contribution modeling has also been redrafted to clarify that the methylmercury contribution is a subset of the overall net effect that maternal consumption of commercial fish could have on fetal neurodevelopment.
3. The use of Z-Scores to express the results of the assessment will not be understood by a broad audience. Convert the results to something more understandable.
We agree with this comment. Z-Scores essentially represent the size or magnitude of a change, such as an improvement or worsening in early age communication ability as a result of eating fish or being exposed to methylmercury. We concluded that the size of an improvement or decrement in such ability could be compared to the size of an IQ shift. Consequently, we convert Z-Scores to "IQ Size Equivalents" (a term we coined for this purpose).
4. Clarify the results by expressing them more quantitatively and not qualitatively, especially with regard to whether a net adverse effect is a remote or plausible possibility.
The draft has been revised to clarify in the manner recommended.
5. A better explanation is needed for Table IV-11.
Table IV-11 (now renumbered as Table V-8) has been substantially clarified to explain that it is essentially a "what if"-type of table. It provides estimates of what the net effect would be if people were exposed to methylmercury at current U.S. levels of exposure but only as a result of eating commercial fish with average concentrations of methylmercury (weighted for popularity).
6. There is a need for more background discussion of variability in the endpoints. Changes in walking/talking are highly variable and are likely to be affected by many factors.
The published reports of the studies from which our data have been obtained generally provide considerable discussion on the variability of the data, e.g., the range of confounders that could affect the results and how the researchers addressed those confounders. We recognize that we have limited the discussion of this variability in our report. The report is still in draft and we will give some consideration to further elaboration on that matter. We note that our report does address the impact of either mercury or fish on behavioral measures as changes in Z-Score or IQ, both of which scale performance relative to background variability.
Fetal Neurodevelopment Modeling – Assumptions:
1.It is a questionable assumption that a combination of data from Iraq and Seychelles can produce a methylmercury effect minus fish.
As a result of this comment we modeled Iraq alone and compared it to the combination of Iraq & Seychelles. There was only a small difference, mainly because the Iraq data dominate the dose-response function. Consequently, we concluded that the representation was adequate for our purposes, although admittedly not perfect.
2. It is questionable assumption that Iraq data can be used to estimate age of first talking/walking because the exact age of the Iraqi children were unknown.
The Iraqi mothers were able to place the ages of their children within 6 month blocks of time. The issue for modeling was whether age within 6 months was adequate for our purposes. We concluded that for modeling, age within 6 months was adequate because a) the effects are, in some cases, larger than 6 months, and b) the inability to determine exact age does not bias the results (i.e. the effects will not be made to look higher or lower). We have added some further clarification on this point in the text.
3. It is a questionable assumption that verbal comprehension data from the U.K. can be combined with age of first talking data because verbal comprehension occurs sooner. The result could be that the net effect model could underestimate the methylmercury effect.
For purposes of our modeling we do not regard it as essential whether verbal comprehension occurs somewhat sooner than age of first talking, assuming that this is in fact the case. It is sufficient that they are both measures of early age verbal development. The verbal comprehension tests were given in the United Kingdom at specifically-chosen ages. Some children in Iraq walked/talked sooner than the ages at which the tests were administered while others talked later than the age at which the test was administered. The overlap in the ages of first talking/walking in Iraq and the ages at which the children were tested in the U.K. was substantial. We have added more text on the likelihood that the net effect modeling might overestimate or underestimate either the methylmercury contribution or the fish contribution. The data on the methylmercury contribution involved some of the most extreme events ever recorded, so we doubt that it would be underestimated to any significant degree in the modeling.
Use of data at later ages:
The National Academy of Science report on methylmercury (NRC 2000) stressed the importance of looking at outcomes at 5 to 7 years of age due to decreased ability to detect neurodevelopmental effects from methylmercury at earlier ages. Why not use endpoints at later ages or at least use sensitivity analysis re later age endpoints?
We used early age verbal development as an indicator of neurodevelopment because we had data on it sufficient to develop dose-response functions for both an adverse contribution of methylmercury to the net effect and a beneficial contribution of fish to the net effect. In order to determine whether this endpoint is sufficiently representative in terms of its sensitivity to methylmercury, we performed a comparative analysis by matching the results against dose-response functions developed for the effect of methylmercury on IQ from the Seychelles, the Faroe Islands, and New Zealand at later ages (Axelrad et al., 2007) and on a wide range of neurodevelopmental tests administered in these three locations, also at later ages (Cohen et al., 2006b). The size of the effect we modeled at an early age was similar in size to the effects seen at later ages on IQ and on the wise range of neurodevelopmental tests.
Use of data from the Faroe Islands and Seychelles studies:
Per the NRC (2000) report, the model should have used data from the Faroe Islands rather than from the Seychelles study. Choosing the Seychelles study seems to counter the desire to understand potential risks and ultimately a true understanding of potential benefits from eating fish.
The NRC 2000 report chose the Faroe Islands study for purposes of developing a benchmark dose, which requires that a conclusion be drawn from a particular study. The concept of benchmark dose is valid and essential for certain types of assessments that develop a single level of exposure deemed to be without appreciable risk. FDA refers to these types of assessments as "safety assessments" and utilizes them for a number of purposes. However, FDA's modeling in this report is not for the purpose of performing a "safety assessment." Most dose-response functions developed for this assessment are based on multiple studies. As it turned out, FDA used results from the Faroe Islands, Seychelles and New Zealand in its comparative analysis for the methylmercury contribution of the net effect (see discussion above on use of data at later ages). Results from these studies were not excluded.
Use of animal data:
1. Whether human data could be supplemented with primate data in modeling fetal neurodevelopment.
We do not think so, as a practical matter. Primate data, if available, is limited to a small number of studies and therefore a small number of primate subjects. Use of such data would require additional assumptions and methods of extrapolation from the laboratory primate data to humans.
2. Whether primate data could be used to model the neurodevelopmental effect of postnatal exposure by young children.
We do not think so that animal data would be helpful in identifying a broader developmental period in humans. The issue of post-natal sensitivity of the young remains a theoretical possibility solely on the basis of information about developmental processes occurring during that period of life and not on the basis of any methylmercury-specific information in either laboratory animals or humans.
Genetic susceptibility to methylmercury:
Explain more fully the possibility that some people might be adversely affected due to genetic susceptibility.
That mention of genetics has been deleted. It was offered as a hypothetical explanation for why some people might be adversely affected while others are not. It is an interesting topic that could be the subject of follow-on research and discussion, but it would involve an extensive narrative that would not be integral to this assessment.
Postnatal Exposure:
1.Regarding the Daniels et al. (2004) study of postnatal fish consumption, the report says that methylmercury levels in the children were not measured and then says that there is no evidence of an association between postnatal exposures and adverse effects in young children. This concluding statement is very speculative.
The statement has been dropped from that section.
2.The report states that there are no reports of neurological effects in mothers in either the Seychelles or Faroe Islands studies. Were the mothers examined for neurological effects?
The section on postnatal effects in adults has been deleted from the report because a decision was made that this report would focus on effects in the developing nervous system (fetus, young children).
3.Provide more detail about one of the studies on postnatal exposure (Nurk et al.,2007).
Discussion of Nurk et al. has been dropped for the reason given in 2., above.
Effect of other contaminants:
1.What about situations where other contaminants in fish affect the same neurodevelopmental endpoints as methylmercury?
This is a significant question that warrants further research. However, addressing this question is beyond the scope of this assessment.
2.The impact of other contaminants and food components on the net effect should be addressed.
See response 1., above. We note in addition that data concerning net beneficial effects on neurodevelopment associated with fish consumptionmust to some extent factor in other possible neurotoxins in the fish consumed by the study populations.
3.Specify that the "what if" modeling does not include health effects from eating other contaminants in fish other than methylmercury.
We think this is clear for all the modeling, in that the assessment does not specifically take into account other contaminants. The net effect on a particular health endpoint inherently takes into account whatever is in the fish, however.
Additional data from other studies should have been used in the assessment of fetal neurodevelopment:
1.In assessing the fish contribution to the net effect, additional data from other studies should have been included. The absence of individual data should not have caused them to be ignored.
The approach that we chose for our modeling required data: (1) that were not significantly confounded by methylmercury; and (2) that measured early age verbal comprehension or early age motor comprehension. Those two factors limited the studies from which we could draw data even if we had been willing to accept less than individual subject data for the model. The published reports from other studies provide results of tests of statistical significance which divide people into categories regarding methylmercury exposure and/or fish consumption but they yield very little information about the dose-response relationships. Also, the other studies presented the problem of successfully differentiating between two highly correlated variables, i.e., fish and methylmercury. The choices we made allowed us to minimize that particular problem. Whether the results would have been significantly different had we modeled the fish contribution from other studies and other aspects of fetal neurodevelopment remains to be seen in future work. However, the data from the studies that have looked for associations between maternal fish consumption and fetal neurodevelopment have produced generally consistent results, as described in Section III of the report.
2.For studies that were not used in the modeling, were the authors of those studies approached for individual subject data?
Yes, authors were approached. FDA used the data that were provided.
Mercury-to-fish ratio:
Please quantitate what is meant by high methylmercury to fish ratio.
That term is no longer used.
Fish consumers vs. non-consumers:
Please specify that only consumers were evaluated in the risk assessment as it is essential that the models do not include non-consumers.
The dose-response modeling does include non-consumers, primarily in order to estimate population effects. The percentage of people who do not eat fish is small and, in any event, does not impact on the results estimated for those individuals who do eat fish.
"Thumbnail Summary" and "Executive Summary:
Several peer reviewers urged revisions to the two summaries at the beginning of the document, based on appropriate level of detail (some asked for more; others less) and length, and one proposed significant edits for style and clarity.
The entire document has been streamlined for readability and this revision has made a lengthy "Executive Summary" less necessary. Consequently, we have dropped the longer summary and revised the shorter summary substantially.
Concerns about "Summary and Interpretations" sections on the modeling:
A range of concerns were expressed on the characterization of the overall results from this assessment, including whether limitations were adequately taken into account.
This section has been deleted from the report in favor of other drafting that is clearer on limitations and other matters.
"Roadmap" of the Contents:
Include a "roadmap" to what can be found and where it can be found in the document, other than simply a table of contents.
We have streamlined the document and reorganized parts of it to make it easier to follow, and as a result we hope readers will find a detailed "roadmap" less necessary. However, we have included a "roadmap" in the introductory Section I that briefly describes what can be found in each section and Appendix.
Threshold question:
Explain what is meant by a "threshold question".
That term has been dropped. It referred to FDA's responsibilities under the Federal Food, Drug, and Cosmetic Act.
Other components:
Explain what is meant by "other components" in fish.
The section in which that term was used has been dropped, but the concept remains in the paper. The term referred to nutrients that could have a beneficial effect on the endpoints addressed in the assessment.
Tables and figures:
Replace some of the tables with figures and add more figures.
We appreciate that some people prefer tables while others prefer figures. Where we have tables that could made into figures, we are considering adding a figure in addition to the table in order to accommodate both preferences.
Target audiences:
The report should specify target audiences for each section.
We believe that the extensive redrafting and streamlining has made the document easier to read for a lay readership. The two appendices that are devoted to the modeling remain technical by design. We so state in the report.
Requests for additional papers to be reviewed in the report:
1.For fetal neurodevelopment, the Oken et al. (2008) study of a cohort in Denmark.
The Oken 2008 Denmark paper was added.
2.For consumer perceptions about risk, "Consumer Perceptions about Seafood – an Internet Survey."
We have not added this paper because consumer perceptions, while important, are beyond the scope of this report.
3. For evaluating national fish advisories, Sherer et al.(2008), Environmental Health Perspectives, 116, 1-9.
We have not added this paper to the report because the evaluation of fish advisories, while important, is beyond the scope of this report.
Hierarchies of Evidence:
Add a better description of the hierarchies of evidence. The report seems to value them in reverse order of their real value.
The issue of "value" goes to:
  • Presentation in the paper: We believe that the division of the paper into two papers addresses this issue.
  • Applicability to the risk and benefit assessment: For the reasons stated in the paper, the data incorporated into this assessment are from observational studies and not from RTCs. For fetal neurodevelopment, RTCs would be difficult to devise. For cardiovascular endpoints we tracked the results of two meta-analyses that analyzed observational data and developed dose-response functions from them that we could utilize in our modeling. Those meta-analyses were limited to studies involving "fish" where the contributions of individual nutrients were not measured. Future risk and benefit assessments may well take into account the contributions of individual nutrients and utilize the results from RTCs.
Randomized Clinical Trials in the Quantitative Assessment:
Data from randomized clinical trials (RCT's) on fish oil and neurodevelopment or risk of coronary heart disease should be added to the quantitative assessment. The idea that RCT's of fish oil somehow do not inform about the effects of fish consumption is disingenuous. RCT's of omega-3's and post-natal exposure are powerfully confirmatory and we should say so and use them.
It is true that no data from RTCs were incorporated into this assessment for the reasons stated above. We recognize the potential significance of such studies, which is why our review of studies in the overview document includes them.
Size of the Confidence Intervals:
Paradoxically, endpoints with relatively little data (methylmercury and neurodevelopment; fish and neurodevelopment) have relatively small confidence intervals while endpoints with enormous amounts of data (fish and CHD) have spuriously large confidence intervals.
The models for 'methylmercury and neurodevelopment' and 'fish and neurodevelopment' are essentially subsets of the modeling for 'net effect on neurodevelopment,' so we think the better comparison may be the size of the confidence intervals for 'net effect on neurodevelopment' and coronary heart disease. The confidence intervals for net effect and for one of the two models we used for coronary heart disease (the 'meta-analysis" model) are actually similar in size, i.e., they are similar in terms of distance from the central estimate to the confidence limits. The size of the confidence intervals for the second model we used for coronary heart disease (the "pooled analysis" model), however, are considerably larger than for both the 'net effect on fetal neurodevelopment' model and the first model for coronary heart disease (the 'meta-analysis' model). We do not regard this difference to be a problem, however, primarily because the central estimates for the two coronary heart disease models are similar. All the central estimates from these models estimate similar numbers of deaths averted due to fish consumption. The wider confidence intervals for the 'pooled analysis' model suggest a small possibility of increased risk under some circumstances, although methylmercury is only one potential cause among many. This small possibility is consistent with data that were incorporated from the various studies into the model. For a regulatory agency such as FDA, the implication is that we should not simply disregard that possibility of some increased risk from some people. In any case, it should be noted that we did reanalyze the CHD and stroke data using a linear model with a maximum effect parameter. The use of such a model reduced the size of the confidence limits with high levels of fish consumption from what they were when reviewed by the peer reviewer.
Risk of fatal coronary heart disease from eating fish:
1.The report says that the likelihood of adverse effects from eating fish appears to be very low. How low is this?
For one of the two models we used for fatal coronary heart disease, the results within the 95 percent confidence intervals only included some number of deaths averted, so the likelihood from that model is at least 95 percent that there are no adverse effects. For the second model, the confidence intervals contained a small possibility of adverse effect. The likelihood from that model is about 85 percent that there are no adverse effects. These percentages are now provided in the report.
2.The report assumes that any increase in risk of fatal coronary heart disease would be from methylmercury. Are there other pollutants or seafood-related toxins that could confound this statement.
The assumption in question was a misstatement and has been dropped. The assessment does not attribute any increase risk from eating fish to an underlying cause, nor do we assume one. The report now says that methylmercury would have to be regarded as one possible cause. There are no other seafood-related toxins that been shown to be a risk factor for coronary heart disease.
Quality Assurance/Quality Control:
The complexity of these analyses makes it imperative that there be a rigorous QA/QC of each set of data and analyses. It would be useful to include summaries of the QA/QC in the appendix.
We agree that there is a need for rigorous review. The exposure methodology and the methodology for the methylmercury contribution to net effect have been published previously. For coronary heart disease and stroke, the dose-response functions for two of the four models have been published previously. All of these publications are cited in the text. In addition, these documents are subject to extensive peer and public review.
Account of Cardiovascular Disease:
The body of the report gives short shrift to CVD. More attention is needed to a full account of this embarrassingly rich data in addition to that in the Appendix, explicitly detailing the relative sizes, strengths, and robustness of these studies, compared with the MeHg and heart disease data.
Section III-B of the Risk and Benefit Assessment document now contains an overview of the research on CVD in greater depth than previously.
Contribution of omega-3 fatty acids (and possibly other nutrients) to net effect:
1. The idea that we could not assess the contribution of omega-3 fatty acids to the net effect is wrong. We should either do it or admit that we could have, even if we did not. EPA and DHA (types of omega-3 fatty acids) are clearly major active ingredients for both neurodevelopment and the coronary heart disease benefits of fish intake.
We see value in being able to assess the contribution of individual nutrients to the beneficial contribution that fish make to the net effect. Doing so would enable us to estimate a species-specific mix of fish that could maximize benefits consistent with minimizing risk. The peer reviewer's point that we should do so is meritorious. It was, however, beyond the scope of this assessment for a number of reasons:
  • This assessment involved the development and application of risk and benefit concepts that were novel to FDA. For this first application we wanted to limit our modeling "fish" as a "package" without attempting to tease out contributions from individual nutrients. Nor was it necessary to do so for the purposes of this particular assessment. Our primary objective was to add to our understanding of risk from methylmercury.
  • It is not yet clear to us whether we could have modeled only one type of nutrient to the exclusion of others. Also, the results of the RTCs on neurodevelopment are mixed, i.e., the evidence on the contribution of omega-3 fatty acids to neurodevelopment is mixed relative to the evidence on coronary heart disease.

We did modify the text, however, to clarify that the modeling of individual nutrients was beyond the scope of this assessment without speculating on whether we could or could not have done so.

2. What is the relationship of low nutrient fish and other contaminants such as PCB's? What about high nutrient fish?
As explained above, and in the report, the assessment does not differentiate between lower or high nutrient fish or address chemicals other than methylmercury.
Impact on policy decisions:
1.Redraft to conclude that the 12 ounce per week limit in the current consumer advisory should be discarded.
2.The impact of "what if" scenarios on policy decisions, e.g., what guidance is appropriate, should be discussed.
3. Reviewer expresses concern over how lack of fish-specific results from the assessment of coronary heart disease would affect final fish advisories.
1., 2., and 3. The draft risk and benefit assessment does not affect the current consumer advisory. Changing the advisory is a risk management measure that is outside the scope of this assessment.
4."What if" scenarios that exclude benefits should clearly be identified and not used for policy decisions. Fish are not consumed without benefits.
4."What if" scenario results from the methylmercury-only modeling (i.e., the methylmercury contribution to the net effect) have been removed from the modeling results in the main text. The main text now only provides "what if" results from the net effect modeling, since we regard those results as more germane to the actual consequences of eating more or less fish. "What if" results from the methylmercury-only modeling are still contained in Appendix B, however, in order to show the estimated size of the methylmercury contribution to the net effect the scenarios.
"What if" scenarios:
1. "What if" scenarios predict capping at 12 ounces of fish per week would reduce national neurodevelopment even if women ate only fish that were low in methylmercury. Would this same effect be seen if it were capped at 16 ounces? Or is it the type of fish that should be capped? Or is it the need for more variability in the type of fish?
We did not have a scenario with a 16 ounce cap so we cannot answer that question directly. We did model current U.S. exposures at "baseline," where people eat lower and higher methylmercury fish, and we compared these results against a scenario in which people are identically exposed to methylmercury, but only from eating fish that are relatively low in methylmercury. The differences in the estimated results (likelihood of adverse effects predicted by one model but not the other) was due` to the types of fish, i.e., to higher methylmercury fish being consumed in one scenario but not in the other.
2. The scenarios were developed only for methylmercury and for commercial sources and thus are limited to the point where they do not match consumer needs (e.g., whether to eat fish or red meat or soy, etc.).
The scenarios were limited in that respect. Moreover, they estimate population shifts in neurodevelopment, or in cardiovascular health, and may not apply equally to each individual within the population.
3. In the "what-if" scenarios the discussion of the impacts are "timid," e.g., the upper bounds of the uncertainty are given more weight than seem sappropriate (e.g., the reader is not given enough guidance as to what is plausible and what is not).
The text includes a brief explanation of central estimates and confidence intervals for readers who are not familiar with these terms. While the upper bounds of a confidence limit is not likely to represent the true value, we assert that they are within the bounds of plausibility.
4.The "what-if" results for CHD need additional explanation of the assumptions and findings. Is it correct that the results do not apply to fish consuming individuals who simply consume more/less fish?
The results are population estimates of how many additional deaths would be averted or would occur if everyone in a particular hypothetical (or some percentage of the population in some hypotheticals) ate more or less fish. The text on the "what-if" scenarios has been reworked for clarity.
Question about one particular table:
Figure AA-11 in Appendix A has both IQ and delayed walking. Are the units the same?
Yes. This question is about units of measurement used in one table. The units are the same because we converted delayed walking to an IQ scale by first normalizing it to a Z-Score and then multiplying by 15 to convert it to IQ.
Adequacy of the discussion of scientific studies:
1. The results of Daniels et al. (2004) should be further discussed in the text. The levels of exposure to methylmercury are likely to be relevant to U.S. populations and the findings show net positive benefits.
The results from Daniels et al. (2004) were used in the assessment for fetal neurodevelopment and are covered in Sections III, IV, Appendix A, and Appendix D. With regard to levels of exposure to methylmercury in that U.K. population, we agree with the peer reviewer's point and have included a table that estimates exposures through the 99th percentile and compares them to U.S. exposures (Appendix D). The estimated U.K. and U.S. exposures match quite closely, even though the U.K. population appears to eaten more fish than are typically consumed in the U.S. This suggests that the fish eaten by the U.K. population were low in methylmercury. It should be noted that Daniels et al. (2004) is also discussed in the "Summary of Published Research" document.
2. More detail should have been provided on the studies that looked for associations between methylmercury and risk of coronary heart disease.
These studies are now addressed in a table.
3. Provide more quantitative detail about the extreme exposures that occurred in Japan and Iraq.
The current version of the draft has been shortened for readability and now contains less detail on the contents of literature accounts of these incidents and of other research studies than it did previously. The FDA report is intended to be a short summary in support of the risk and benefit assessment. Details can be found in the literature.
4. Were there cases in Japan and Iraq where the fetus was not more sensitive?
Yes, there were cases where the mother was more adversely affected than her child, but the trend was substantially in the other direction. The report states that the poisoning incidents in Japan and Iraq provide the key evidence in support of fetal sensitivity. The current version of the report does not go into detail about the data on the percentage of mothers who were more adversely affected than the fetus, since it is not relevant to the assessment.
5. Provide size of the study populations in the table that inventories the research described in current section III on fetal neurodevelopment.
The sizes of the study populations have been added to tables IIIA-1 and IIIA-2.
6. Clarify whether there was any population overlap between the two Oken et al. studies and add population sizes.
Population sizes have been added but extent of population overalap is a level of detail that no longer exists in this report on the research results.
Call it a risk and benefit assessment, not a risk assessment:
Change the name of Section IV from "Scientific Basis for Risk Assessment" to "Scientific Basis for Risk and Benefit Assessment."
We have essentially made this change.
Non-fatal coronary heart disease and stroke:
1. The assumption that benefits of fish for non-fatal coronary heart disease are proportional to benefits for fatal coronary heart disease is not justified.
The risk and benefits assessment only modeled fatal events for coronary heart disease and stroke. We included a brief discussion that speculated what the effects of fish consumption would be on non-fatal events if the effects were proportional to those for fatal events.
2.Explain whether the ratio of non-fatal to fatal stroke occurrence was age adjusted.
The report no longer contains a ratio of fatal to non-fatal stroke occurrence. See 1., above.
Assumptions and related text about methylmercury in the modeling for fatal coronary heart disease and stroke:
1. For modeling fatal coronary heart disease and stroke, what are the implications of the assumption that all fish in the studies used in the assessment contained methylmercury and that exposure to methylmercury correlated with fish ingestion?
The studies used in the assessment of fatal coronary heart disease and fatal stroke involved the consumption of fish where the exposure to methylmercury was not measured. We no longer refer to the presence of methylmercury in those fish as an assumption. There is universal agreement as reflected by the published literature that methylmercury can be found in all fish and that exposure correlates with fish consumption. As a general rule, the higher the level of fish consumption, the higher the level of methylmercury exposure. As a result, the estimates of net effect include the impact of methylmercury.
2. The statement that FDA assumes that the highest percentiles of fish consumption, which the coronary heart disease model associates with the lowest risk, tend to involve the highest levels of methylmercury exposure, is a major assumption and warrants additional evaluation.
See response to 1., above. However, the sentence in question has been dropped as not necessary to support the results of the modeling.
3. Provide references for the statement that FDA has no data to suggest methylmercury levels in the U.S. marketplace are higher than elsewhere.
We should have provided references at that point. They would have been the same references as cited on page 83 of that draft of the report. In any case, the statement in question has been deleted.
Use of biomarkers to measure effects in studies of coronary heart disease:
Clarify the meaning of "biomarkers" in the inclusion/exclusion criterion for studies on coronary heart disease and stroke. The criterion excludes studies that measured effects only in terms of biomarkers,
The criterion excludes studies that do not measure effects in terms of coronary events, but that only measure effects in terms of biomarkers such as levels of cholesterol. The criterion has been clarified to exclude studies that measured effect only in terms of biomarkers, rather than coronary events.
Gender differences in death rates for coronary heart disease:
Question whether gender differences in rates of coronary heart disease in the 45-50 year age group are relevant for calculating gender differences in rates of coronary heart disease sex rates in the 15-44 year age group.
We have no data that differentiates death rates by gender in the 15-44 year age group. The closest we have to it is data that differentiates rates by gender in the 45-50 age group, so we applied that differentiation to the 15-44 age group. Coronary heart disease in the 15-44 year age group is low, so the effect of using the 40-50 year age group data to differentiate the low rates between men and women in the 15-44 year age group should be minor.
U.S. vs Non-U.S. data in the studies used in the modeling for coronary heart disease:
Would excluding non-U.S. data for use in the assessment of coronary heart disease be justified? Consider a sensitivity analysis with and without non-U.S. data.
The studies used for coronary heart disease are fairly well divided between U.S. and non-U.S. studies. We have appreciated this division as including a wide range of possible lifestyles and variations that could be present in the United States. The idea of performing a sensitivity analysis that compares the U.S. results against non-U.S. results is intriguing and one that we are considering.
Male vs. female data in the studies used for modeling stroke:
Seven of 10 studies used in the stroke modeling are for males only. How was this handled?
In developing separate dose-response functions for males and females we considered whether to weight the studies by gender, i.e., when developing a dose-response function for men we would give the male studies more weight than the other studies and when developing a dose-response function for females we would assign more weight to the female or mixed studies than to the males-only studies. Ultimately, we decided not to weight the studies because we could not decide on how much weight to assign in each case. However, for purposes of sensitivity analysis we did assign weights to the studies and developed alternate dose-response functions to determine whether weighting would significantly affect the results. The weights were assigned somewhat arbitrarily given the previous concern about how much weight to assign. The differences in the weighted dose-response functions versus the unweighted ones turned out to be small. We have added a discussion of this point in Appendix A of the text.
Reduction of net beneficial effect is not the same as net adverse effect:
Redraft to say that reducing benefits is not the same as net harm.
We have reviewed the document for clarity on that point and have modified text where necessary.
Safety assessment does not take benefits into account:
Redraft to point out that safety assessment does not address what constitutes adequate fish consumption.
The materials on safety assessment have been deleted from the text in order to shorten and streamline the document, but the peer reviewer's point is valid.
Different forms of Mercury:
1.Always refer to "methylmercury" in the report when that is what is meant. In some cases the report just refers to "mercury."
Researchers nearly always measure total mercury in hair and blood rather than methylmercury, and then often assume that the results essentially represent methylmercury. Where the researchers measured total mercury, or the data we are working with involve total mercury, we use the term "mercury," rather than methylmercury. We recognize that the distinction between the two can be a source of confusion so we try to make clear how and why we are using both terms.
2.Explain the difference between ethylmercury and methylmercury.
The report emphasizes that methylmercury is the only organic form of mercury that is addressed by this risk and benefit assessment. No other organic form is mentioned. A discussion of ethylmercury would have involved a lengthy digression on a subject that is not relevant to this report.
Ages of first walking and talking:
The central estimates for age of first walking and talking are inaccurate.
The central estimates are based on data from the Seychelles and probably do not accurately represent the U.S. median. We regard the central estimates as being close enough to enable the reader to compare the estimated length of delay (usually less than a day) against the ages that children typically begin walking and talking.
Significance of estimated effects:
1. Are the estimated effects clinically meaningful? If not, then should predictions of less than one IQ point or a delay of less than a day be defined as "no effect?"
The clinical significance of the findings is beyond the scope of this report.
2. There are probably population differences larger than any likely adverse impact of methylmercury in these populations and it would be helpful to make some note of this in the report.
For differences within a population, we did not research how the size of the effects estimated by the fetal neurodevelopment modeling compares to such differences. For differences among populations in performance on different tests, the variation (i.e., the standard deviation) may still be treated as a global measure. We have added text and a reference (WHO 2006) on this point.
Question about IQ effects estimated by Axelrad et al. (2007):
The report states that the Axelrad et al. (2007) IQ estimate is employed in our analysis as a normal distribution. Explain why it was assumed to be normal.
A normal distribution is an assumption made by Axelrad et al. (2007). We simply used their function. We have added text to state that the normal distribution was "per Axelrad."
Discussion of diet-blood relationships:
In Appendix A there is a discussion of diet-blood relationships and results that could be noted in more detail in the text.
We have added some additional material in the text in Section IV, although most of the descriptions remains in the Appendix due to its technical nature.
Sufficiency of documentation on the technical details of the modeling:
The documentation accompanying the Excel files that constitute the underlying analysis for the modeling was insufficient to allow a detailed analysis of the model.
We included on a disc the computer programs with explanations that were used in the modeling. This comment refers to the sufficiency of the explanatory materials. Different reviewers who wish to replicate and run the model have different questions about various aspects of it. It would take a significant amount of text to anticipate all questions. Consequently, we attempt to provide a basic explanation and then respond to questions that still might arise after a reading of the basic explanations. In this case the reviewer took some educated guesses as to what was meant and in each case the reviewer was correct. However, in response to the comments, we have modified the computer disc to add an overview description of the Excel files comprising the model and the supporting analysis. The disc will be made part of the public record and members of the public who have questions about any aspect of it are welcome to contact FDA.
Appendices
Comments were offered on Appendices E through I.
These appendices have been deleted from the text in order for the risk and benefit assessment to be the primary focus. The appendices covered a range of topics not directly related to the results of the assessment.
Autism:
The report should address autism as a health endpoint.
This is a difficult, important, and sensitive subject. We gave a lot of thought to this request but decided that since this is an assessment of health endpoints for which methylmercury in fish is a potential risk factor based on findings from research studies in the peer reviewed scientific literature, autism would not be a relevant topic.

PEER REVIEW CONCERNS and FDA RESPONSES: REPORT CONTAINING SUMMARY OF PUBLISHED RESEARCH on OMEGA-3 and FISH BENEFITS

Concern
Response
Requests for additional papers to be reviewed in the report:
1. For cardiovascular effects: two papers on clinical trials on effects of omega-3fatty acids: JELIS and GISSI-HF.
The JELIS and GISSI-HF papers were added.
2.For fetal neurodevelopment, Helland et al., 2003; Dunstan et al., 2008
The paper by Helland et al., 2003 was already covered. The paper by Dunstan et al, 2008 was added.
Hierarchies of Evidence:
Add a better description of the hierarchies of evidence. The report seems to value them in reverse order of their real value.
FDA addresses hierarchies of evidence in the context of health claims for foods in its Guidance for Industry, Evidence-Based Review System for the Scientific Evaluation of Health Claims. Section III of that guidance describes how FDA categorizes studies by type.
Randomized Clinical Trials in the Quantitative Assessment:
Data from randomized clinical trials (RCT's) on fish oil and neurodevelopment or risk of coronary heart disease should be added to the quantitative assessment. The idea that RCT's of fish oil somehow do not inform about the effects of fish consumption is disingenuous. RCT's of omega-3's and post-natal exposure are powerfully confirmatory and we should say so and use them.
The creation of a companion, stand-alone document that covers the scientific evidence regarding cardiovascular and neurodevelopmental benefits, including RCTs, should provide RCTs with a stature in terms of presentation that was lacking in the version of the report reviewed by the peer reviewer.
Account of Cardiovascular Disease:
The body of the report gives short shrift to CVD. More attention is needed to a full account of this embarrassingly rich data in addition to that in the Appendix, explicitly detailing the relative sizes, strengths, and robustness of these studies, compared with the MeHg and heart disease data.
A companion, stand-alone document covering the scientific evidence regarding cardiovascular and neurodevelopmental health benefits was created in place of the corresponding Appendices in the previous draft. Having a stand-alone document gives more prominence to the lines of scientific evidence for the cardiovascular benefits. Additionally, a summary of the cardiovascular evidence, in bullet form, was added at the end of the cardiovascular benefits section, and a new introductory summary was added. The summary bullets emphasize the characteristics of the evidence noted by the reviewer.
Role of omega-3 biomarker studies of cardiovascular health benefits:
Observational biomarker studies of association of omega-3 fatty acids and CHD death should be mentioned.
Summaries of observational studies of omega-3 fatty acid biomarkers and CHD risk were added in the section on the U.K. Scientific Advisory Committee on Nutrition (SACN). Estimates of the dose-response relationship between omega-3 fatty acid intake and biomarker levels were added in the section on the AHRQ reports. Additional information on omega 3 biomarkers was added in the new section on measurement error.
RCTs of fish oil for decreased CHD risk are underemphasized.
RCTs of fish oil for decreased CHD risk provide some of the strongest evidence and should be especially emphasized.
The discussion of RCTs of fish oil for decreased CHD risk was expanded and updated to include the three trials of patients with cardiac defibrillators, as well as the large JELIS study. A new table was added, Table AC-9, highlighting updated meta-analyses of RCTs, and this was covered in the summing up bullets.
RCTs of fish oil or omega-3s for neurodevelopment are underemphasized.
RCTs of fish oil or omega-3s for neurodevelopment provide some of the strongest evidence and should be especially emphasized.
The discussion of RCTs of DHA for neurodevelopment, both prenatal and postnatal, shows that this body of evidence is mixed, rather than consistent and strong. The summary and conclusions of the neurodevelopment benefits section evaluates and identifies the lines of evidence from RCTs and observational studies that support the neurodevelopmental benefits of fish or fish oil. A new introduction also highlights this summary information.
Adequacy of the review of scientific studies:
1. The results of Daniels et al. (2004) should be further discussed in the text. The levels of exposure to methylmercury are likely to be relevant to U.S. populations and the findings show net positive benefits.
The results of Daniels et al. are covered in the companion report in the text and in Table AD-13.
2. More detail should have been provided on the studies that looked for associations between methylmercury and risk of coronary heart disease.
The summary of the studies on mercury and heart disease risk was expanded and a second table was added (now Tables A-7 and A-8).
Hooper Meta-Analysis:
We should not have included the Hooper meta-analysis in considering coronary heart disease. Its major limitations should be further emphasized.
We included a discussion of this meta-analysis in the benefits review companion document because it exists and thus should be addressed. Because of the controversial nature of that meta-analysis we include the criticisms that have been raised about it in other literature sources that we also review. A new table was added, Table A-9, highlighting meta-analyses of RCTs that update the Hooper et al meta analysis, and this was covered in the summing up bullets.
Self-reported fish intake versus biomarkers:
Misclassification or measurement error of estimated fish intake compared to methylmercury biomarkers should be addressed.
This comment points out that we could be underestimating benefits associated with fish consumption relative to adverse effects associated with methylmercury exposure because in the studies we use, exposure to fish was measured by self-reported fish consumption, i.e., dietary intake questionnaires, while exposure to methylmercury was measured by biomarkers, i.e., levels of mercury in blood and hair. Biomarkers are regarded as more accurate measures, although they can also be subject to some error. We added a discussion of measurement error in the companion document.
Discussion of substitution among food groups
In the section on Dietary Guidelines for Americans, "Interestingly" should be replaced by "As expected" in the text and the limitations of arguments about the effect of substitution of fish for meat, poultry and other food groups should be described.
This discussion was expanded slightly for clarity and the term, "interestingly", was dropped.

Page Last Updated: 11/17/2014
Note: If you need help accessing information in different file formats, see Instructions for Downloading Viewers and Players.