Elemental Analysis Manual: Section 3.5 Reference Materials
Version 1 (June 2008)
Authors: William C. Cunningham
Stephen G. Capar
Table of Contents
Reference materials are used for method validation, verification of correct use of a method, calibration and quality control. Quality control is FDA's primary use of RMs. The sections below provide information on the use of RMs. Additional information is available1-3.
- Reference Material (RM) — Material or substance one or more of whose property values are sufficiently homogeneous, stable, and well established to be used for the calibration of an apparatus, the assessment of a measurement method, or for assigning values to materials4.
- Certified Reference Material (CRM) — Reference material, accompanied by a certificate, one or more of whose property values are certified by a procedure which establishes its traceability to an accurate realization of the unit in which the property values are expressed, and for which each certified value is accompanied by an uncertainty at a stated level of confidence4.
- In-house Reference Material (in-house RM) — Reference material developed by a laboratory for its own internal use.
CRMs and in-house RMs are simply types of RMs. A CRM is a RM with an associated certificate that satisfies traceability requirements. When a certificate expires, the material in the unit (or container) associated with that certificate continues to be an RM but is, by definition, no longer traceable and therefore no longer a CRM. Standard Reference Material (SRM) is a trade name that the U. S. National Institute of Standards and Technology (NIST) uses for a CRM.
RMs are analyzed with a batch of samples to verify the accuracy and overall performance of the analysis. CRMs provide traceability and are from producers such as the NIST and the National Research Council of Canada. All other non-certified RMs, including expired CRMs and RMs with established consensus values (e.g., Roelandts and Gladney5), can be used for demonstrating the repeatability aspects of an analysis. CRMs may be used in any RM application but must be included, alone or in combination with non-certified RMs, in regulatory analyses. CRMs are typically not required for investigational and surveillance analyses.
Follow instructions on the certificate for the recommended minimum analytical portion, the procedure to use for determining basis weight, storage requirements, etc. The use of z-scores (see §3.5.3) is an accepted procedure for demonstrating the quality of results.
RMs are chosen to closely match the matrix and analyte concentration of interest. However, choosing an appropriate RM is often difficult because of the relatively small variety of RMs available. For FDA labs, the difficulty in choosing an appropriate RM is compounded by the large variety of food matrices and numerous analytes. The following sections provide guidance in obtaining or preparing RMs.
An in-house RM is usually developed when the matrices or reference levels of commercially available RMs do not closely match the samples to be analyzed. The steps for development of an in-house RM are outlined below.
Ideally, the material will be available in an ample supply, with minimal cost, and needing little or no preparation. Materials requiring freezing, pulverizing, sieving, blending, sterilization, packaging, etc., should be selected only if necessary.
Logically, the material would be expected to be homogeneous and stable with a long shelf life. Refrigerated storage may be needed. Analyte levels, interference issues, and analysis difficulty need to suit the purpose of the RM. Challenging materials having interferences or digestion complications are generally undesirable but in some circumstances these characteristics may be needed to demonstrate ruggedness.
Analyses provide data for defining the weight basis procedure and for setting analyte levels, minimum analytical portion mass, and uncertainties.
The weight basis is a defined, reproducible condition for an analytical portion to obtain its mass. Typical conditions used include freeze-dried, oven-dried, as received, desiccator-dried, equilibrium mass state, and reconstituted. Analyte concentrations are best established using a variety of analytical techniques and methods in different laboratories by different analysts. Use of multiple sets of data in this way compensates for laboratory bias. Limits on laboratory bias may be determined or estimated based on analytical data provided by the different techniques, methods, and laboratories6-7. For each analyte, the reference value should be established using a combination of at least 2 different analytical techniques or laboratories. The reference values may be straight averages or weighted averages, depending on the specifics of the data sets. Determination of which procedure to use for setting the reference values should be made under consultation with a statistician.
When an in-house RM is to be used solely for measuring repeatability and the analytical method, laboratory, and analyst will always remain the same, then technique bias is of no consequence. In this case, analyte reference values are applicable only to the given method, laboratory, and analyst.
22.214.171.124 Random Error and Homogeneity
Measuring the RSD for the minimum analytical portion mass to be used routinely is recommended. Under these conditions, RSD represents the total random error that accounts for errors from the measurement and errors due to analyte nonhomogeneity. Total random error is needed for generating uncertainties. Note that homogeneity depends on the mass of the analytical portion.
If nonhomogeneity information is needed, such as for determining whether the RM may be suitable for another application, compare the RSD with the method's relative standard uncertainty for the measurement. When the RSD is smaller than the relative standard uncertainty then the latter may have been overestimated. When an RSD is less than or equal to the relative standard error for the measurement, homogeneity is better than the method can detect for that analyte. When the RSD is greater than the relative standard uncertainty the effects of nonhomogeneity have been observed.
Nonhomogeneity can be defined as the RSD from analyte variations within the material, RSDnonhomogeneity, and expressed as relative percent. The total RSD, RSDobserved, is assumed to be related to nonhomogeneity and random variations due to the measurement process, RSDrandom error8, and expressed alternatively as:
When random error does not adequately account for RSDobserved, the additional variability is assumed to be due to nonhomogeneity. Nonhomogeneity is therefore set equal to the square root of the difference obtained by subtracting the random error squared from RSDobserved squared. Random error is estimated using random uncertainty, the combined random components of measurement uncertainty.
The adequacy of random error alone to account for an observed data distribution can be evaluated on the basis of the integral of the distribution function Px(χ², ν) from x2 = χ²(observed) to x2 = ∞, where χ² is chi-square distribution and ν is the number of degrees of freedom9. Here, the integral of the distribution function is referred to simply as probability. When the probability is ≤10%, a nonhomogeneity component can be calculated.
In general, nonhomogeneities that are equal to the associated random measurement uncertainties would be expected to have probabilities <10%. Therefore, when a probability is >10%, the nonhomogeneity is known to be less than the random measurement uncertainty and the latter can be taken as an upper limit for nonhomogeneity. However, when RSDobserved is lower than the random measurement uncertainty, RSDobserved is taken instead as an upper limit for nonhomogeneity.
Uncertainties need to be generated for RMs so they can be used in combination with the analytical uncertainties from measurements to demonstrate accuracy. Uncertainty limits should be computed using a 95% confidence level. The statistical methods for determining uncertainties depend on the data sets and associated analytical information. As the methods vary and can be non-trivial, the assistance of a statistician is recommended.
A document should be prepared that provides instructions on using the in-house RM including storage requirements, procedure for determining weight basis, minimum analytical portion mass, and estimates of analyte(s) level(s).
Re-verification, the process that shows a RM is still fit for purpose, is based on observations and analytical results. If only selected elements of interest are re-verified, the re-verification will only apply to these elements. Also, re-verification applies only to the unit (or container) being tested and unopened units whose physical integrity is unquestionable. Analytical results may be obtained specifically for re-verifying the RM for specific elements or generated during routine analysis of the RM.
RMs do not typically have expiration dates but the levels for analytes of interest must be re-verified annually or with each use of the RM. Re-verification is accomplished using the RM uncertainties and the analytical uncertainties associated with the current measurements. Measurement uncertainties are ideally determined along with the measurements but when they are not, uncertainties may be assigned for well-defined methods (such as 10% for EAM methods) or set to zero.
Continuous monitoring of RM results is useful because changes in element levels can be observed in a timely fashion. Maintaining a plot of element levels over time is useful for observing trends. A change rate that would predict an element level exceeding the material's uncertainty within the following year would be very significant.
Use visual inspection to verify the absence of evidence that would cause one to question a RM unit's physical integrity. For example, change in color, presence of mold or seeing liquid when the material should be dry would disqualify an RM unit.
Analyze at least 2 analytical portions of the RM being re-verified concurrently with at least one analytical portion of a CRM. Compare the current CRM and RM results with the certified and reference values, respectively, by using z-scores10. A z-score is equal to the difference between the result and certified value divided by the square root of the sum of the squares of the uncertainties from both the reference and measured values (see Explanatory Note below). Use absolute values for z-scores and interpret as:
|z-score| of 2 or less is acceptable (result in agreement with reference value)
|z-score| between 2 and 3 is questionable (result in questionable agreement with reference value)
|z-score| of 3 or more is unacceptable (result in disagreement with reference value)
Results indicate successful re-verification for an element if at least two-thirds of the z-scores are in the acceptable range and none are in the unacceptable range. Thus, when only 1 or 2 analytical portions of a CRM or RM are analyzed, every z-score must be in the acceptable range. When 3 analytical portions are analyzed, at least 2 of the z-scores must be in the acceptable range and one may be in the questionable range. For 4 or 5 portions, only one can be in the questionable range, etc.
If successful re-verification is obtained, then extend the expiration date of the RM unit to one year from the analysis date and document the elements for which the extension applies.
If re-verification is unsuccessful, then ensure method performance was satisfactory. Mistakes such as data entry or calculation errors may need only correcting. Analytical problems may require repairing equipment or obtaining new reagents. Unexplained findings may require reanalysis. Repeated failure to re-verify may indicate a faulty RM or CRM unit.
Explanatory note about z-scores
A z-score10 indicates how many standard deviations an observation is from the mean or reference value. Use of a z-score to examine data quality is a standardized way to evaluate results and provides an additional perspective besides that given by recoveries, which are based on reference concentration values, or that given by precision, which is based on RSD of concentration measurements. For this application, the z-score is defined as:
|xm =||measured analyte level|
|xc =||is the reference level|
|σm =||combined uncertainty (one sigma, corresponding to a confidence level of approximately 67%) of the measured level|
|σc =||combined uncertainty (one sigma) of the accepted level.|
When σm is not determined for a measurement, it may be assigned based on experience. This generally includes generating an uncertainty budget and using past performance data.
Absolute values of z-scores of ≤2, between 2 and 3, and ≥3 are used as indications of agreement, questionable agreement, or disagreement, respectively, between measured values and reference values (i.e., certified or consensus values).
- National Institute of Standards and Technology (NIST)
- National Research Council of Canada (NRC)
- Institute for Reference Materials and Measurements (IRMM)
- European Reference Materials (ERM) [Partners: Institute for Reference Materials and Measurements (IRMM) of the European Commission's Directorate General Joint Research Centre, Belgium; Bundesanstalt für Materialforschung und -prüfung (BAM), Germany; LGC, United Kingdom]
- National Institute for Environmental Studies (NIES)
- International Atomic Energy Agency (IAEA)
- Resource Technology Corporation (RTC) [USA Distributor]
126.96.36.199 FDA Cocoa Powder (CP)
Current certificate of analysis: FDA Cocoa Powder Certificate (2012, amended 2013) Excel 2010 (.xlsx) file
FDA CP Certificate (2012) was issued May 16, 2012. With this issuance, FDA CP was revalidated for continued use as an in-house RM for quality control/quality assurance for determination of element mass fractions in food and other biological materials.
Development work for FDA CP started in 1994 when approximately 10 kg of commercially-produced cocoa powder from a single production lot was obtained from a local grocery store. The material was re-packaged into pre-washed amber glass bottles and has been stored at FDA CFSAN. Bottles are sent to FDA labs on request. Each bottle contains about 300 grams of cocoa powder.
Reference values are derived from analytical results that have been reported. Since data have been received on an on-going basis, the reference values have evolved over time. The first FDA CP certificate was issued August 9, 1996. Minor updates occurred in 1998, 1999, and 2000 and a major update occurred May 4, 2006. As expected, changes in numerical values have been quite small.
Several enhancements were realized with the 2012 certificate:
The certificate is presented electronically in the form of an Excel 2010 workbook named "FDA Cocoa Powder Certificate (2012).xlsx". It replaces the traditional hard-copy certificate but one may be obtained by printing the first three worksheets. The workbook is password-protected with changes permitted only in specific cells designed for user input. Calculations are therefore protected from being altered inadvertently.
Basis mass is relative to the material exposed to air at 30% relative humidity, which is expected to be the median condition for typical FDA laboratories. An analytical portion is to be exposed to laboratory air for at least 2 hours before measuring the mass. If a mass uncertainty of ±1% is acceptable, then no further calculation is needed. If ±0.5% is desired, an adjustment may be performed using the actual laboratory relative humidity. This calculation can be done automatically using a tool within the certificate.
The data set used to derive the reference values has expanded considerably. Data are now available for 43 elements from 17 sources (methods, laboratories, techniques, etc.). Detection limits are provided for 6 additional elements.
Reference values were set using state-of-the-art "concensification" software developed at NIST for generating SRM certificate values. Values are provided for Confirmed Values (those confirmed via multiple sources) and Unconfirmed Values (those from a single source).
Reference values are provided in duplicate listings. One listing applies for the bulk material and is useful for evaluating means obtained by averaging results from replicate analyses. The second listing applies for individual analytical portions and is useful for evaluating individual analysis results. The former is typical for CRM documents while the latter includes sample-to-sample non-homogeneity effects.
Users select the coverage factor for displaying uncertainties. For example, by entering the number one (1), the reference values will have standard uncertainties (1-sigma or about 67% confidence); or, by entering the number two (2), the reference values will have uncertainties at about 95% confidence, which is typical for commercial RM certificates.
The certificate has a z-score evaluation capability. When a user enters analysis uncertainty (e.g., 10%), ranges are automatically generated to show the acceptable and questionable ranges for results. And, a hard-copy z-score evaluation may be generated to include with a report of analysis, a user needs only to enter the individual results for an analysis.
- European co-operation for Accreditation , Eurolab and Eurachem (EEE) Working Group on Reference Materials (2003) The Selection and Use of Reference Materials. EA 4/14 (rev00). Online: (March 31, 2008).
- Lawn, R., Roper, P., Holcombe, G., and Stuart, B. (2001), Online: Application Notes for the Production of Low-Cost Quality Control Matrix Reference Materials . (December, 2005).
- Brookman, B., (1998), Guidelines for the In-House Production of Reference Materials - Version 2. Available from: National Measurement System - Chemical and Biological Metrology .
- International Organization for Standardization (1992) ISO Guide 30: Terms and definitions used in conjunction with reference materials. International Organization for Standardization, Geneva, Switzerland
- Roelandts, I., and Gladney, E. S. (1998) Consensus values for NIST biological and environmental standard reference materials. Fresenius' J. Anal. Chem. 360, 327-338.
- Paule, R. C., and Mandel, J. (1982) Consensus values and weighting factors. J. Res. of National Bureau of Standards 87, 377-384.
- Schiller, S. B., and Eberhardt, K. R. (1991) Combining data from independent chemical analysis methods. Spectrochim. Acta 46B, 1607-1613.
- Keith, L. H., Crummett, W., Deegan, J., Libby, R. A., Taylor, J. K., and Wentler, G. (1983) Principles of Environmental Analysis. Anal. Chem. 55, 2210-2218.
- Bevington, P. R. and Robinson, D. K. (1983). Data Reduction and Error Analysis for the Physical Sciences, 3rd edition pp 195-197. McGraw-Hill, New York.
- Thompson, M., and Wood, R. (1993). The international harmonized protocol for the proficiency testing of (chemical) analytical laboratories. Pure Appl. Chem. 65, 2123-2144.