XVI. Appendix to Class II Special Controls Guidance Document: Antimicrobial Susceptibility Test (AST) Systems

  TABLE 1: Recommendations for Antimicrobial Susceptibility Devices^a 

		MIC/BP Formats	Fastidiousb	Additional methods of Inoculation/Readingc
Number of Sites (including 1 in-house)		3	3	3
Organisms	Freshd Clinical/stocke	100/site	75/site	0
	CDC Challengef	75/one site	50/one site	75 or 50/one site
Reproducibilityg		25/site or 10x3x3/site	25/site or 10x3x3/site	25/site or 10x3x3/site
Interpretive Standards		FDA	FDA	FDA
Stability (3 lots)		Real time (on file)	Real time (on file)	Real time (on file)
QC Reference and Test Device Results	CLSI Strains	20 results/site	20 results/site	20 results/site
	(Other Mfg. Recommended)	Optional	Optional	Optional
	On-scale	At least 1	At least 1	At least 1
	Inoculum density checkh	QC, reproducibility, fresh	QC, reproducibility, fresh	QC, reproducibility
CLSI Reference Method		MIC	MIC	MIC

1. See Tables 5, 8, and 9 for statistical calculation examples and evaluable results.
2. For Fastidious organisms such as Streptococcus, Haemophilus, anaerobes, etc. that have an CLSI approved standard methodology, FDA interpretive criteria and quality control recommendations, refer to CLSI approved standard M100 4 Table 1A. The routine testing of rare isolates is not recommended.
3. Minimal data to establish performance should be presented for each procedural option of the method of inoculation (growth, direct colony suspension etc.), reading of results (visual vs. automated), or combinations of options.
4. Fresh clinical - an organism isolated from a clinical specimen which has been on an agar plate for less than 7 days and not frozen.
5. Stock organisms - any organism from a clinical specimen which has been isolated greater than 7 days prior to testing. Should not include organisms for which the antimicrobial agent is not intended. Selection should be supplemental based on the listing in the FDA approved pharmaceutical antimicrobial agent package insert and should not comprise more than 50% of the clinical isolates.
6. Challenge - CDC or reference laboratory source with known results (preferably on-scale) to be tested on the test system. Organisms that are intended for the testing with the antimicrobial agent as stated in the FDA approved pharmaceutical antimicrobial agent package insert (microbiology section), should be selected for testing on the test device.
7. All on-scale results.
8. Inoculum density check should be performed daily on the QC isolates, on reproducibility isolates, and 10% of fresh isolates. Alternate approaches may be substituted if the inoculum method uses a spectrophotometric device.

  TABLE 2: Recommendations for the Removal of Limitations from Antimicrobial Susceptibility Devices^a

Items		Performance	Insufficient Resistant Strains	Quality Controlb	Reproducibility
Number of Sitesc		3	1	3	3
Organisms	Fresh or Stock Clinical	100/site	75d	0	NA
	Challenge	75/one site	as needed	0
Reproducibilityae		NA	NA	NA	25/site or 10x3x3/site
Quality Control		Daily	Daily	20	Daily

1. For Statistically evaluable numbers see Tables 5, 8 and 9
2. To be used to demonstrate that the QC ranges are now in the same ranges as the FDA/CLSI
3. One may be in-house
4. A minimum of 75 organisms either resistant or clustered near the susceptible threshold.
5. On-scale

Note:

1. If changes have been made to the device to alter the overall performance, the testing should include all organisms previously tested.
   Refer to Table 1.
2. Perform testing for all procedural options.

  TABLE 3: Presentation of Summary Data for Both Challenge and Clinical Data

Clinical Data

Organism Group	Total Tested	# EA	%EA	Total Evaluable	# EA of Evaluable	%EA of Evaluable	# CA	%CA	# R	# vmj	# maj	# min
K. pneumoniae	79	74	93.7	64	62	92.2	70	88.6	20	0	1	8
P. aeruginosa	96	90	93.8	91	85	93.4	88	91.7	40	0	1	7
C. freundii	26	21	80.8	18	14	72.2	23	88.5	10	0	0	3
E. aerogenes	22	21	95.5	21	21	95.2	21	95.5	2	0	0	1
E. cloacae	57	53	93	50	48	92	51	89.5	13	1	1	4
M. morganii	15	14	93.3	12	12	91.7	15	100	7	0	0	0
P. mirabilis	34	26	76.5	34	29	76.5	34	100	33	0	0	0
E. coli	92	85	92.4	62	59	88.7	87	94.6	43	2	1	2
S. marcescens	50	49	98	48	48	100	47	94	32	0	0	3
Acinetobacter spp	41	40	97.6	29	28	96.6	34	82.9	14	0	0	7
TOTAL	512	473	92.4	429	406	94.6	470	91.8	214	3	4	35

Challenge

Organism Group	Total Tested	# EA	%EA	Total Evaluable	# EA of Evaluable	%EA of Evaluable	# CA	%CA	# R	# vmj	# maj	# min
K. pneumoniae	10	10	100	10	10	100	10	100	6	0	0	2
P. aeruginosa	20	18	90	17	15	88.2	19	95	15	1	0	0
C. freundii	10	10	100	9	9	100	9	90	2	0	1	0
E. aerogenes	5	4	80	4	3	75	5	100	0	0	1	1
E. cloacae	5	5	100	2	2	100	5	100	4	0	0	1
Aeromonas sp.	5	4	80	4	4	100	5	100	0	0	0	0
P. mirabilis	10	9	90	8	6	75	9	90	4	0	0	2
S. marcescens	10	10	80	9	9	100	9	90	2	0	0	1
Acinetobacter spp	10	8	80	5	5	100	8	80	2	1	0	2
TOTAL	85	78	91.8	68	63	92.6	79	92.9	35	2	2	9

Clinical and Challenge Combined

Organism Group	Total Tested	# EA	%EA	Total Evaluable	# EA of Evaluable	%EA of Evaluable	# CA	%CA	# R	# vmj	# maj	# min
All organisms	597	551	92.3	497	469	94.4	549	92.0	249	5	6	44

TABLE 4: Example of Reporting Format for Quality Control Data

Antimicrobial agent: ___________________________

QC Organism	Expected Result		Reference Result Frequency			New Device Frequency
E. coli ATCC 25922	0.25 - 1.0 mg/mL		Site 1	Site 2	Site 3	Site 1	Site 2	Site 3
		<.25				1	1
		.25	14	18	14	4	14	14
		.5	6	2	6	4	5	6
		1.0				11
		>1.0
E. cloacae Ref 1611	2 - 8 mg/mL	<2
		2				12	14	4
		4	14	15	12	2	6	5
		8	6	5	8	8		11
		>8		>	>	>	>	>
Pseudomonas aeruginosa ATCC 27853	0.5 - 2 mg/mL	<.25	>		2			2
		.5	10	10	18		2
		1	5	8	2	20	18	18
		2	5	2				4
		4
Enterococcus faecalis ATCC 29212	4 - 16 mg/mL	2
		4	18	2	18	20	12	6
		8	2	18	2		8	14
		16		>	>	>	>	>

• Performed daily with a minimum of 20 per site.
• List all reference and test results including out of range results that required repeat testing.
• To be used for all procedural options.

TABLE 5: Sample Table Format for Device Performance

Antimicrobic agent: Oxacillin  Test Results  Reference Results   ≤ 0.25a 0.5 1 2 S 4  R 8 ≥ 16 ≤ 0.25 6a a 1       a 0.5 102 100 21 2     a 1 a 10 8 1     a 2 S a 6 11 1     a 4 R a           a ≥ 8 a       a a 31a Evaluable Results   116 41 4	Organism: Staphylococcus aureus Evaluation Overall EAb   108/116 93.1% EA based on evaluable resultsab   3/11 27.3% CA based on interpretationc     100%
Antimicrobic agent: Oxacillin  Test Results  Reference Results   ≤ 0.25 S 0.5 R 1 2 4 8 ≥ 16 ≤ 0.25 S 29a a         a 0.5 R a           a 1 a           1a 2 a   1 1   1 a 4 a       1   a ≥ 8 a     7 12a 12a 52a Evaluable Results     1 8 1 1	Organism: Staphylococcus epidermidis Evaluation Overall EAb   108/116 93.1% EA based on evaluable resultsab   3/11 27.3% CA based on interpretationc     100%
Antimicrobic agent: Oxacillin  Test Results  Reference Results   ≤ 0.25 S 0.5 R 1 2 4 8 ≥ 16 ≤ 0.25 S 25a 1a vmj         1a 0.5 R a 8 3       a 1 a           1a 2 a           a 4 a           a ≥ 8 1a (maj)       2a 3a 26a Evaluable Results   8 3         see footnotes on Table 5A	Organism: other CNS Evaluation Overall EAb   67/70 95.7% EA based on evaluable resultsab   11/11 100% CA based on interpretationc   67/70 100%

Present all fresh and stock results for organisms in a genus/species in a separate diagonal. For example all Staphylococcus aureus results from all sites would appear in the first diagonal.

Antimicrobic agent: Oxacillin  Test Results  Reference Results   ≤ 0.12 0.25 0.5 1 S 2 I 4 R ≥ 8 ≤ 0.12 259a a         a 0.25 a 4 2       a 0.5 6a 4 2       a 1 S a   10 10 5 1 a 2 I a   2 9 10 11 a 4 R a       8 10 a ≥ 8 a         1a 53a Evaluable Results   8 16 19 23 22

Organism: All Evaluation Overall EAb   398/407 97.8% EA based on evaluable resultsab   85/88 96.6% CA based on interpretationc   371/407 100% CA Minord   35/407 8.6% CA Majore   0 0% CA Very Minorf   1/76 1.3%

^a Results are not considered evaluable if they fall in this box

^b Essential Agreement (EA) = (# exact agreement or within plus or minus one two-fold dilution of the reference method / total # organisms tested) X 100

^c Category Agreement (CA) = (# with interpretive agreement to the reference method / total # organisms tested) X 100

^d Minor Discrepancy (min) = (# min discrepancies based on interpretation / total # organisms tested) X 100

^e Major Discrepancy (maj) = (# maj discrepancies based on interpretation / total # susceptible organisms by reference method) X 100

^f Very Major Discrepancy (vmj) = (# vmj discrepancies based on interpretation / totla # resistant organisms by reference method) X 100

To be used for:

• All fresh and stock organisms for each site presented in one 2A diagonal.
• All fresh and stock organisms for all sites combined presented in one 2A diagonal.
• All challenge organisms presented in one 2A diagonal chart.
• All challenge organisms presented in one 2A diagonal chart for each method variation.
• All organisms (challenge, fresh, stock) combined and presented in one 2A diagonal chart. Do not combine different method variations.
  

TABLE 6: Presentation of Reproducibility Results a by Organism

	Difference in the number of wells between test result and test mode
	Off- scale	-2	-1	0	+1	+2	Off- scale	Site 1 result	Site 2 result	Site 3 result	Test Modeb
P. aeruginosa				3				0.5	0.5	0.5	0.5
P. aeruginosa				2	1			4	2	2	2
P. aeruginosa			1	2				16	32	32	32
P. aeruginosa				2	1			8	16	8	8
E. coli				3				2	2	2	2
E. coli			1	2				8	16	16	16
E. coli				2	1			4	2	2	2
E. coli				3				1	1	1	1
E. coli			1	2				2	4	4	4
E. coli				3				32	32	32	32
M. morganii			1	2				16	16	8	16
C. diversus				2	1			0.5	0.5	1	0.5
C. freundii				2	1			1	2	1	1
C. freundii				3				16	16	16	16
E. cloacae				2	1			16	32	16	16
E. cloacae				2		1		0.5	0.5	2	0.5
E. cloacae				3				4	4	4	4
P. mirabilis				3				8	8	8	8
P. mirabilis				3				2	2	2	2
S. marcescens				2	1			2	1	1	1
S. marcescens				2	1			32	16	16	16
S. marcescens				2	1			4	2	2	2
K. pneumoniae				2			1	32	32	>32	32
K. pneumoniae				2	1			0.25	0.5	0.25	0.25
P. stuartii		1		2				16	16	4	16
Total		1	4	58	10	1	1
Between-site reproducibilityc	73/75 = 97.3%d 72/75 = 96%e

^a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 25 on-scale organisms, tested at 3 sites.
^b Most frequent new test result. If there is no mode, the median should be used.
^c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all three results are off-scale they should still be included in the calculation as part of the best-worst case calculations. For this study the denominator would always be 75 whether the results are off or on-scale.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.

TABLE 6A: Presentation of Reproducibility Results^a by Organism and Site

	Difference in the number of wells between new test result and test mode
	Off-scale	-2	-1	0	+1	+2	Off-scale	Test Modeb
				Site 1
E. coli 1			6	3				0.5
E. coli 2				9				2
E. coli 3			6	3				2
Pseudomonas 4				3	6			16
Pseudomonas 5			3	5			1	32
Klebsiella 6				8		1		8
Enterobacter 7				8	1			8
Serratia 8				4	5			4
Serratia 9				5	4			16
Proteus 10			5	4				1
Total			20	52	16	1	1
Within-Site Reproducibilityc	89/90 = 98.9%d 88/90 = 97.8%e
				Site 2
E. coli 1			3	6				0.5
E. coli 2				9				2
etc…

^a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
^b Most frequent test result.
^c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should still be included in the calculation. For this study the denominator would be ninety whether they are all on scale or not.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.

TABLE 6B: Presentation of Reproducibility Results^a by Organism, Pooled Across Sites 

	Difference in the number of wells between new test result and test mode
	Off-scale	-2	-1	0	+1	+2	Off-scale	Test Modeb
All Sites
E. coli 1			9	18				0.5
E. coli 2				25	2			2
E. coli 3		3	9	12	3			2
Pseudomonas 4			6	12	9			16
Pseudomonas 5			2	17			8	32
Klebsiella 6			7	14	5	1		8
Enterobacter 7			5	14	8			8
Serratia 8			4	15	8			4
Serratia 9			3	20	4			16
Proteus 10			3	14	10			1
Total		3	48	161	49	1	8
Between-site reproducibilityc	266/270 = 98.5%d 258/270 = 95.6%e

^a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
^b Most frequent test result.
^c Total number of results that fall within 1 well (+/-1) of the mode result divided by Total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should be included in the calculation with a denominator of 270 for all calculations.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode

  TABLE 7: Report Format for Inoculum Density

ORGANISMa	NUMBER TESTED	SOURCE	METHODb	MEAN	MINUMUM	MAXIMUM	STD. DEV.
S. aureus ATCC #	20	QC ATCC	Reference	6 X 10 5	2 x 10 5	9 x 10 5
S. aureusATCC #	20	QC ATCC	Direct inoculum	5 x 10 5	2 x 10 5	8 x 10 5
S. aureusATCC #	20	QC ATCC	Growth inoculum	5 x 10 5	2 x 10 5	6 x 10 5
MRSA	13	Reproducibility, clinical	Direct inoculum	7 x 10 5	4 x 10 5	6 x 10 6
MRSA	13	Reproducibility, clinical	Growth inoculum	6 X 10 5	2 x 10 5	8 x 10 6
MSSE	3	Reproducibility	Direct inoculum	8 x 10 5	5 x 10 5	7 x 10 5
MSSE	3	Reproducibility	Growth inoculum	7 x 10 5	4 x 10 5	12 x 10 5
MRSE	19	Reproducibility	Direct inoculum	6 X 10 5	2 x 10 5	8 x 10 6
MRSE	19	Reproducibility	Growth inoculum	7 x 10 5	5 x 10 5	7 x 10 5
Enterococcus	4	Clinical	Direct inoculum			9 x 10 5
Enterococcus	4	Clinical	Growth inoculum
MSSA	15	Clinical	Direct inoculum
MSSA	15	Clinical	Growth inoculum
			Direct inoculum
			Growth inoculum

1. Data should be available upon request for “by site” evaluation, by organism, etc.
2. Inoculum density should be performed on all methods of inoculation unless a standardized method (photometric device) is used.

  TABLE 8: Number of VMJ Discrepancies as a Function of the Number of Resistant Organisms Tested

Number of Resistant Organisms	Acceptable Number of Discrepancies	Estimated Ratea	95% Confidence Intervalb for True VMJ Rate
48	0	0.00	(0.00, 7.40)
50	0	0.00	(0.00, 7.11)
60	0	0.00	(0.00, 5.96)
70	0	0.00	(0.00, 5.13)
72	1	1.39	(0.04, 7.50)
80	1	1.25	(0.03, 6.77)
90	1	1.11	(0.03, 6.04)
94	2	2.13	(0.26, 7.48)
100	2	2.00	(0.24, 7.04)
110	2	1.82	(0.22, 6.41)
120	3	2.50	(0.52, 7.13)
130	3	2.31	(0.48, 6.60)
140	4	2.86	(0.78, 7.15)
150	4	2.67	(0.73, 6.69)
160	5	3.13	(1.00, 7.20)
170	5	2.94	(0.94, 6.78)
180	6	3.33	(1.21, 7.16)
190	7	3.68	(1.48, 7.48)
200	7	3.50	(1.40, 7.12)
250	8	3.20	(1.38, 6.24)
300	9	3.00	(1.37, 5.64)
400	11	2.75	(1.37, 4.88)
500	13	2.60	(1.39, 4.41)
600	15	2.50	(1.40,4.09)
700	19	2.43	(1.42,3.86)

1. Est. Rate = estimated vmj rate = number of vmj discrepancies divided by number of resistant organisms.
2. Exact confidence intervals based on the binomial distribution.

  TABLE 9: Essential Agreement as Function of the Number of Evaluable Organisms Tested

Number of Evaluablea Organisms	Acceptable Number of Disagreements	Estimated Essential Agreement (EA)b	95% Confidence Intervalc for True EA
35	0	100.00 %	(90.00, 100.00)
54	1	98.15	(90.11, 99.95)
55	1	98.18	(90.28, 99.95)
60	1	98.33	(91.06, 99.96)
65	1	98.46	(91.72, 99.96)
70	2	97.14	(90.06, 99.65)
75	2	97.33	(90.70, 99.68)
80	2	97.50	(91.26, 99.70)
85	3	96.47	(90.03, 99.27)
90	3	96.67	(90.57, 99.31)
95	3	96.84	(91.05, 99.34)
100	4	96.00	(90.07, 98.90)
110	4	96.36	(90.95, 99.00)
120	5	95.83	(90.54, 98.63)
130	6	95.38	(90.22, 98.29)
140	6	95.71	(90.91, 98.41)
150	7	95.33	(90.62, 98.10)
160	8	95.00	(90.39, 97.82)
170	9	94.71	(90.19, 97.55)
180	10	94.44	(90.02, 97.30)
190	10	94.74	(90.53, 97.45)
200	11	94.50	(90.37, 97.22)

1. Evaluable (e.g. on-scale) organisms are those that fall within the test range of the reference and have the opportunity for a result on the test method that could also be on-scale. Any reference result that falls in the < or > category is considered not evaluable.
2. Estimated Essential Agreement = percent agreement = number of evaluable test results that are equal to or with in one dilution of the expected result divided by number of organisms that are evaluable.
3. Exact confidence intervals based on the binomial distribution.