MOLDX Analytical Performance for NGS as of 5/22/2015

This is the MOLDX article for "Analytical Performance Specifications for Comprehensive Genomic Profiling (M00118, V1)" as captured on 9/4/2015, reflected the web edition dated 5/22/2015.

Original is/was here.
__________________________________________

(M00118, V1)

MolDX

Analytical Performance Specifications for Comprehensive Genomic Profiling (M00118, V1)

The following criteria will be used to assess the analytic performance of comprehensive genomic profiling for Next Generation Sequencing (NGS):

 

General Laboratory Requirements:

1. CLIA certification and
2. College of American Pathologists¹ (CAP) accreditation* or
3. New York State Department of Health² (NYDOH) final test approval* and
4. Participation in an external proficiency testing (PT) program for NGS testing that addresses all reported variant classes, when available. Any variant classes for which external PT is not available should be assessed by alternative PT, as required by CLIA.

*Note: Palmetto will accept a completed application for either CAP accreditation or NYDOH test approval as provisional documentation for coverage until the final CAP or NYDOH accreditation or approval is available. When CAP/NYDOH provides the laboratory with the final accreditation/approval, the laboratory must notify Palmetto and submit validation of the final accreditation/approval.    

 

Pre-Analytical Requirements:

1. NGS-based analysis must be validated for all sample types accepted for testing (e.g., blood, bone marrow, tissue, purified DNA, etc.) as listed in the laboratory’s sample acceptance criteria¹; 
2. The estimated percentage and viability of neoplastic cells in the tested sample as determined by an appropriate method (e.g., histo-morphology or flow cytometry) must be documented;
3. When the specimen consists of a tissue sample and the percentage of neoplastic cells is determined by morphology,
   1. The review must be performed by an American Board of Pathology-certified anatomic pathologist and
   2. A representative slide of the tissue or a digital image of the slide must be archived. 
4. For specimens that require enrichment (e.g., via microdissection or coring) to achieve the requisite level of tumor cellularity and/or viability, representative slides of pre- and post-enrichment tissue sections (or digital images of the slides) must be archived.

Analytical Requirements

The following four general classes of sequence variants can cause disease:

1. Single nucleotide variants (SNVs)
2. Small insertions and deletions (indels)
3. Copy number variants (CNVs) and
4. Structural variants (SVs), such as translocations

Laboratories need only address requirements for variant classes, alteration sizes, and variant allele frequencies (VAFs) that they report or claim to report based on their validation.  For example, if indels >10 bp or homozygous deletions are not reported, then these requirements do not apply.  Similarly, if the laboratory only claims to report deletions ≤ 8 bp with VAFs > 20%, then only row 7 (not rows 8-10) in Table 2 apply.

 

1.    Accuracy of sequencing performed on a reference cell line

Using its standard clinical testing protocols (including equipment and personnel), the laboratory will sequence NIST Reference Material 8398 (or the HapMap cell line NA12878) for the genes included in its panel, and compare the experimental sequence against the reference values provided (or public reference sequence available at 1000genomes.org). The point estimate for each reported variant class for the positive percent agreement (PPA) and positive predictive value (PPV) should exceed 99.9%.

 

Table 1: Accuracy of sequencing performed on a reference cell line

 

		Orthogonal Reference Results
		Positive	Negative	Total
NGS Results	Positive	A	B	A+B
	Negative	C	D	C+D
	Total	A+C	B+D	A+B+C+D

“Positive” = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

 

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

 

2.    Accuracy and precision of sequencing performed on reference cell lines: Mixing experiments

Using its standard clinical testing protocols (including equipment and personnel), the analytic performance metrics for each variant class reported by the laboratory must be specified and satisfy the requirements below, and compare the experimental sequence against the public reference sequence available at 1000genomes.org. Note that the laboratory will likely need to utilize HapMap cell lines (e.g., NA12878, NA19240, NA18507, NA19129, etc.) in mixing experiments to document the analytical accuracy at different VAFs, including the stated lower limit of detection for each variant class. 

 

Table 2: Accuracy and precision of sequencing performed on reference cell lines: Mixing experiments

 

Row	Variant Type	Detail	Lower 95% Confidence Intervala
			PPAb	PPVb	Intermediate Precisionc
1	Single nucleotide variants	Expected VAF > 10%	≥ 99.0%	≥ 99.0%	≥ 95.0%
2	Single nucleotide variants	Expected VAF = 5-10%	≥ 95.0%	≥ 95.0%	≥ 90.0%
3	Insertions	≤ 10 bp Expected VAF > 20%	≥ 85.0%	≥ 95.0%	≥ 80.0%
4	Insertions	≤ 10 bp Expected VAF = 10-20%	≥ 75.0%	≥ 85.0%	≥ 75.0%
5	Insertions	11-70 bp Expected VAF > 20%	≥ 85.0%	≥ 95.0%	≥ 80.0%
6	Insertions	11-70 bp Expected VAF = 10-20%	≥ 75.0%	≥ 85.0%	≥ 75.0%
7	Deletions	≤ 10 bp Expected VAF > 20%	≥ 85.0%	≥ 95.0%	≥ 80.0%
8	Deletions	≤ 10 bp Expected VAF = 10-20%	≥ 75.0%	≥ 85.0%	≥ 75.0%
9	Deletions	11-70 bp Expected VAF > 20%	≥ 85.0%	≥ 95.0%	≥ 80.0%
10	Deletions	11-70 bp Expected VAF = 10-20%	≥ 75.0%	≥ 85.0%	≥ 75.0%
11	Copy number alterations – Amplifications	Ploidy < 4 Expected CN ≥ 8 > 30% tumor nuclei	≥ 90.0%	≥ 90.0%	≥ 85.0%
12	Copy number alterations – Amplifications	Ploidy < 4 Expected CN ≥ 8 20-30% tumor nuclei	≥ 60.0%	≥ 80.0%	≥ 75.0%
13	Copy number alterations – Homozygous Deletions	Ploidy < 4 Expected CN = 0 > 30% tumor nuclei	≥ 80.0%	≥ 85.0%	≥ 60.0%
14	Copy number alterations – Homozygous Deletions	Ploidy < 4 Expected CN = 0 20-30% tumor nuclei	≥ 50.0%	≥ 75.0%	≥ 50.0%
15	Translocations	≥ 20% tumor nuclei	≥ 85.0%	≥ 85.0%	≥ 90.0%

^a For calculating 95% CIs, use Score or Clopper-Pearson method as described in CLSI EP12-A.

 

b

		Orthogonal Reference Results
		Positive	Negative	Total
NGS Results	Positive	A	B	A+B
	Negative	C	D	C+D
	Total	A+C	B+D	A+B+C+D

“Positive” = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

 

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

 

^c This is based on at least 3 different operators, runs/batches/days, manufacturing reagent lots (for critical reagents), and (if applicable) sequencers and sites.  Agreement is defined as concordance at the variant level for all called (i.e., not necessarily reported but excluding no calls) variants. In other words, did each called variant agree across different operators, runs/batches/days, manufacturing reagent lots, and (if applicable) sequencers and sites?

 

3.    Accuracy of sequencing performed on clinical samples

Using samples that include all sample types accepted for testing (e.g., blood, bone marrow, tissue, purified DNA, etc.) as listed in the laboratory’s sample acceptance criteria, the laboratory will report the point estimates, sample size, and 95% confidence intervals (using the Score or Clopper-Pearson method as described in CLSI EP12-A) for PPA and PPV for each reported variant class based on at least five different variants per row in Table 3 below.

 

Table 3: Accuracy of sequencing performed on clinical samples

 

Row	Variant Type	Detail
1	Single nucleotide variants	Expected VAF > 10%
2	Single nucleotide variants	Expected VAF = 5-10%
3	Insertions	≤ 10 bp Expected VAF > 20%
4	Insertions	≤ 10 bp Expected VAF = 10-20%
5	Insertions	11-70 bp Expected VAF > 20%
6	Insertions	11-70 bp Expected VAF = 10-20%
7	Deletions	≤ 10 bp Expected VAF > 20%
8	Deletions	≤ 10 bp Expected VAF = 10-20%
9	Deletions	11-70 bp Expected VAF > 20%
10	Deletions	11-70 bp Expected VAF = 10-20%
11	Copy number alterations – Amplifications	Ploidy < 4 Expected CN ≥ 8 > 30% tumor nuclei
12	Copy number alterations – Amplifications	Ploidy < 4 Expected CN ≥ 8 20-30% tumor nuclei
13	Copy number alterations – Homozygous Deletions	Ploidy < 4 Expected CN = 0 > 30% tumor nuclei
14	Copy number alterations – Homozygous Deletions	Ploidy < 4 Expected CN = 0 20-30% tumor nuclei
15	Translocations	≥ 20% tumor nuclei

 

		Orthogonal Reference Results
		Positive	Negative	Total
NGS Results	Positive	A	B	A+B
	Negative	C	D	C+D
	Total	A+C	B+D	A+B+C+D

“Positive” = Variant called/identified when compared to current build of human genome (e.g., hg19 human genome assembly, version 37)

 

Positive percent agreement (PPA) = A/(A+C); Positive predictive value (PPV) = A/(A+B)

 

Post-Analytical Testing Requirements

1. The variants identified, clinical interpretations, and therapeutic recommendations must be reported by a physician, board certified in Molecular Genetic Pathology by the American Board of Pathology, or in Molecular Genetics by the American Board of Medical Genetics and Genomics, or has equivalent experience and expertise.  A PhD is not a recognized Medicare provider.
2. Each sequenced run must include a control to document that quality control metrics for the assay (including at least read depth and sequence quality) have been met for that run for all variant classes reported.
3. The bioinformatics pipeline must exclude specimen contamination as the source of identified variants for the variant classes and variant allele frequencies (VAFs) reported^3,4.

References:

1. Aziz N, Zhao Q, Bry L, et al.  College of American Pathologists’ Laboratory Standards for Next-Generation Sequencing Clinical Tests. Arch Pathol Lab Med 2015;139:481-93
2. NextGenSeq website (PDF, 218 KB)
3. Jun G, Flickinger M, Hetrick KN, et al.  Detecting and estimating contamination of human DNA Samples in sequencing and array-based genotype data.  Am J Hum Genet. 2012;91:839-48.
4. Sehn J, Spencer DH, Pfeifer JD, et al.  Occult specimen contamination in routine clinical NGS testing.  Am J Clin Pathol.  In press.

last updated on 05/22/2015