There are several potential sources of reproducibility error that can
occur in NGS testing, and validation experiments should be designed to
address each. These sources include variations in instrumentation,
laboratory technicians, and reagent lots. To determine reproducibility,
at least three samples should be tested across each potential
variability source (i.e., performed by multiple laboratory personnel on
multiple instruments with different reagent lots). In addition to these
between-run, repeatability tests, duplicate, within-run tests should be
performed without any anticipated source of variability to determine
Variability between the outcome of each test should be quantified
across multiple steps of the NGS workflow. For example, variability in
nucleic acid yield following extraction, library prep quality control
metrics, sequencing read outcome metrics, and final variant calls should
all be noted.
Limits of detection
AMP’s recommends that the lower limit of detection (LLOD) is defined as
the lowest allele fraction at which the allele will be reliably detected
for 95% of samples. In order to achieve 95% confidence, mathematically,
at least 59 samples must be tested. If it is not possible to test 59
samples, additional controls designed to determine sensitivity should be
Interfering substances and carryover
NGS tests are potentially sensitive to interference by reagents or
biological materials that are not effectively removed during the nucleic
acid extraction process. Potential interfering substances might include
fixatives such as heavy metals, cellular components like melanin or
hemoglobin, or interfering nucleic acids—for instance RNA for a
DNA-based NGS assay.
This so-called carryover issue should be addressed during validation,
particularly for detecting variants that are expected to exhibit a low
allele burden. Carryover testing can be performed by comparing samples
to standards and through the inclusion of no template controls (NTCs).
As bioinformatic approaches can also be used to identify carryover
issues—for instance, by detecting known human interfering
substances—bioinformatics is an essential part of the interfering
substances and carryover validation process.
All possible types of interfering substances should be evaluated
systematically throughout the validation process, and carryover
monitored and quantified at each step of the NGS workflow. AMP suggests
including NTCs in every sequencing run, though the controls need not be
evaluated throughout the entire workflow.
Validation of the bioinformatics pipeline
The bioinformatics pipeline is critical for assay performance and
intimately tied to the entire NGS workflow. For instance, the required
coverage depth for variant detection will vary depending on the
The bioinformatics pipeline should be validated based on
methods-based paradigms using well-characterized cell lines that reflect
the variant population and variant allele frequency anticipated in the
clinical service. If appropriate physical cell line samples cannot be
obtained for validation, it is also possible to validate the
bioinformatics pipeline in silico using sequence files generated from well-characterized samples.
Our role in the validation process
Fabric Genomics provides artificial-intelligence (AI) driven data analysis solutions for clinical NGS workflows. Our industry-leading AI takes the guesswork out of the data analysis process and decreases variability while delivering superior sensitivity. Because bioinformatics analysis is an integral part of the validation process and ongoing testing, we developed this guide to help get you started in planning your own NGS validation project. We are here to support you every step of the way, so if you want help planning your validation experiments, please get in touch.