One of the most important attributes for a DNA sequencing system is accuracy. Scientists want the highest quality, reproducible data. Sequencing systems have strengths and weaknesses driven by the unique chemistry of each. One criticism of the Ion Torrent™ sequencing chemistry has been that the fast and efficient lack of termination (which also gives it great speed) results in difficulty in long stretches of the same nucleotide (called a homopolymer tract). This results in a relatively high insertion and deletion (InDel) error rate. A sometimes less recognized feature of the Ion Torrent chemistry is that only a single nucleotide is presented at a given time to a polymerase. The absence of four competitive nucleotides results in a lower substitution error rate for the Ion Torrent system relative to systems with all four fluorescently tagged nucleotides presented simultaneously. The Ion PGM sequencer’s high substitution accuracy has been observed by others in peer reviewed literature as well (Junemann et al., Nature Biotechnology 2013 31(14)294-296.). This leads to a question that many have asked before – which system is more accurate? Ion Torrent’s Ion PGM™ Sequencer or Illumina’s MiSeq Sequencer?
As sequencing technologies continue to gain acceptance in the clinical research world, new tools are becoming available online to help verify sequencing performance within an individual laboratory. The US National Institute of Standards and Technology (NIST) has invested a lot of effort in their Genome In A Bottle (GIAB) initiative where a single reference sample is sequenced with many different technologies to derive a platform independent sequence. This is a great tool to verify both broad panels like whole exomes or whole genome sequencing. But when more tightly focused panels are involved, a single well-documented genome doesn’t have enough genetic variation to get a good measure of accuracy. For example, in the GIAB sample, looking only at the targeted region on a cancer research panel like the Ion AmpliSeq™ Cancer Hotspot Panel v2, one would expect to see only 14 variants – not enough variants to measure the accuracy of a panel with any robustness. To truly test if a gene panel is accurately sequenced, having a sample with broader truth would provide a better test. The AcroMetrix arm of Thermo Fisher Scientific has recently introduced a new AcroMetrix™ Oncology Hotspot Control to help address this problem. (We have recently written about the AcroMetrix panel on Behind the Bench.) AcroMetrix synthesized over 500 COSMIC variants to be added at a few levels of concentration to the GIAB genomic sample. Because they spiked into this genomic background, users can determine for themselves the level of abundance to simulate somatic mutation detection conditions.
Having access to this wonderfully broad “truth” control sample from AcroMetrix allows researchers to compare the accuracy of two sequencing platforms for a very focused set of top oncology targets using popular PCR amplification techniques. We at Ion Torrent ran a first trial at this question by comparing the whole systems from amplicon-generation to variant calling across both our Ion PGM™ System and the MiSeq sequencer from Illumina. Our simple experiment involved running two runs with either a 1% or 3% somatic levels using Illumina’s TruSeq Amplicon Caner Panel coupled with MiSeq sequencing and their recommended mapping and variant calling algorithms. We compared this to three runs of the Ion AmpliSeq™ Cancer Hotspot Panel v2 run on the Ion PGM™ Sequencer with our new Ion PGM™ Hi-Q™ Sequencing Kit. Due to differences in targeted regions, we focused the comparison on the 13.6 kb of overlap between both panels. To compare accuracy, we measured both raw read accuracy across the length of the read and then consensus accuracy across read depth by measuring the sensitivity and positive predictive values between the two sequencing platforms.
Measuring the accuracy of single read at a time, the mean raw read error for the 3% mix was 0.55% for MiSeq and 0.42% for the Ion PGM™ Sequencer. There was a lot of variability across the length of the read, so one could argue that the difference in these means is not statistically significant, but they are certainly comparable. Looking deeper into the accuracy comparison, we wanted to measure consensus accuracy when the redundancy of multiple reads is leveraged. The overall sensitivity (both SNPs and InDels) of MiSeq was 93.8% compared with 95.1% for the Ion PGM™ Sequencer. The Positive Predictive Values of MiSeq was 98.1% compared to 100% for Ion PGM™ Sequencer The results from the 1% mix were similar to the 3% mix. We also combined the 1% and 3% runs together for simplicity and sensitivity was 94.1% vs 91.3% and PPV was 99.8% vs 98% for PGM vs MiSeq respectively. Either separate or combined, the consensus accuracy results were statistically significant.
We at Ion Torrent were pleased with these results because it’s a reflection of an enormous amount of investment that we’ve made in improving the accuracy of the Ion PGM™ System. The new Ion PGM™ Hi-Q™ sequencing kit harnessed an exhaustive enzymatic screening process to find a sequencing chemistry with lower systematic error. Also, continued enhancements to our informatics pipeline improves our overall accuracy with significantly fewer false positive calls when compared with previous versions of Ion PGM™ sequencing kits. The Ion PGM™ sequencer coupled with the Ion AmpliSeq™ chemistry to selectively amplify targets from as little as 10 ng of FFPE DNA brings enormous value to the oncology research market.
Having broad truth available from the Acrometrix Oncology Hotspot Control Kit and the result of this 2×5 run comparison allow us to ask the question – is the Ion PGM™ System with Ion AmpliSeq™ Cancer Hotspot Panel v2 and Ion PGM Hi-Q™ Sequencing Kit the most accurate benchtop sequencer for targeted oncology panels? We’ve released the data and methods on Ion Community http://ioncommunity.thermofisher.com/docs/DOC-9157 for anyone to inspect and the Acrometrix® Oncology Hotspot Control is commercially available for anyone to run the experience themselves. What we do know is that the Ion PGM™ Sequencer continues to mature in its accuracy measurements. If you haven’t run a PGM since 2011 or 2012, please fire it up with our latest HiQ chemistry and see how it’s performing in your hands.