March 12, 2019

Developing best practices for benchmarking: The quiet but crucial work behind sequencing today

The Global Alliance for Genomics and Health publishes new guidelines for comparing a patient’s test results to a reference human genome

Finding the difference between a patient’s DNA sequence and a reference sequence – also known as variant calling – is central to cancer research, but approaches to variant calling differ from lab to lab. Comparing – or benchmarking – one lab’s approach to another lab is important to the development of new sequencing and analysis tools, yet there are no widely-accepted standards for benchmarking variant calls.

To develop these standards and address common benchmarking challenges, a group of stakeholders from government agencies, academic bioinformatics groups, sequencing technology developers and other organizations around the world gathered to create the Global Alliance for Genomics and Health (GA4GH) Benchmarking Team. They’ve recently published their best practices for benchmarking genome sequencing results in Nature Biotechnology.

“Technology is improving rapidly, but we’ve lacked ways to know the strengths and weaknesses of new sequencing and genome analysis methods,” says Dr. Justin Zook, lead author of the study from the National Institute of Standards and Technology. “This paper gives people tools to develop accurate sequencing tests for precision medicine.”

The adoption of these practices – and their continual improvement – can also help facilitate collaboration between research and clinical laboratories while improving the performance of shared tools and methods.

The framework was developed in part by Dr. Paul Boutros and his lab members at OICR, who have used crowdsourcing to develop benchmarking foundations for individual variants as well as broader genetic variation.

Read more about this benchmarking work on Genome Web.