MultiQC: summarize analysis results for multiple tools and samples in a single report Philip Ewels Måns Magnusson Sverker Lundin Max Käller

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

 MultiQC: summarize analysis results for multiple tools and samples in a single report 2016 Bioinformatics 10.1093/bioinformatics/btw354 ewels2016-multiqc-analysis-results ewels2016-multiqc-analysis-results.txt, the project's staged copy d0813407d8d1543dc43f58a590d19629d165ac51 cc-by bioinformatics

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

 131 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them cross-tool and cross-sample integration and visualization of next-generation-sequencing quality-control results how to replace repetitive per-sample QC compilation with an extensible summary that exposes global trends, outliers, biases, and batch effects MultiQC, a directory-scanning modular tool that parses many analysis logs into a single interactive report and reusable data exports method quality-control visibility, batch-effect detection, outlier detection, reporting effort, extensibility, and scaling across many sequencing samples system-or-tool-buildingsystem-or-tool-building Specified directories are recursively scanned for recognized output files.recursive directory search Each supported-tool parser queries configurable patterns and contributes only when logs exist.tool submodules A loaded template is parsed to render the final report.Jinja2 Parsed data are exported for downstream processing.tab delimited text, YAML or JSON Aligned sample visualizations expose differences hidden across separate files.shared plots Browser-side rendering provides resizeable, interactive, zoomable plots.HighCharts Large reports use runtime-rendered static images to keep the standalone file manageable.MatPlotLib Sample renaming, hiding, and highlighting support focused inspection.report toolbox External plugins access and modify internal functionality.code hooks Independent modules and templates execute as part of the main program.setuptools entry points Facility metadata enter reports and parsed summaries flow into a database.MultiQC_NGI plugin Branding and self-contained output support delivery reports.templates Existing QC logs and reports are predominantly per-sample and require manual project-wide compilation. Expanding sample counts increase vulnerability to subtle confounding batch effects. Per-sample switching can conceal subtle differences, outliers, and batch effects. HighCharts becomes impractical for reports containing hundreds of samples. Static large-sample reports sacrifice browser interactivity for consistent standalone size. Research groups may require bespoke parsers and customization beyond built-in tools. Routine value depends on the recognized logs produced by upstream tools. The paper reports adoption and applications but no controlled comparison of error detection against manual review. quality controlquality control next-generation sequencing datanext-generation sequencing data QC metricsQC metrics analysis resultsanalysis results batch effectsbatch effects outlier samplesoutlier samples MultiQCMultiQC tooltool single reportsingle report global trendsglobal trends logslogs reportsreports shared plotsshared plots confounding factorsconfounding factors custom comparative scriptscustom comparative scripts recognized filesrecognized files SubmodulesSubmodules templatetemplate Jinja2 packageJinja2 package parsed dataparsed data HTML reportHTML report HighChartsHighCharts report toolboxreport toolbox MatPlotLibMatPlotLib static reportsstatic reports custom codecustom code Code hooksCode hooks external pluginsexternal plugins setuptools entry pointssetuptools entry points modulesmodules single cellsingle cell population studiespopulation studies post-processingpost-processing dataset filteringdataset filtering sequencing facilitysequencing facility MultiQC_NGI pluginMultiQC_NGI plugin LIMSLIMS central collection pointcentral collection point analysis pipelinesanalysis pipelines sample numberssample numbers ExtensionExtension Advances in next-generation sequencingAdvances in next-generation sequencing avalanche of dataavalanche of data MultiQC → toolMultiQC is introduced appositively as a tool for integrated reporting. HighCharts → MatPlotLibLarge reports switch from HighCharts to MatPlotLib image rendering. single report → global trendsA single integrated report enables rapid identification of global trends and biases. shared plots → batch effectsShared cross-sample plots aid detection of subtle batch effects. MultiQC → custom comparative scriptsMultiQC removes the need for custom cross-sample comparison scripts. sample numbers → batch effectsGrowing sample sets increase vulnerability to confounding batch effects. Submodules → recognized filesEach submodule parses only when recognized logs are found. template → HTML reportJinja2 parses a selected template to render the report. MatPlotLib → static reportsMatPlotLib produces embedded images when interactive rendering becomes too large. Code hooks → external pluginsCode hooks allow external plugins to modify MultiQC internals. parsed data → post-processingSaved parsed data can be used for post-processing and filtering. MultiQC_NGI plugin → LIMSThe plugin inserts LIMS metadata into reports and stores summaries in a database. Extension → central collection pointExtension and export let MultiQC function as a pipeline-end collection point. MultiQC → batch effectsRoutine early use can reduce downstream risk from batch effects. quality control as project-wide comparison rather than isolated per-sample inspection batch effect as a subtle confound exposed through shared visualization single report as an integrated visualization plus reusable data export submodule as a conditional parser that is silent when its logs are absent self-contained report as a portable file with embedded plots extension as separately maintained code executing inside the main program large-project QC as rapid detection of abnormal samples across thousands of inputs central collection point as the reporting terminus of an analysis pipeline builds-on → per-tool logs and QC reportsnote: MultiQC consumes outputs of established analysis and QC tools. contrasts-with → manual per-sample result compilationnote: One cross-sample report replaces repeated switching and compilation. extends → existing bioinformatics analysis pipelinesnote: MultiQC adds an extensible reporting terminus and data export. contrasts-with → custom comparative scriptsnote: The common report removes the need for bespoke comparison code. Advances in next-generation sequencing —causes→ avalanche of datadirection on the measure: increases · the relation itself is separately evidenced · strength of the relation: 0.95 · note: Direct flat effect statement. · the relation stated: Advances in next-generation sequencing are leading to an avalanche of data.Advances in next-generation sequencingavalanche of data Per-sample QC logs make project-wide assessment slow, repetitive, complex, and error prone. MultiQC creates one report from multiple tools and many samples so global trends and biases can be identified quickly. Shared plots expose subtle sample differences that are difficult to see by switching among files. Central visualization supports batch-effect detection and reduces the chance that confounders affect downstream results. The same command works across analysis types because submodules contribute only when matching logs are found. At publication MultiQC supported 22 aligners, processing tools, and QC programs. The report is a shareable self-contained HTML file with interactive, exportable plots. For hundreds of samples MultiQC switches from browser plotting to embedded MatPlotLib images. Modules, templates, and plugins can remain in separate code bases while executing within MultiQC. Users had contributed four modules and at least three research groups had written plugins by publication. MultiQC had already been downloaded thousands of times and integrated into bcbio-nextgen. MultiQC parses thousands of samples within minutes and makes abnormal samples easy to identify. Sequencing-facility reports expose failing samples and irregular sample groups. The MultiQC_NGI plugin links LIMS metadata and stores parsed summaries in an internal database. Extension and data export let MultiQC serve as a central collection point at the end of analysis pipelines. a tool that summarizes outputs from multiple tools and samples in one reportMultiQC a supported-tool component that searches for and parses recognized logssubmodule a self-contained HTML artifact combining statistics and visualizations across samplesreport an extension point through which external plugins modify program internalscode hook a separately distributable presentation component used to render or brand reportstemplate external code integrated with MultiQC functionality through hooks and entry pointsplugin subtle confounding differences across expanding sample setsbatch effects the pipeline terminus where analysis metadata are integrated and exportedcentral collection point 22 — common bioinformatics tools supported at publication hundreds of samples — report scale that triggers static image rendering four — new modules contributed by users at least three — research groups known to have written plugins thousands of times — downloads during the first months thousands of samples within minutes — reported large-project parsing capacity sample numberssample numbers batch effectsbatch effects outlier samplesoutlier samples key statisticskey statistics analysis typeanalysis type parsed dataparsed data

MultiQC: summarize analysis results for multiple tools and samples in a single report

Abstract

Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.

Results: We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.

Availability and implementation: MultiQC is available with an GNU GPLv3 license on GitHub, the Python Package Index and Bioconda. Documentation and example reports are available at http://multiqc.info

Contact: phil.ewels@scilifelab.se

1 Introduction

Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.

Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.

MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.

Top of a typical MultiQC report. The general statistics table can be seen with metrics from a number of different tools gathered for each sample (Color version of this figure is available at Bioinformatics online.)

2 Materials and methods

2.1 Running MultiQC

MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.

2.2 MultiQC reports

MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.

Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).

Each report contains an interactive walk through of features. Tutorial videos can be found at http://multiqc.info along with tutorials and documentation describing installation, usage and troubleshooting.

2.3 Extending MultiQC

MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.

Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).

3 Typical applications

3.1 Single cell data and population studies

Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.

3.2 Sequencing facilities

MultiQC was originally developed for use in a high throughput sequencing facility. Reports give the overview required to spot failing samples and highlighting helps to identify groups of samples behaving in an irregular manner.

Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.

4 Conclusion

As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.

Acknowledgements

The authors would like to thank D. Klevebring, L. Pantano, G. Carrasco and R. Andeer for contributed modules and discussion.

Funding

This work was supported by the Science for Life Laboratory and the National Genomics Infrastructure, NGI.

Conflict of Interest : none declared.

References