Philip EwelspersName, Måns MagnussonpersName, Sverker LundinpersName, Max KällerpersName (2016date). “MultiQC: summarize analysis results for multiple tools and samples in a single report.” Bioinformaticstitle·j.
Object of study — cross-tool and cross-sample integration and visualization of next-generation-sequencing quality-control results
Research question — how to replace repetitive per-sample QC compilation with an extensible summary that exposes global trendsterm, outliers, biases, and batch effectsterm
Contribution — MultiQCterm, a directory-scanning modular toolterm that parses many analysis logsterm into a single interactive reportterm·def and reusable data exports
Cross-disciplinary analysis
content-tags & facets:
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | quality-control visibility, batch-effect detection, outlier detection, reporting effort, extensibility, and scaling across many sequencing samples — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | |
| claim-type | method — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | |
| method — evidence | — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | |
| object of study — evidence | — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | |
| research question — evidence | — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | |
| contribution — evidence | — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | |
| mechanism | recursive directory search Specified directories are recursively scanned for recognized output files. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | mechanisms[0] |
| mechanism | tool submodules Each supported-tool parser queries configurable patterns and contributes only when logs exist. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | mechanisms[1] |
| mechanism | Jinja2 A loaded template is parsed to render the final report. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | mechanisms[2] |
| mechanism | tab delimited text, YAML or JSON Parsed data are exported for downstream processing. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | mechanisms[3] |
| mechanism | shared plots Aligned sample visualizations expose differences hidden across separate files. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | mechanisms[4] |
| mechanism | HighCharts Browser-side rendering provides resizeable, interactive, zoomable plots. — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | mechanisms[5] |
| mechanism | MatPlotLib Large reports use runtime-rendered static images to keep the standalone file manageable. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | mechanisms[6] |
| mechanism | report toolbox Sample renaming, hiding, and highlighting support focused inspection. — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | mechanisms[7] |
| mechanism | code hooks External plugins access and modify internal functionality. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | mechanisms[8] |
| mechanism | setuptools entry points Independent modules and templates execute as part of the main program. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | mechanisms[9] |
| mechanism | MultiQC_NGI plugin Facility metadata enter reports and parsed summaries flow into a database. — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | mechanisms[10] |
| mechanism | templates Branding and self-contained output support delivery reports. — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | mechanisms[11] |
| builds-on | per-tool logs and QC reports — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | claim_relations[0] |
| contrasts-with | manual per-sample result compilation — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[1] |
| extends | existing bioinformatics analysis pipelines — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | claim_relations[2] |
| contrasts-with | custom comparative scripts — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[3] |
| sense | quality control as project-wide comparison rather than isolated per-sample inspection — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | senses[0] |
| sense | batch effect as a subtle confound exposed through shared visualization — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | senses[1] |
| sense | single report as an integrated visualization plus reusable data export — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | senses[2] |
| sense | submodule as a conditional parser that is silent when its logs are absent — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | senses[3] |
| sense | self-contained report as a portable file with embedded plots — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | senses[4] |
| sense | extension as separately maintained code executing inside the main program — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | senses[5] |
| sense | large-project QC as rapid detection of abnormal samples across thousands of inputs — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | senses[6] |
| sense | central collection point as the reporting terminus of an analysis pipeline — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | senses[7] |
| note | per-tool logs and QC reports — MultiQC consumes outputs of established analysis and QC tools. — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | claim_relations[0] |
| weight | per-tool logs and QC reports — 0.95 — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | claim_relations[0] |
| note | manual per-sample result compilation — One cross-sample report replaces repeated switching and compilation. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[1] |
| weight | manual per-sample result compilation — 0.95 — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[1] |
| note | existing bioinformatics analysis pipelines — MultiQC adds an extensible reporting terminus and data export. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | claim_relations[2] |
| weight | existing bioinformatics analysis pipelines — 0.95 — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | claim_relations[2] |
| note | custom comparative scripts — The common report removes the need for bespoke comparison code. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[3] |
| weight | custom comparative scripts — 0.95 — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | claim_relations[3] |
| effect direction | Advances in next-generation sequencing —causes→ avalanche of data — increases — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concept_edges[0] |
| note | Advances in next-generation sequencing —causes→ avalanche of data — Direct flat effect statement. — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concept_edges[0] |
| relation | Advances in next-generation sequencing —causes→ avalanche of data — Advances in next-generation sequencing are leading to an avalanche of data. — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concept_edges[0] |
| weight | Advances in next-generation sequencing —causes→ avalanche of data — 0.95 — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concept_edges[0] |
| relation | MultiQC → tool — MultiQC is introduced appositively as a tool for integrated reporting. — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | unhoused_relations[0] |
| relation | HighCharts → MatPlotLib — Large reports switch from HighCharts to MatPlotLib image rendering. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[1] |
| weight | HighCharts → MatPlotLib — 0.95 — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[1] |
| relation | single report → global trends — A single integrated report enables rapid identification of global trends and biases. — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | unhoused_relations[2] |
| weight | single report → global trends — 0.95 — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | unhoused_relations[2] |
| relation | shared plots → batch effects — Shared cross-sample plots aid detection of subtle batch effects. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[3] |
| weight | shared plots → batch effects — 0.95 — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[3] |
| relation | MultiQC → custom comparative scripts — MultiQC removes the need for custom cross-sample comparison scripts. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[4] |
| weight | MultiQC → custom comparative scripts — 0.95 — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[4] |
| relation | sample numbers → batch effects — Growing sample sets increase vulnerability to confounding batch effects. — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | unhoused_relations[5] |
| weight | sample numbers → batch effects — 0.85 — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | unhoused_relations[5] |
| relation | Submodules → recognized files — Each submodule parses only when recognized logs are found. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[6] |
| weight | Submodules → recognized files — 0.95 — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[6] |
| relation | template → HTML report — Jinja2 parses a selected template to render the report. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[7] |
| weight | template → HTML report — 0.95 — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[7] |
| relation | MatPlotLib → static reports — MatPlotLib produces embedded images when interactive rendering becomes too large. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[8] |
| weight | MatPlotLib → static reports — 0.95 — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[8] |
| relation | Code hooks → external plugins — Code hooks allow external plugins to modify MultiQC internals. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | unhoused_relations[9] |
| weight | Code hooks → external plugins — 0.95 — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | unhoused_relations[9] |
| relation | parsed data → post-processing — Saved parsed data can be used for post-processing and filtering. — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | unhoused_relations[10] |
| weight | parsed data → post-processing — 0.95 — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | unhoused_relations[10] |
| relation | MultiQC_NGI plugin → LIMS — The plugin inserts LIMS metadata into reports and stores summaries in a database. — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | unhoused_relations[11] |
| weight | MultiQC_NGI plugin → LIMS — 0.95 — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | unhoused_relations[11] |
| relation | Extension → central collection point — Extension and export let MultiQC function as a pipeline-end collection point. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[12] |
| weight | Extension → central collection point — 0.95 — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[12] |
| relation | MultiQC → batch effects — Routine early use can reduce downstream risk from batch effects. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[13] |
| weight | MultiQC → batch effects — 0.7 — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[13] |
| no type for this | MultiQC → tool — MultiQC is introduced appositively as a tool for integrated reporting. — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | unhoused_relations[0] |
| no type for this | HighCharts → MatPlotLib — Large reports switch from HighCharts to MatPlotLib image rendering. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[1] |
| no type for this | single report → global trends — A single integrated report enables rapid identification of global trends and biases. — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | unhoused_relations[2] |
| no type for this | shared plots → batch effects — Shared cross-sample plots aid detection of subtle batch effects. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[3] |
| no type for this | MultiQC → custom comparative scripts — MultiQC removes the need for custom cross-sample comparison scripts. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | unhoused_relations[4] |
| no type for this | sample numbers → batch effects — Growing sample sets increase vulnerability to confounding batch effects. — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | unhoused_relations[5] |
| no type for this | Submodules → recognized files — Each submodule parses only when recognized logs are found. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[6] |
| no type for this | template → HTML report — Jinja2 parses a selected template to render the report. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | unhoused_relations[7] |
| no type for this | MatPlotLib → static reports — MatPlotLib produces embedded images when interactive rendering becomes too large. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | unhoused_relations[8] |
| no type for this | Code hooks → external plugins — Code hooks allow external plugins to modify MultiQC internals. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | unhoused_relations[9] |
| no type for this | parsed data → post-processing — Saved parsed data can be used for post-processing and filtering. — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | unhoused_relations[10] |
| no type for this | MultiQC_NGI plugin → LIMS — The plugin inserts LIMS metadata into reports and stores summaries in a database. — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | unhoused_relations[11] |
| no type for this | Extension → central collection point — Extension and export let MultiQC function as a pipeline-end collection point. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[12] |
| no type for this | MultiQC → batch effects — Routine early use can reduce downstream risk from batch effects. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | unhoused_relations[13] |
| limitation | Existing QC logs and reports are predominantly per-sample and require manual project-wide compilation. — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | limitations[0] |
| limitation | Expanding sample counts increase vulnerability to subtle confounding batch effects. — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | limitations[1] |
| limitation | Per-sample switching can conceal subtle differences, outliers, and batch effects. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | limitations[2] |
| limitation | HighCharts becomes impractical for reports containing hundreds of samples. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | limitations[3] |
| limitation | Static large-sample reports sacrifice browser interactivity for consistent standalone size. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | limitations[4] |
| limitation | Research groups may require bespoke parsers and customization beyond built-in tools. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | limitations[5] |
| limitation | Routine value depends on the recognized logs produced by upstream tools. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | limitations[6] |
| limitation | The paper reports adoption and applications but no controlled comparison of error detection against manual review. — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | limitations[7] |
Content tags
content-observations:
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Per-sample QC logs make project-wide assessment slow, repetitive, complex, and error prone. — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | content_tags.findings[0] |
| finding | MultiQC creates one report from multiple tools and many samples so global trends and biases can be identified quickly. — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | content_tags.findings[1] |
| finding | Shared plots expose subtle sample differences that are difficult to see by switching among files. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | content_tags.findings[2] |
| finding | Central visualization supports batch-effect detection and reduces the chance that confounders affect downstream results. — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | content_tags.findings[3] |
| finding | The same command works across analysis types because submodules contribute only when matching logs are found. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | content_tags.findings[4] |
| finding | At publication MultiQC supported 22 aligners, processing tools, and QC programs. — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | content_tags.findings[5] |
| finding | The report is a shareable self-contained HTML file with interactive, exportable plots. — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | content_tags.findings[6] |
| finding | For hundreds of samples MultiQC switches from browser plotting to embedded MatPlotLib images. — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | content_tags.findings[7] |
| finding | Modules, templates, and plugins can remain in separate code bases while executing within MultiQC. — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | content_tags.findings[8] |
| finding | Users had contributed four modules and at least three research groups had written plugins by publication. — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | content_tags.findings[9] |
| finding | MultiQC had already been downloaded thousands of times and integrated into bcbio-nextgen. — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | content_tags.findings[10] |
| finding | MultiQC parses thousands of samples within minutes and makes abnormal samples easy to identify. — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | content_tags.findings[11] |
| finding | Sequencing-facility reports expose failing samples and irregular sample groups. — “MultiQC was originally developed for use in a high throughput sequencing facility. Reports give the overview required to spot failing samples and highlighting helps to identify groups of samples behaving in an irregular manner.” | content_tags.findings[12] |
| finding | The MultiQC_NGI plugin links LIMS metadata and stores parsed summaries in an internal database. — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | content_tags.findings[13] |
| finding | Extension and data export let MultiQC serve as a central collection point at the end of analysis pipelines. — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | content_tags.findings[14] |
| quantity | 22 — common bioinformatics tools supported at publication — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | content_tags.quantities[0] |
| quantity | hundreds of samples — report scale that triggers static image rendering — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | content_tags.quantities[1] |
| quantity | four — new modules contributed by users — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | content_tags.quantities[2] |
| quantity | at least three — research groups known to have written plugins — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | content_tags.quantities[3] |
| quantity | thousands of times — downloads during the first months — “Extensive documentation makes adding to MultiQC simple; four new modules have been contributed by users to date and we are aware of at least three plugins written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQC has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).” | content_tags.quantities[4] |
| quantity | thousands of samples within minutes — reported large-project parsing capacity — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | content_tags.quantities[5] |
| variable | sample numbers (project scale and confounding-risk driver) — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | content_tags.variables[0] |
| variable | batch effects (quality-control target) — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | content_tags.variables[1] |
| variable | outlier samples (quality-control target) — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | content_tags.variables[2] |
| variable | key statistics (cross-sample summary inputs) — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | content_tags.variables[3] |
| variable | analysis type (workflow variation handled by module discovery) — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | content_tags.variables[4] |
| variable | parsed data (post-processing and filtering output) — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | content_tags.variables[6] |
| definition | MultiQC — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | content_tags.definitions[0] |
| definition | submodule — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | content_tags.definitions[1] |
| definition | report — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | content_tags.definitions[2] |
| definition | code hook — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | content_tags.definitions[3] |
| definition | template — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | content_tags.definitions[4] |
| definition | plugin — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | content_tags.definitions[5] |
| definition | batch effects — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | content_tags.definitions[6] |
| definition | central collection point — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | content_tags.definitions[7] |
Concepts
terms this paper carries, each with the sentence it was cut from — the layer that joins one paper to another. Terms with no quote yet are on CONCEPTS.html.
| concept | quality control — not defined in this paper's own words; assessment of sequencing and analysis quality — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[0] |
| concept | next-generation sequencing data — not defined in this paper's own words; high-volume sequencing study data — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[1] |
| concept | QC metrics — not defined in this paper's own words; tool-specific quality measurements — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[2] |
| concept | analysis results — not defined in this paper's own words; outputs assessed across a project — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[3] |
| concept | batch effects — not defined in this paper's own words; cross-sample confounding differences — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[4] |
| concept | outlier samples — not defined in this paper's own words; abnormal samples requiring detection — “Motivation: Fast and accurate quality control is essential for studies involving next-generation sequencing data. Whilst numerous tools exist to quantify QC metrics, there is no common approach to flexibly integrate these across tools and large sample sets. Assessing analysis results across an entire project can be time consuming and error prone; batch effects and outlier samples can easily be missed in the early stages of analysis.” | concepts[5] |
| concept | MultiQC — not defined in this paper's own words; cross-tool cross-sample QC reporting tool — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | concepts[6] |
| concept | tool — not defined in this paper's own words; software that creates an integrated report — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | concepts[7] |
| concept | single report — not defined in this paper's own words; integrated multi-tool multi-sample output — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | concepts[8] |
| concept | global trends — not defined in this paper's own words; project-wide patterns exposed by aggregation — “We present MultiQC, a tool to create a single report visualising output from multiple tools across many samples, enabling global trends and biases to be quickly identified. MultiQC can plot data from many common bioinformatics tools and is built to allow easy extension and customization.” | concepts[9] |
| concept | logs — not defined in this paper's own words; per-tool records of analysis results — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | concepts[10] |
| concept | reports — not defined in this paper's own words; per-sample quality artifacts — “Most bioinformatics programs produce logs detailing their results. Dedicated QC tools such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logs and reports are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.” | concepts[11] |
| concept | shared plots — not defined in this paper's own words; cross-sample visual comparison surface — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | concepts[12] |
| concept | confounding factors — not defined in this paper's own words; biases threatening downstream study results — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | concepts[13] |
| concept | custom comparative scripts — not defined in this paper's own words; bespoke code replaced by common reporting — “MultiQC addresses this problem by scanning given analysis directories for log files and QC reports, creating a single summary report visualizing results across all samples. Collecting data within a single report provides a fast way to scan key statistics quickly and easily ( Fig. 1 ). Shared plots allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effect detection and minimizes the risk of confounding factors affecting the results of the study. MultiQC is the first tool of its type within the field; it has the potential to greatly improve quality control and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scripts.” | concepts[14] |
| concept | recognized files — not defined in this paper's own words; inputs matched by supported-tool search settings — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | concepts[15] |
| concept | Submodules — not defined in this paper's own words; tool-specific file query and parsing components — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | concepts[16] |
| concept | template — not defined in this paper's own words; report rendering definition — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | concepts[17] |
| concept | Jinja2 package — not defined in this paper's own words; template renderer — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | concepts[18] |
| concept | parsed data — not defined in this paper's own words; structured extracted metrics for reuse — “MultiQC is written in Python and run on the command line; specified directories are searched recursively for any recognized files. Submodules for each supported tool are run, querying input files using configurable search settings. If any log files are found they are parsed, otherwise the module exits silently. Once all submodules have finished, the specified template is loaded and parsed using the Jinja2 package to render the final report. Parsed data is saved as tab delimited text, YAML or JSON for downstream use. Because submodules only contribute to the report if they find logs, MultiQC is run in the same way for every analysis type. At the time of writing, MultiQC supports 22 common bioinformatics tools including aligners, processing tools and QC programs.” | concepts[19] |
| concept | HTML report — not defined in this paper's own words; self-contained shareable reader artifact — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | concepts[20] |
| concept | HighCharts — not defined in this paper's own words; interactive browser plotting library — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | concepts[21] |
| concept | report toolbox — not defined in this paper's own words; sample visibility and emphasis controls — “MultiQC generates a single s elf-contained HTML report which can be shared and opened in any modern web browser. Reports render plots using the JavaScript plotting library HighCharts ( http://www.highcharts.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolbox. Plots can be exported in a range of publication-ready formats.” | concepts[22] |
| concept | MatPlotLib — not defined in this paper's own words; runtime static-image plotting fallback — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | concepts[23] |
| concept | static reports — not defined in this paper's own words; embedded-image output for large cohorts — “Reports with hundreds of samples become too large for use with HighCharts; instead MultiQC switches to rendering plots as images at run-time using the Python plotting library MatPlotLib ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reports are also suitable for conversion to PDF using tools such as Pandoc ( http://pandoc.org ).” | concepts[24] |
| concept | custom code — not defined in this paper's own words; research-group extension logic — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | concepts[25] |
| concept | Code hooks — not defined in this paper's own words; external access to program internals — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | concepts[26] |
| concept | external plugins — not defined in this paper's own words; separately maintained extensions — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | concepts[27] |
| concept | setuptools entry points — not defined in this paper's own words; plugin discovery and integration points — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | concepts[28] |
| concept | modules — not defined in this paper's own words; supported-tool parser extensions — “MultiQC supports a lot of common bioinformatics tools but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQC is built in such a way that custom code can be tied into its functionality easily. Code hooks allow external plugins to access and modify the internal workings of the program. The use of Python setuptools entry points allows modules, templates and plugins to be kept within a separate code base, whilst still executing as part of the main MultiQC program.” | concepts[29] |
| concept | single cell — not defined in this paper's own words; large multi-sample application setting — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | concepts[30] |
| concept | population studies — not defined in this paper's own words; large multi-dataset application setting — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | concepts[31] |
| concept | post-processing — not defined in this paper's own words; downstream use of parsed data — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | concepts[32] |
| concept | dataset filtering — not defined in this paper's own words; downstream sample or data selection — “Single cell and population studies are perhaps the perfect examples of large projects where accurate quality control of numerous datasets is critical. MultiQC is able to parse data for thousands of samples within minutes, adapting report output as required. Parsed data saved by MultiQC can be used for post-processing and dataset filtering. Reports reveal overall analysis success and make it easy to identify abnormal samples.” | concepts[33] |
| concept | sequencing facility — not defined in this paper's own words; high-throughput operational deployment context — “MultiQC was originally developed for use in a high throughput sequencing facility. Reports give the overview required to spot failing samples and highlighting helps to identify groups of samples behaving in an irregular manner.” | concepts[34] |
| concept | MultiQC_NGI plugin — not defined in this paper's own words; LIMS and database integration extension — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | concepts[35] |
| concept | LIMS — not defined in this paper's own words; facility information system supplying metadata — “Plugins allow integration with in-house systems: we have written the MultiQC_NGI plugin which inserts meta data from our LIMS into reports and stores summary results parsed by MultiQC in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templates allow report branding and reports are self-contained, making MultiQC an ideal tool for creating delivery reports.” | concepts[36] |
| concept | central collection point — not defined in this paper's own words; metadata integration terminus of a pipeline — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | concepts[37] |
| concept | analysis pipelines — not defined in this paper's own words; upstream chains whose outputs MultiQC integrates — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | concepts[38] |
| concept | sample numbers — not defined in this paper's own words; project-scale count that raises confounding risk — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concepts[39] |
| concept | Extension — not defined in this paper's own words; customization and export capability — “As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics tools producing ever more verbose descriptions of data. Integrating these statistics across tools with large sample sets is difficult and time-consuming. MultiQC can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extension and data export allow MultiQC to function as a central collection point at the terminus of analysis pipelines. Routine use can aid quality control steps early on in data processing, reducing risk of batch effects and other downstream analysis problems.” | concepts[40] |
| concept | Advances in next-generation sequencing — not defined in this paper's own words; technical progress increasing data volume — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concepts[41] |
| concept | avalanche of data — not defined in this paper's own words; rapidly expanding sequencing output — “Advances in next-generation sequencing are leading to an avalanche of data. Whilst opening doors to new analysis types and experimental designs, expanding sample numbers make studies increasingly vulnerable to confounding batch effects ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality control measures.” | concepts[42] |
Full text — tagged
MultiQCterm: summarize analysis resultsterm for multiple toolsterm and samples in a single reportterm Abstract Motivation: Fast and accurate quality controlterm is essential for studies involving next-generation sequencing dataterm. Whilst numerous toolsterm exist to quantify QC metricsterm, there is no common approach to flexibly integrate these across toolsterm and large sample sets. Assessing analysis resultsterm across an entire project can be time consuming and error prone; batch effectsterm and outlier samplesterm can easily be missed in the early stages of analysis.
Results: We present MultiQCterm, a toolterm to create a single reportterm visualising output from multiple toolsterm across many samples, enabling global trendsterm and biases to be quickly identified. MultiQCterm can plot data from many common bioinformatics toolsterm and is built to allow easy extensionterm and customization.
Availability and implementation: MultiQCterm is available with an GNU GPLv3 license on GitHub, the Python Package Index and Bioconda. Documentation and example reportsterm are available at http://multiqcterm.info Contact: phil.ewelspersName@scilifelab.se 1 Introduction Advances in next-generation sequencingterm are leading to an avalanche of dataterm. Whilst opening doors to new analysis typesterm·var and experimental designs, expanding sample numbersterm make studies increasingly vulnerable to confounding batch effectsterm ( Leek et al., 2010 ; Meyer and Liu 2014 ; Taub et al., 2010 ). Such biases are often subtle and difficult to detect and require careful quality controlterm measures.
Most bioinformatics programs produce logsterm detailing their results. Dedicated QC toolsterm such as FastQC ( http://www.bioinformatics.babraham.ac.uk/projects/fastqc ), Qualimap ( Okonechnikov et al. , 2015 ) and RSeQC ( Wang et al. , 2012 ) are excellent at highlighting potential problems in data. However, nearly all of these logsterm and reportsterm are produced on a per-sample basis, requiring the user to find and compile QC results. This process is time consuming, repetitive and complex, making it prone to errors.
MultiQCterm addresses this problem by scanning given analysis directories for logterm files and QC reportsterm, creating a single summary reportterm·def visualizing results across all samples. Collecting data within a single reportterm provides a fast way to scan key statisticsterm·var quickly and easily ( Fig. 1 ). Shared plotsterm allow accurate comparison between samples, allowing detection of subtle differences not noticeable when switching between different files. Data visualization aids batch effectterm detection and minimizes the risk of confounding factorsterm affecting the results of the study. MultiQCterm is the first toolterm of its type within the field; it has the potential to greatly improve quality controlterm and reporting for researchers involved in next-generation sequencing, removing the need for custom comparative scriptsterm.
Top of a typical MultiQCterm reportterm·def. The general statistics table can be seen with metrics from a number of different toolsterm gathered for each sample (Color version of this figure is available at Bioinformatics online.) 2 Materials and methods 2.1 Running MultiQCterm MultiQCterm is written in Python and run on the command line; specified directories are searched recursively for any recognized filesterm. Submodulesterm for each supported toolterm are run, querying input files using configurable search settings. If any logterm files are found they are parsed, otherwise the moduleterm exits silently. Once all submodulesterm have finished, the specified templateterm is loaded and parsed using the Jinja2 packageterm to render the final reportterm·def. Parsed dataterm is saved as tab delimited text, YAML or JSON for downstream use. Because submodulesterm only contribute to the reportterm·def if they find logsterm, MultiQCterm is run in the same way for every analysis typeterm·var. At the time of writing, MultiQCterm supports 22 common bioinformatics toolsterm including aligners, processing toolsterm and QC programs.
2.2 MultiQCterm reportsterm MultiQCterm generates a single s elf-contained HTML reportterm which can be shared and opened in any modern web browser. Reportsterm render plots using the JavaScript plotting library HighChartsterm ( http://www.highchartsterm.com ). Plots are resizeable and interactive, some with click and drag zooming. Samples can be renamed, hidden and highlighted using a report toolboxterm. Plots can be exported in a range of publication-ready formats.
Reportsterm with hundreds of samples become too large for use with HighChartsterm; instead MultiQCterm switches to rendering plots as images at run-time using the Python plotting library MatPlotLibterm ( Hunter, 2007) . Images are embedded within the HTML, maintaining a stand-alone file with consistent file size. These static reportsterm are also suitable for conversion to PDF using toolsterm such as Pandoc ( http://pandoc.org ).
Each reportterm·def contains an interactive walk through of features. Tutorial videos can be found at http://multiqcterm.info along with tutorials and documentation describing installation, usage and troubleshooting.
2.3 Extending MultiQCterm MultiQCterm supports a lot of common bioinformatics toolsterm but it is inevitable that research groups may have their own bespoke scripts or require other customization. To accommodate this, MultiQCterm is built in such a way that custom codeterm can be tied into its functionality easily. Code hooksterm allow external pluginsterm to access and modify the internal workings of the program. The use of Python setuptools entry pointsterm allows modulesterm, templatesterm and pluginsterm·def to be kept within a separate code base, whilst still executing as part of the main MultiQCterm program.
Extensive documentation makes adding to MultiQCterm simple; four new modulesterm have been contributed by users to date and we are aware of at least three pluginsterm·def written by different research groups. Adoption by the bioinformatics community has been rapid: MultiQCterm has been downloaded thousands of times within the past few months and is already integrated as standard within the popular bcbio-nextgen analysis toolkit ( http://bcb.io ).
3 Typical applications 3.1 Single cellterm data and population studiesterm Single cellterm and population studiesterm are perhaps the perfect examples of large projects where accurate quality controlterm of numerous datasets is critical. MultiQCterm is able to parse data for thousands of samples within minutes, adapting reportterm·def output as required. Parsed dataterm saved by MultiQCterm can be used for post-processingterm and dataset filteringterm. Reportsterm reveal overall analysis success and make it easy to identify abnormal samples.
3.2 Sequencing facilities MultiQCterm was originally developed for use in a high throughput sequencing facilityterm. Reportsterm give the overview required to spot failing samples and highlighting helps to identify groups of samples behaving in an irregular manner.
Pluginsterm·def allow integration with in-house systems: we have written the MultiQC_NGI pluginterm which inserts meta data from our LIMSterm into reportsterm and stores summary results parsed by MultiQCterm in our database. This functionality is enormously powerful, facilitating large scale internal data collection that would otherwise require numerous custom scripts. Templatesterm allow reportterm·def branding and reportsterm are self-contained, making MultiQCterm an ideal toolterm for creating delivery reportsterm.
4 Conclusion As the field of next-generation sequencing matures, there are increasing numbers of bioinformatics toolsterm producing ever more verbose descriptions of data. Integrating these statistics across toolsterm with large sample sets is difficult and time-consuming. MultiQCterm can automate the parsing of this metadata, providing powerful visualizations with a simple interface. Extensionterm and data export allow MultiQCterm to function as a central collection pointterm at the terminus of analysis pipelinesterm. Routine use can aid quality controlterm steps early on in data processing, reducing risk of batch effectsterm and other downstream analysis problems.
Acknowledgements The authors would like to thank D. Klevebring, L. Pantano, G. Carrasco and R. Andeer for contributed modulesterm and discussion.
Funding This work was supported by the Science for Life Laboratory and the National Genomics Infrastructure, NGI.
Conflict of Interest : none declared.
References