Zachary D. StephenspersName, Skylar Y. LeepersName, Faraz FaghripersName (2015date). “Big Data: Astronomical or Genomical?.” PLOS Biologytitle·j, UrbanaplaceName.
Object of study — The computational demands of genomicsterm·dom in 2025date - acquisitionterm·mth, storageterm, distributionterm and analysisterm - set against astronomyterm·dom, YouTube and Twitter.
Research question — Genomicsterm·dom is growing fast, but is it about to become the most computationally demanding Big Dataterm·dom science, or merely one among several?
Contribution — A like-for-like projection across four domainsterm·var showing genomicsterm·dom is on par with or ahead of all of them on every phase of the data lifecycleterm·dom.
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | growth of genomic data volume — “The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ).” | |
| claim-type | prediction — “We therefore estimate between 100 million and as many as 2 billion human genomes could be sequenced by 2025, representing four to five orders of magnitude growth in ten years and far exceeding the growth for the three other Big Data domains.” | |
| method — evidence | — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | |
| object of study — evidence | — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | |
| research question — evidence | — “Genomics is a Big Data science and is going to get much bigger, very soon, but it is not known whether the needs of genomics will exceed other Big Data domains.” | |
| contribution — evidence | — “Our estimates show that genomics is a “four-headed beast”—it is either on par with or the most demanding of the domains analyzed here in terms of data acquisition, storage, distribution, and analysis.” | |
| transfer | from computer science / data science (machine learning, big-data computing): large-scale machine learning and big-data computing infrastructure applied to genomics — “large-scale machine learning systems, and a computing infrastructure that can support flexible and dynamic queries” | transfers[0] |
| transfer | from experimental particle physics: the data-retention practice of particle physics - discard the raw data after processing and keep only the reduced product - held up as the lesson genomics should learn as sequencing volumes outgrow storage — “For RNA-seq and other ‘omics applications, genomics will benefit greatly from the lessons learned in particle physics, in which in most cases raw data are discarded almost as fast as they are generated in favor of higher level and greatly compressed summaries.” | transfers[1] |
| mechanism | a common four-part framework - the dataset life cycle - is what makes four unlike domains comparable; without it the comparison has no cells to fill — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | mechanisms[0] |
| mechanism | acquisition architecture differs by domain: astronomy centralises at a few facilities, YouTube and Twitter distribute under standard protocols, genomics distributes with no standard format at all — “By contrast, YouTube and Twitter acquire data in a highly distributed manner, but under a few standardized protocols.” | mechanisms[1] |
| mechanism | the projection is anchored on a doubling time, and holds under three different ones - seven, twelve or eighteen months - which is what makes it robust rather than a single extrapolation — “Interestingly, even at the more conservative estimates of doubling every 12 months (Illumina’s current own estimate [ 12 ]) or every 18 months (equivalent to Moore’s law), we should reach exabase-scale genomics well within the next decade.” | mechanisms[2] |
| mechanism | a domain's data footprint is set by what it DISCARDS, not only by what it acquires - particle physics generates massive volumes and keeps almost none — “Particle physics also produces massive quantities of raw data, although the footprint is surprisingly limited since the vast majority of data are discarded soon after acquisition using the processing power that is coupled to the sensors [ 8 ].” | mechanisms[3] |
| builds-on | the OmicsMaps catalogue of sequencing instruments and Illumina's own growth estimate — “Interestingly, even at the more conservative estimates of doubling every 12 months (Illumina’s current own estimate [ 12 ]) or every 18 months (equivalent to Moore’s law), we should reach exabase-scale genomics well within the next decade.” | claim_relations[0] |
| extends | the Big Data literature on astronomy, computational social science and video platforms — “Projecting to the year 2025, we compared genomics with three other major generators of Big Data: astronomy, YouTube, and Twitter.” | claim_relations[1] |
| sense | Big Data - as a comparison class of domains, each with its own acquisition architecture, rather than as a property a dataset has — “Projecting to the year 2025, we compared genomics with three other major generators of Big Data: astronomy, YouTube, and Twitter.” | senses[0] |
| sense | data volume - as ACQUIRED, which for particle physics is enormous — “Particle physics also produces massive quantities of raw data, although the footprint is surprisingly limited since the vast majority of data are discarded soon after acquisition using the processing power that is coupled to the sensors [ 8 ].” | senses[1] |
| sense | data volume - as STORED, which for genomics is a small fraction of what is produced and for particle physics is smaller still — “However, the 3.6 petabases represent a small fraction of the total produced; most of it is not yet in these archives.” | senses[2] |
| sense | Storage / distribution / analysis - three lifecycle components alongside acquisition — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | senses[3] |
| sense | Compressive genomics - storage reduction constrained by decompression time and fidelity — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | senses[4] |
| sense | Long-read sequencing - mature technology that may remove the need to retain raw reads — “Are the emerging “third-generation” single-molecule sequencing technologies with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence data may not need to be stored at all” | senses[5] |
| sense | Delta encoding - retention of reference-relative variant lists instead of complete genomes — “More significant reductions in storage demand will come when improvements in sequencing accuracy and database comprehensiveness reach the point at which genome sequences themselves do not need to be stored, just the list of variants relative to a reference collection (“delta encoding”)” | senses[6] |
| sense | Streaming omics analysis - real-time inference performing at least as well as nonstreaming methods — “In contrast, we do see great opportunities for data reduction and real-time analysis of other ‘omics analysis. For example, once sequencing becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithms have been published for this purpose, performing as well as or superior to their nonstreaming counterparts” | senses[7] |
| sense | Hierarchical genomic archive - data and summaries exposed at multiple compression and latency levels — “To make it practical to search and query through such vast collections, the data will be stored in hierarchical systems that make data and their statistical summaries available at different levels of compression and latency” | senses[8] |
| sense | Cloud genomic analysis - code moves to remote machines already holding the data — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | senses[9] |
| sense | Homomorphic encryption - controlled analysis of encrypted data without revealing raw genomes — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | senses[10] |
| sense | Variant calling - computationally intensive identification of variants in each individual genome — “For population and medical genomics, identifying the genomic variants in each individual genome is currently one of the most computationally complex phases” | senses[11] |
| sense | Whole-genome alignment - comparative method for phylogeny reconstruction and genome annotation — “Whole genome alignment is another important form of genomic data analysis, used for a variety of goals, from phylogeny reconstruction to genome annotation via comparative methodologies. Just a single whole genome alignment between human and mouse consumes ~100 CPU hours [ 43 ]. Aligning all pairs of the ~2.5 million species expected to be available by 2025 amounts to 50–100 trillion such whole genome alignments, which would need to be six orders of magnitude faster than possible today” | senses[12] |
| sense | I/O bottleneck - data shuttling between storage and processors rather than CPU arithmetic — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | senses[13] |
| sense | Genomic data sharing - protection against isolated collections too underpowered for subtle signals — “If we do not commit as a scientific community to sharing now, we run the risk of establishing thousands of isolated, private data collections, each too underpowered to allow subtle signals to be extracted” | senses[14] |
| edge | acquisition —part-of→ life cycle — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | concept_edges[0] |
| edge | genomics —is-a→ Big Data — “Genomics is a Big Data science and is going to get much bigger, very soon, but it is not known whether the needs of genomics will exceed other Big Data domains.” | concept_edges[1] |
| audit | Big Data —operationalised-as→ life cycle — Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the life cycle is the frame for comparing the domains; it is not an operationalisation of Big Data. — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | concept_edges[4] |
| note | storage —part-of→ life cycle — Storage is explicitly one of four components comprising the dataset lifecycle. — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concept_edges[7] |
| note | distribution —part-of→ life cycle — Distribution is explicitly one of four components comprising the dataset lifecycle. — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concept_edges[8] |
| note | analysis —part-of→ life cycle — Analysis is explicitly one of four components comprising the dataset lifecycle. — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concept_edges[9] |
| audit | effective data compression —causes→ storage requirements — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says these needs CAN be diminished with effective compression; hedged. — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | concept_edges[10] |
| effect direction | effective data compression —causes→ storage requirements — reduces — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” — “These needs can be diminished with effective data compression” | concept_edges[10] |
| note | effective data compression —causes→ storage requirements — The paper states storage needs can be diminished with effective compression. — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | concept_edges[10] |
| effect direction | cloud computing —causes→ bandwidth for distribution of genomic data — reduces — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” — “cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data” | concept_edges[11] |
| note | cloud computing —causes→ bandwidth for distribution of genomic data — Cloud computing is stated to decrease distribution bandwidth. — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concept_edges[11] |
| effect direction | remote machines that already have data —causes→ computing resources necessary for distribution — reduces — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” — “thus significantly reducing the computing resources necessary for distribution” | concept_edges[12] |
| note | remote machines that already have data —causes→ computing resources necessary for distribution — Running applications where data reside is stated to reduce distribution resources. — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concept_edges[12] |
| audit | CPU capabilities —contrasted-with→ input/output (I/O) hardware — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says the bigger bottleneck MAY not be in CPU but in I/O hardware; hedged. — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concept_edges[13] |
| note | CPU capabilities —contrasted-with→ input/output (I/O) hardware — The paper hedges that the larger future bottleneck may be I/O rather than CPU capability. — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concept_edges[13] |
| weight | CPU capabilities —contrasted-with→ input/output (I/O) hardware — 0.8 — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concept_edges[13] |
| for which use | limitation — the paper's central comparative question, which it leaves open: IT IS NOT KNOWN WHETHER THE NEEDS OF GENOMICS WILL EXCEED OTHER BIG DATA DOMAINS — “it is not known whether the needs of genomics will exceed other Big Data domains” | limitations[0] |
| for which use | limitation — the far end of the projections: THE PROJECT GROWTH BECOMES MORE UNPREDICTABLE THE FURTHER OUT WE CONSIDER — “Over the next ten years, we expect sequencing capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider.” | limitations[1] |
| for which use | limitation — every storage figure quoted: OUR ESTIMATES HERE IGNORE THE REPLICATION FACTOR THAT MULTIPLIES STORAGE NEEDS BY ~4, FOR REDUNDANCY — “(Our estimates here ignore the “replication factor” that multiplies storage needs by ~4, for redundancy.)” | limitations[2] |
| for which use | limitation — the scope of the four-domain comparison: WE DO NOT INCLUDE THE DOMAIN IN FULL DETAIL HERE - particle physics, a fifth massive producer, is left out — “Consequently, we do not include the domain in full detail here, although that model of rapid filtering and analysis will surely play an increasingly important role in genomics as the field matures.” | limitations[3] |
| for which use | limitation — the storage projections specifically: PROJECTIONS OF STORAGE REQUIREMENTS FOR SEQUENCE DATA DEPEND ON THE ACCURACY AND APPLICATION OF THE SEQUENCING — “Projections of storage requirements for sequence data depend on the accuracy and application of the sequencing.” | limitations[4] |
| duplicate of | limitation — limitations[3] — “Consequently, we do not include the domain in full detail here, although that model of rapid filtering and analysis will surely play an increasingly important role in genomics as the field matures.” | limitations[5] |
| for which use | limitation — the same scope as limitations[3] - four domains compared, not five — “Consequently, we do not include the domain in full detail here, although that model of rapid filtering and analysis will surely play an increasingly important role in genomics as the field matures.” | limitations[5] |
| duplicate of | limitation — limitations[1] — “Over the next ten years, we expect sequencing capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider.” | limitations[6] |
| for which use | limitation — the same scope as limitations[1] - reliability falls with horizon — “Over the next ten years, we expect sequencing capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider.” | limitations[6] |
| for which use | limitation — the baseline the whole projection is built from: THE 3.6 PETABASES REPRESENT A SMALL FRACTION OF THE TOTAL PRODUCED; MOST OF IT IS NOT YET IN THESE ARCHIVES — “However, the 3.6 petabases represent a small fraction of the total produced; most of it is not yet in these archives.” | limitations[7] |
| for which use | limitation — every bases-to-bytes conversion in the paper: AN APPROXIMATE CONVERSION FACTOR TO USE IN INTERPRETING THESE NUMBERS IS 4 BASES = 1 BYTE - stated as approximate, and it propagates — “(An approximate conversion factor to use in interpreting these numbers is 4 bases = 1 byte, though we will revisit this below.)” | limitations[8] |
| weight | single-molecule sequencing technologies → raw sequence data — 0.6 — “Are the emerging “third-generation” single-molecule sequencing technologies with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence data may not need to be stored at all” | unhoused_relations[4] |
| replication | untested | |
| no type for this | delta encoding → list of variants relative to a reference collection — Delta encoding represents genomes as reference-relative variant lists; represents-as is absent from the edge registry. — “More significant reductions in storage demand will come when improvements in sequencing accuracy and database comprehensiveness reach the point at which genome sequences themselves do not need to be stored, just the list of variants relative to a reference collection (“delta encoding”)” | unhoused_relations[0] |
| no type for this | Homomorphic encryption systems → privacy — Homomorphic encryption enables controlled queries without raw-data disclosure; enables is absent from the edge registry. — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | unhoused_relations[1] |
| no type for this | trust among institutions, scientists, and the public → connecting these resources — Trust is a prerequisite for connecting genomic resources; prerequisite-for is absent from the edge registry. — “connecting these resources requires trust among institutions, scientists, and the public to ensure the collections will be used for medical purposes and not to discriminate or penalize individuals because of their genetic makeup” | unhoused_relations[2] |
| no type for this | application programming interfaces (APIs) → discovering and querying large datasets on remote systems — APIs enable discovery and query of remote datasets; enables is absent from the edge registry. — “the community needs to develop application programming interfaces (APIs) for discovering and querying large datasets on remote systems” | unhoused_relations[3] |
| no type for this | single-molecule sequencing technologies → raw sequence data — Mature long-read methods may eliminate the need to retain raw sequence data; enables-omission is absent from the edge registry. — “Are the emerging “third-generation” single-molecule sequencing technologies with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence data may not need to be stored at all” | unhoused_relations[4] |
| limitation | it is not known whether genomics needs will exceed other Big Data domains — “it is not known whether the needs of genomics will exceed other Big Data domains” | limitations[0] |
| limitation | Projections become less reliable the further ahead they reach. — “Over the next ten years, we expect sequencing capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider.” | limitations[1] |
| limitation | Storage estimates omit the replication factor for redundancy. — “(Our estimates here ignore the “replication factor” that multiplies storage needs by ~4, for redundancy.)” | limitations[2] |
| limitation | Particle physics, a fifth massive data producer, was deliberately left out of the comparison. — “Consequently, we do not include the domain in full detail here, although that model of rapid filtering and analysis will surely play an increasingly important role in genomics as the field matures.” | limitations[3] |
| limitation | Storage projections are contingent on sequencing accuracy and application. — “Projections of storage requirements for sequence data depend on the accuracy and application of the sequencing.” | limitations[4] |
| limitation | Particle physics was excluded from the full comparison, so the framework is applied to four domains rather than to the five the authors identify. — “Consequently, we do not include the domain in full detail here, although that model of rapid filtering and analysis will surely play an increasingly important role in genomics as the field matures.” | limitations[5] |
| limitation | The authors state the projections become less reliable the further out they run. — “Over the next ten years, we expect sequencing capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider.” | limitations[6] |
| limitation | The archived data is a small and unquantified fraction of what is produced, so the baseline for the whole projection is an estimate from instrument specifications rather than a count. — “However, the 3.6 petabases represent a small fraction of the total produced; most of it is not yet in these archives.” | limitations[7] |
| limitation | Bases are converted to bytes at a stated approximate factor, which propagates into every storage figure. — “(An approximate conversion factor to use in interpreting these numbers is 4 bases = 1 byte, though we will revisit this below.)” | limitations[8] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | genomics is on par with or the most demanding of the Big Data domains — “it is either on par with or the most demanding of the domains analyzed here in terms of data acquisition, storage, distribution, and analysis” | content_tags.findings[0] |
| finding | data acquisition is expected to grow by up to two orders of magnitude — “data acquisition in these domains is expected to grow by up to two orders of magnitude” | content_tags.findings[1] |
| evidence status note | finding — QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. | content_tags.findings[1] |
| finding | human genomes may grow four to five orders of magnitude by 2025 — “We therefore estimate between 100 million and as many as 2 billion human genomes could be sequenced by 2025, representing four to five orders of magnitude growth in ten years and far exceeding the growth for the three other Big Data domains” | content_tags.findings[2] |
| finding | EVIDENCE CORRECTED (RE-GROUNDED - THE POINTER WAS WRONG, NOT THE CLAIM) · even conservative doubling (12-18 months) still yields exabase-scale genomics — “Interestingly, even at the more conservative estimates of doubling every 12 months (Illumina’s current own estimate [ 12 ]) or every 18 months (equivalent to Moore’s law), we should reach exabase-scale genomics well within the next decade.” | content_tags.findings[3] |
| retracted | finding — RE-GROUNDED - THE POINTER WAS WRONG, NOT THE CLAIM. The old evidence was verbatim, anchored and correctly cut, and it contained NONE of this claim's numbers (12, 18). The paper states the claim in one sentence elsewhere and the tagger did not use it. the project, 2026-08-23: 'WE HAVE SPENT THE DAY VERIFYING THAT QUOTES ARE REAL AND NONE OF IT ASKS WHETHER THEY ARE RELEVANT.' Confirmed by reading the claim, the old span and the candidate together; the candidate carries every number AND the claim's own content words. | content_tags.findings[3] |
| finding | Genomics is a “four-headed beast”: on par with or the most demanding of the domains analysed in data acquisition, storage, distribution and analysis. — “Our estimates show that genomics is a “four-headed beast”—it is either on par with or the most demanding of the domains analyzed here in terms of data acquisition, storage, distribution, and analysis.” | content_tags.findings[4] |
| finding | Sequence data has been doubling roughly every seven months. — “The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ).” | content_tags.findings[5] |
| finding | The four Big Data domains differ sharply in how data are acquired. — “The four Big Data domains differ sharply in how data are acquired.” | content_tags.findings[6] |
| finding | Across all domains, acquisition volume is expected to rise by up to two orders of magnitude in a decade. — “In short, data acquisition in these domains is expected to grow by up to two orders of magnitude in the next decade.” | content_tags.findings[7] |
| quantity | 7 months — doubling time of sequence data produced — “doubling approximately every seven months” | content_tags.quantities[0] |
| quantity | 2,500 instruments — high-throughput sequencers worldwide — “The OmicsMaps catalog of all known sequencing instruments in the world [ 11 ] reports that currently there are more than 2,500 high-throughput instruments” | content_tags.quantities[1] |
| quantity | 1,000 centers in 55 countries — global spread of sequencing centers — “The OmicsMaps catalog of all known sequencing instruments in the world [ 11 ] reports that currently there are more than 2,500 high-throughput instruments, manufactured by several different companies, located in nearly 1,000 sequencing centers in 55 countries” | content_tags.quantities[2] |
| quantity | 3.6 petabases — raw sequence data held in the SRA archive — “The SRA currently contains more than 3.6 petabases of raw sequence data” | content_tags.quantities[3] |
| quantity | 7.5 terabytes/second — ASKAP telescope image-data acquisition rate — “currently acquires 7.5 terabytes/second of sample image data” | content_tags.quantities[4] |
| quantity | 500 million tweets/day — current Twitter data generation — “Twitter generates 500 million tweets/day” | content_tags.quantities[5] |
| quantity | doubling every seven months — observed growth rate of sequence data production, the basis of the central projection — “The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ).” | content_tags.quantities[6] |
| quantity | one zettabase per year by 2025 — projected genomic acquisition under the seven-month doubling — “If the growth continues at the current rate by doubling every seven months, then we should reach more than one exabase of sequence per year in the next five years and approach one zettabase of sequence per year by 2025 ( Fig 1 , Table 1 ).” | content_tags.quantities[7] |
| quantity | ~25 zettabytes per year by 2025 — projected astronomy acquisition, from ASKAP scaling 7.5 to 750 terabytes/second — “the Australian Square Kilometre Array Pathfinder (ASKAP) project currently acquires 7.5 terabytes/second of sample image data, a rate projected to increase 100-fold to 750 terabytes/second (~25 zettabytes per year) by 2025” | content_tags.quantities[8] |
| quantity | 1–2 exabytes of video per year by 2025 — projected YouTube acquisition, from 300 hours uploaded per minute today — “YouTube currently has 300 hours of video being uploaded every minute, and this could grow to 1,000–1,700 hours per minute (1–2 exabytes of video data per year) by 2025 if we extrapolate from current trends ( S1 Note ).” | content_tags.quantities[9] |
| quantity | 500 million tweets/day at ~3 kilobytes each — Twitter's acquisition baseline, the smallest of the four by orders of magnitude — “Today, Twitter generates 500 million tweets/day, each about 3 kilobytes including metadata ( S2 Note ).” | content_tags.quantities[10] |
| quantity | 3.6 petabases archived; ~35 petabases/year capacity — what is in the SRA against what the world's instruments can produce - the gap the paper flags — “However, the 3.6 petabases represent a small fraction of the total produced; most of it is not yet in these archives. Based on manufacturer specifications of the instruments, we estimate the current worldwide sequencing capacity to exceed 35 petabases per year, including the sixteen Illumina X-Ten systems that have been sold so far [ 13 ], each with a capacity of ~2 petabases per year [ 14 ].” | content_tags.quantities[11] |
| variable | worldwide sequencing capacity (estimated/projected throughput driver) — “worldwide sequencing capacity to exceed 35 petabases per year” | content_tags.variables[0] |
| variable | conversion factor (unit conversion for interpreting volumes) — “An approximate conversion factor to use in interpreting these numbers is 4” | content_tags.variables[2] |
| variable | domain (independent) — “Projecting to the year 2025, we compared genomics with three other major generators of Big Data: astronomy, YouTube, and Twitter.” | content_tags.variables[4] |
| definition | Sequence Read Archive (SRA) — “Sequence Read Archive (SRA) maintained by the United States National Institutes of Health” | content_tags.definitions[0] |
| definition | WGS — “WGS, whole genome” [UNVERIFIED AGAINST THE STAGED TEXT] | content_tags.definitions[1] |
| evidence status note | definition — marked quote-not-in-current-staged-text - today's merge gate would REFUSE this quote: Paper.locate() does not find it in the current staged text. | content_tags.definitions[1] |
| definition | The Cancer Genome Atlas (TCGA) — “The Cancer Genome Atlas (TCGA), aggregating over several thousand tumor/normal genome pairs” | content_tags.definitions[2] |
| definition | Exome Aggregation Consortium (ExAC) — “the Exome Aggregation Consortium (ExAC),” | content_tags.definitions[3] |
| definition | life cycle — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | content_tags.definitions[4] |
| definition | genomics — “Genomics is a Big Data science and is going to get much bigger, very soon, but it is not known whether the needs of genomics will exceed other Big Data domains.” | content_tags.definitions[5] |
| definition | conversion factor — “(An approximate conversion factor to use in interpreting these numbers is 4 bases = 1 byte, though we will revisit this below.)” | content_tags.definitions[6] |
| key-terms | genomics, Big Data, sequencing, data acquisition, astronomy, genome, storage, petabases, exabase, computational challenges, acquisition, distribution, analysis, life cycle, zettabase, sequencing capacity, Moore’s law, particle physics |
| concept | genomics — “For genomics, data acquisition is highly distributed and involves heterogeneous formats.” | concepts[0] |
| concept | big data — “Genomics is a Big Data science and is going to get much bigger, very soon, but it is not known whether the needs of genomics will exceed other Big Data domains.” | concepts[1] |
| concept | life cycle — not defined in this paper's own words; the four-phase framework - acquisition, storage, distribution, analysis - imposed uniformly so that unlike domains can be compared cell by cell — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis ( Table 1 ).” | concepts[4] |
| concept | acquisition — not defined in this paper's own words; the first phase: how and where data enters a domain - centralised, distributed-standardised, or distributed-heterogeneous — “Data Acquisition The four Big Data domains differ sharply in how data are acquired” | concepts[5] |
| concept | astronomy — not defined in this paper's own words; the comparison domain with the longest Big Data history and the most centralised acquisition — “Most astronomy data are acquired from a few highly centralized facilities [ 9 ].” | concepts[6] |
| concept | particle physics — not defined in this paper's own words; the domain excluded from the comparison because it discards nearly everything it acquires - large generation, small footprint — “Particle physics also produces massive quantities of raw data, although the footprint is surprisingly limited since the vast majority of data are discarded soon after acquisition using the processing power that is coupled to the sensors [ 8 ].” | concepts[7] |
| concept | sequencing capacity — not defined in this paper's own words; what the world's installed instruments could produce per year, estimated from manufacturer specifications rather than from what was archived — “Based on manufacturer specifications of the instruments, we estimate the current worldwide sequencing capacity to exceed 35 petabases per year, including the sixteen Illumina X-Ten systems that have been sold so far [ 13 ], each with a capacity of ~2 petabases per year [ 14 ].” | concepts[8] |
| concept | Moore’s law — not defined in this paper's own words; invoked as the slowest of three candidate doubling times, and used as a conservative bound rather than as a claim about hardware — “Interestingly, even at the more conservative estimates of doubling every 12 months (Illumina’s current own estimate [ 12 ]) or every 18 months (equivalent to Moore’s law), we should reach exabase-scale genomics well within the next decade.” | concepts[9] |
| concept | storage — not defined in this paper's own words; dataset-lifecycle component covering retained data and archive capacity — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concepts[10] |
| concept | distribution — not defined in this paper's own words; dataset-lifecycle component covering data movement and bandwidth — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concepts[11] |
| concept | analysis — not defined in this paper's own words; dataset-lifecycle component covering computation that derives usable information — “To compare these four disparate domains, we considered the four components that comprise the “life cycle” of a dataset: acquisition, storage, distribution, and analysis” | concepts[12] |
| concept | effective data compression — not defined in this paper's own words; method that can diminish projected genomic storage requirements — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | concepts[13] |
| concept | storage requirements — not defined in this paper's own words; required archive capacity that compression can diminish — “Projections of storage requirements for sequence data depend on the accuracy and application of the sequencing. For every 3 billion bases of human genome sequence, 30-fold more data (~100 gigabases) must be collected because of errors in sequencing, base calling, and genome alignment. This means that as much as 2–40 exabytes of storage capacity will be needed by 2025 just for the human genomes ( S3 Note ). These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | concepts[14] |
| concept | compressive genomics — not defined in this paper's own words; genomic compression setting constrained by decompression time and fidelity — “These needs can be diminished with effective data compression [ 27 ], but decompression times and fidelity are a major concern in compressive genomics” | concepts[15] |
| concept | single-molecule sequencing technologies — not defined in this paper's own words; long-read platforms that may eventually eliminate retention of raw reads — “Are the emerging “third-generation” single-molecule sequencing technologies with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence data may not need to be stored at all” | concepts[16] |
| concept | raw sequence data — not defined in this paper's own words; primary read data that mature long-read methods may make unnecessary to retain — “Are the emerging “third-generation” single-molecule sequencing technologies with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence data may not need to be stored at all” | concepts[17] |
| concept | delta encoding — not defined in this paper's own words; storage representation retaining variants relative to a reference collection — “More significant reductions in storage demand will come when improvements in sequencing accuracy and database comprehensiveness reach the point at which genome sequences themselves do not need to be stored, just the list of variants relative to a reference collection (“delta encoding”)” | concepts[18] |
| concept | list of variants relative to a reference collection — not defined in this paper's own words; reference-relative representation proposed instead of storing complete genomes — “More significant reductions in storage demand will come when improvements in sequencing accuracy and database comprehensiveness reach the point at which genome sequences themselves do not need to be stored, just the list of variants relative to a reference collection (“delta encoding”)” | concepts[19] |
| concept | data reduction — not defined in this paper's own words; real-time conversion of raw omics measurements into smaller summaries — “In contrast, we do see great opportunities for data reduction and real-time analysis of other ‘omics analysis. For example, once sequencing becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithms have been published for this purpose, performing as well as or superior to their nonstreaming counterparts” | concepts[20] |
| concept | real-time analysis — not defined in this paper's own words; on-the-fly inference that permits raw RNA-seq reads to be discarded — “In contrast, we do see great opportunities for data reduction and real-time analysis of other ‘omics analysis. For example, once sequencing becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithms have been published for this purpose, performing as well as or superior to their nonstreaming counterparts” | concepts[21] |
| concept | “streaming” algorithms — not defined in this paper's own words; published online algorithms matching or outperforming nonstreaming counterparts — “In contrast, we do see great opportunities for data reduction and real-time analysis of other ‘omics analysis. For example, once sequencing becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithms have been published for this purpose, performing as well as or superior to their nonstreaming counterparts” | concepts[22] |
| concept | nonstreaming counterparts — not defined in this paper's own words; batch-processing comparator for streaming omics algorithms — “In contrast, we do see great opportunities for data reduction and real-time analysis of other ‘omics analysis. For example, once sequencing becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithms have been published for this purpose, performing as well as or superior to their nonstreaming counterparts” | concepts[23] |
| concept | hierarchical systems — not defined in this paper's own words; archive design exposing data and summaries at multiple compression and latency levels — “To make it practical to search and query through such vast collections, the data will be stored in hierarchical systems that make data and their statistical summaries available at different levels of compression and latency” | concepts[24] |
| concept | cloud computing — not defined in this paper's own words; remote-compute placement that reduces genomic-data distribution bandwidth — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concepts[25] |
| concept | bandwidth for distribution of genomic data — not defined in this paper's own words; network capacity reduced when computation runs where genomic data reside — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concepts[26] |
| concept | remote machines that already have data — not defined in this paper's own words; compute hosts colocated with genomic data — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concepts[27] |
| concept | computing resources necessary for distribution — not defined in this paper's own words; bandwidth and movement resources reduced by remote execution — “For large-scale analysis, cloud computing is particularly suited to decreasing the bandwidth for distribution of genomic data [ 37 ] so that applications can run on remote machines that already have data [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distribution” | concepts[28] |
| concept | data reliability — not defined in this paper's own words; cloud-genomics requirement paired with security and privacy — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | concepts[29] |
| concept | security — not defined in this paper's own words; safeguard required for medically sensitive genomic data — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | concepts[30] |
| concept | privacy — not defined in this paper's own words; protection requirement made unusually stringent by permanent genomic disclosure consequences — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | concepts[31] |
| concept | Homomorphic encryption systems — not defined in this paper's own words; controlled-query encryption that operates without disclosing raw genomic data — “new methods of data reliability and security are required to ensure privacy, much more so than for the other three domains. A serious breach of medically sensitive genomic data would have permanent consequences and could seriously hinder the development of genomic medicine. Homomorphic encryption systems, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research” | concepts[32] |
| concept | Variant calling — not defined in this paper's own words; computationally complex identification of variants in individual genomes — “For population and medical genomics, identifying the genomic variants in each individual genome is currently one of the most computationally complex phases” | concepts[33] |
| concept | Whole genome alignment — not defined in this paper's own words; pairwise genome comparison used for phylogeny and comparative annotation — “Whole genome alignment is another important form of genomic data analysis, used for a variety of goals, from phylogeny reconstruction to genome annotation via comparative methodologies. Just a single whole genome alignment between human and mouse consumes ~100 CPU hours [ 43 ]. Aligning all pairs of the ~2.5 million species expected to be available by 2025 amounts to 50–100 trillion such whole genome alignments, which would need to be six orders of magnitude faster than possible today” | concepts[34] |
| concept | CPU capabilities — not defined in this paper's own words; compute capability that may not be the dominant future analysis bottleneck — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concepts[35] |
| concept | input/output (I/O) hardware — not defined in this paper's own words; storage-to-processor shuttle identified as a possible larger Big Data bottleneck — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concepts[36] |
| concept | parallel I/O hardware and algorithms — not defined in this paper's own words; hardware-and-algorithm research response to the I/O bottleneck — “Improvements to CPU capabilities, as anticipated by Moore’s Law, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genome analysis, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Data analysis in the future may not be in CPU capabilities but in the input/output (I/O) hardware that shuttles data between storage and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithms that can effectively utilize them” | concepts[37] |
| concept | integrative approaches — not defined in this paper's own words; cross-lifecycle approach required for the genomics data problem — “New integrative approaches need to be developed that take into account the challenges in all four aspects: it is unlikely that a single advance or technology will solve the genomics data problem” | concepts[38] |
| concept | single advance or technology — not defined in this paper's own words; insufficient standalone solution contrasted with integrative approaches — “New integrative approaches need to be developed that take into account the challenges in all four aspects: it is unlikely that a single advance or technology will solve the genomics data problem” | concepts[39] |
| concept | isolated, private data collections — not defined in this paper's own words; unshared private collections at risk of inadequate statistical power — “If we do not commit as a scientific community to sharing now, we run the risk of establishing thousands of isolated, private data collections, each too underpowered to allow subtle signals to be extracted” | concepts[40] |
| concept | trust among institutions, scientists, and the public — not defined in this paper's own words; institutional, scientific, and public condition required to connect genomic resources — “connecting these resources requires trust among institutions, scientists, and the public to ensure the collections will be used for medical purposes and not to discriminate or penalize individuals because of their genetic makeup” | concepts[41] |
| concept | application programming interfaces (APIs) — not defined in this paper's own words; interfaces for discovering and querying large remote datasets — “the community needs to develop application programming interfaces (APIs) for discovering and querying large datasets on remote systems” | concepts[42] |
| concept | large-scale machine learning systems — not defined in this paper's own words; analysis component integrated with biological expertise and large computing infrastructure — “Our ultimate goal is to be able to interpret genomic sequences and answer how DNA mutations, expression changes, or other molecular measurements relate to disease, development, behavior, or evolution. Accomplishing this goal will clearly require integration of biological domain expertise, large-scale machine learning systems, and a computing infrastructure that can support flexible and dynamic queries to search for patterns over very large collections in very high dimensions” | concepts[43] |
| concept | data science into graduate, undergraduate, and high-school curricula — not defined in this paper's own words; education route proposed for training future quantitative genomics specialists — “starting with integrating data science into graduate, undergraduate, and high-school curricula to train the next generations of quantitative biologists, bioinformaticians, and computer scientists and engineers” | concepts[44] |
| exponential-growth doubling time | — “The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ).” | content_tags.statistics[0] |
| extrapolation projection | — “YouTube currently has 300 hours of video being uploaded every minute, and this could grow to 1,000–1,700 hours per minute (1–2 exabytes of video data per year) by 2025 if we extrapolate from current trends ( S1 Note ).” | content_tags.statistics[1] |
| order-of-magnitude orders of magnitude | — “In short, data acquisition in these domains is expected to grow by up to two orders of magnitude in the next decade.” | content_tags.statistics[2] |
| claim type | prediction — a second reading reaching the same value; recorded: prediction — “If the growth continues at the current rate by doubling every seven months, then we should reach more than one exabase of sequence per year in the next five years and approach one zettabase of sequence per year by 2025 ( Fig 1 , Table 1 ).” | alternatives.claim_type |
| method | comparative — a second reading DIFFERING from the recorded value; recorded: computational-model — “Projecting to the year 2025, we compared genomics with three other major generators of Big Data: astronomy, YouTube, and Twitter.” | alternatives.method |
| phenomenon | exponential growth of genomic sequence data relative to other Big Data domains — a second reading DIFFERING from the recorded value; recorded: growth of genomic data volume — “The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ).” | alternatives.phenomenon |
Full text — tagged
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
Abstract Genomicsterm·dom is a Big Dataterm·dom science and is going to get much bigger, very soon, but it is not known whether the needs of genomicsterm·dom will exceed other Big Dataterm·dom domainsterm·var. Projecting to the year 2025date, we compared genomicsterm·dom with three other major generators of Big Dataterm·dom: astronomyterm·dom, YouTube, and Twitter. Our estimates show that genomicsterm·dom is a “four-headed beast”—it is either on par with or the most demanding of the domainsterm·var analyzed here in terms of data acquisitionterm·key, storageterm, distributionterm, and analysisterm. We discuss aspects of new technologies that will need to be developed to rise up and meet the computational challengesterm·key that genomicsterm·dom poses for the near future. Now is the time for concerted, community-wide planning for the “genomical” challenges of the next decade.
This perspective considers the growth of genomicsterm·dom over the next ten years and assesses the computational needs that we will face relative to other "Big Dataterm·dom" activities such as astronomyterm·dom, YouTube, and Twitter.
We compared genomicsterm·dom with three other major generators of Big Dataterm·dom: astronomyterm·dom, YouTube, and Twitter. Astronomyterm·dom has faced the challenges of Big Dataterm·dom for over 20 years and continues with ever-more ambitious studies of the universe. YouTube burst on the scene in 2005 and has sparked extraordinary worldwide interest in creating and sharing huge numbers of videos. Twitter, created in 2006, has become the poster child of the burgeoning movement in computational social science [ 6 ], with unprecedented opportunities for new insights by mining the enormous and ever-growing amount of textual data [ 7 ]. Particle physicsterm·dom also produces massive quantities of raw data, although the footprint is surprisingly limited since the vast majority of data are discarded soon after acquisitionterm·mth using the processing power that is coupled to the sensors [ 8 ]. Consequently, we do not include the domainterm·var in full detail here, although that model of rapid filtering and analysisterm will surely play an increasingly important role in genomicsterm·dom as the field matures.
To compare these four disparate domainsterm·var, we considered the four components that comprise the “life cycleterm·mth” of a dataset: acquisitionterm·mth, storageterm, distributionterm, and analysisterm ( Table 1 ).
Table 1. Four domainsterm·var of Big Dataterm·dom in 2025date.
In each of the four domainsterm·var, the projected annual storageterm and computing needs are presented across the data lifecycleterm·dom.
Data Phase Astronomyterm·dom Twitter YouTube Genomicsterm·dom Acquisitionterm·mth 25 zetta-bytes/year 0.5–15 billion tweets/year 500–900 million hours/year 1 zetta-bases/year Storageterm 1 EB/year 1–17 PB/year 1–2 EB/year 2–40 EB/year Analysisterm In situ data reductionterm Topic and sentiment mining Limited requirements Heterogeneous data and analysisterm Real-time processing Metadata analysisterm Variant callingterm, ~2 trillion central processing unit (CPU) hours Massive volumes All-pairs genometerm·key alignments, ~10,000 trillion CPU hours Distributionterm Dedicated lines from antennae to server (600 TB/s) Small units of distributionterm Major component of modern user’s bandwidth (10 MB/s) Many small (10 MB/s) and fewer massive (10 TB/s) data movement Data Acquisitionterm·key The four Big Dataterm·dom domainsterm·var differ sharply in how data are acquired. Most astronomyterm·dom data are acquired from a few highly centralized facilities [ 9 ]. By contrast, YouTube and Twitter acquire data in a highly distributed manner, but under a few standardized protocols. Astronomyterm·dom, YouTube, and Twitter are expected to show continued dramatic growth in the volume of data to be acquired. For example, the Australian Square Kilometre Array Pathfinder (ASKAP) project currently acquires 7.5 terabytes/second of sample image data, a rate projected to increase 100-fold to 750 terabytes/second (~25 zettabytes per year) by 2025date [ 9 , 10 ]. YouTube currently has 300 hours of video being uploaded every minute, and this could grow to 1,000–1,700 hours per minute (1–2 exabytes of video data per year) by 2025date if we extrapolate from current trends ( S1 Note ). Today, Twitter generates 500 million tweets/day, each about 3 kilobytes including metadata ( S2 Note ). While this figure is beginning to plateau, a projected logarithmic growth rate would suggest a 2.4-fold growth by 2025date, to 1.2 billion tweets per day, 1.36 petabytes/year. In short, data acquisitionterm·key in these domainsterm·var is expected to grow by up to two orders of magnitude in the next decade.
For genomicsterm·dom, data acquisitionterm·key is highly distributed and involves heterogeneous formats. The rate of growth over the last decade has also been truly astonishing, with the total amount of sequence data produced doubling approximately every seven months ( Fig 1 ). The OmicsMaps catalog of all known sequencingterm·mth instruments in the world [ 11 ] reports that currently there are more than 2,500 high-throughput instruments, manufactured by several different companies, located in nearly 1,000 sequencingterm·mth centers in 55 countries in universities, hospitals, and other research laboratories. These centers range in size from small laboratories with a few instruments generating a few terabases per year to large dedicated facilities producing several petabasesterm·key a year. (An approximate conversion factorterm·var to use in interpreting these numbers is 4 bases = 1 byte, though we will revisit this below.) Fig 1. Growth of DNA sequencingterm·mth.
The plot shows the growth of DNA sequencingterm·mth both in the total number of human genomesterm·key sequenced (left axis) as well as the worldwide annual sequencing capacityterm·mth (right axis: Tera-basepairs (Tbp), Peta-basepairs (Pbp), Exa-basepairs (Ebp), Zetta-basepairs (Zbps)). The values through 2015 are based on the historical publication record, with selected milestones in sequencingterm·mth (first Sanger through first PacBio human genometerm·key published) as well as three exemplar projects using large-scale sequencingterm·mth: the 1000 Genomesterm·key Project, aggregating hundreds of human genomesterm·key by 2012 [ 3 ]; The Cancer Genome Atlas (TCGA)term·def, aggregating over several thousand tumor/normal genometerm·key pairs [ 4 ]; and the Exome Aggregation Consortium (ExAC)term·def, aggregating over 60,000 human exomes [ 5 ]. Many of the genomesterm·key sequenced to date have been whole exome rather than whole genometerm·key, but we expect the ratio to be increasingly favored towards whole genometerm·key in the future. The values beyond 2015 represent our projection under three possible growth curves as described in the main text.
The raw sequencingterm·mth reads used in most published studies are archived at either the Sequence Read Archive (SRA)term·def maintained by the United States National Institutes of Health National Center for Biotechnology Information (NIH/NCBI) or one of the international counterparts. The SRA currently contains more than 3.6 petabasesterm·key of raw sequence dataterm ( S1 Fig ), which reflects the ~32,000 microbial genomesterm·key, ~5,000 plant and animal genomesterm·key, and ~250,000 individual human genomesterm·key that have been sequenced or are in progress thus far [ 12 ]. However, the 3.6 petabasesterm·key represent a small fraction of the total produced; most of it is not yet in these archives. Based on manufacturer specifications of the instruments, we estimate the current worldwide sequencing capacityterm·var to exceed 35 petabasesterm·key per year, including the sixteen Illumina X-Ten systems that have been sold so far [ 13 ], each with a capacity of ~2 petabasesterm·key per year [ 14 ].
Over the next ten years, we expect sequencingterm·mth capacities will continue to grow very rapidly, although the project growth becomes more unpredictable the further out we consider. If the growth continues at the current rate by doubling every seven months, then we should reach more than one exabaseterm·key of sequence per year in the next five years and approach one zettabaseterm·key of sequence per year by 2025date ( Fig 1 , Table 1 ). Interestingly, even at the more conservative estimates of doubling every 12 months (Illumina’s current own estimate [ 12 ]) or every 18 months (equivalent to Moore’s lawterm·dom), we should reach exabase-scale genomicsterm·dom well within the next decade. We anticipate this sequencingterm·mth will encompass genometerm·key sequences for most of the approximately 1.2 million described species of plants and animals [ 15 ]. With these genomesterm·key, plus those of thousands of individuals of “high value” species for energy, environmental, and agricultural reasons, we estimate that there will be at least 2.5 million plant and animal genometerm·key sequences by 2025date. For example, the genomicsterm·dom powerhouse BGI, in conjunction with the International Rice Research Institute and the Chinese Academy of Agricultural Sciences, has already sequenced 3,000 varieties of rice [ 16 ] and announced a massive project of their own to sequence one million plant and animal genomesterm·key [ 17 ]. The Smithsonian Institute also has similar plans to “capture and catalog all the DNA from the world’s flora and fauna.” There also will be genomesterm·key for several millions of microbes, with explosive growth projected for both medical and environmental microbe metagenomic sequencingterm·mth [ 18 , 19 ].
These estimates, however, are dwarfed by the very reasonable possibility that a significant fraction of the world’s human population will have their genomesterm·key sequenced. The leading driver of this trend is the promise of genomicterm·dom medicine to revolutionize the diagnosis and treatment of disease, with some countries contemplating sequencingterm·mth large portions of their populations: both England [ 20 ] and Saudi Arabia [ 21 ] have announced plans to sequence 100,000 of their citizens, one-third of Iceland’s 320,000 citizens have donated blood for genetic testing [ 22 ], and researchers in both the US [ 23 ] and China [ 17 ] both aim to sequence 1 million genomesterm·key in the next few years. With the world’s population projected to top 8 billion by 2025date, it is possible that as many as 25% of the population in developed nations and half of that in less-developed nations will have their genomesterm·key sequenced (comparable to the current worldwide distributionterm of Internet users [ 24 ]).
We therefore estimate between 100 million and as many as 2 billion human genomesterm·key could be sequenced by 2025date, representing four to five orders of magnitude growth in ten years and far exceeding the growth for the three other Big Dataterm·dom domainsterm·var. Indeed, this number could grow even larger, especially since new single-cell genometerm·key sequencingterm·mth technologies are starting to reveal previously unimagined levels of variation, especially in cancers, necessitating sequencingterm·mth the genomesterm·key of thousands of separate cells in a single tumor [ 10 ].
Moreover, the technology used to sequence DNA is deployed creatively for other applications (e.g., transcriptome, epigenome, proteome, metabolome, and microbiome sequencingterm·mth) necessitating generating new sequencingterm·mth data multiple times per person to monitor molecular activity [ 25 ]. These applications require precise quantitative counts of sequencingterm·mth reads to capture diversity of expression or diversity of abundances, thus requiring millions of reads to accurately estimate underlying distributionsterm as they change over time. For medicine, just having the genometerm·key will not be sufficient: for each individual, it will need to be coupled with other relevant ‘omics data sets, some collected periodically and from different tissues, to compare healthy and diseased states [ 26 ]. Computational challengesterm·key will increase because of dramatic increases in the total volume of genomicterm·dom data per person, as will the complexities of integrating these diverse data sources to improve health and cure diseases. Genomicsterm·dom thus appears to pose the greatest challenges for data acquisitionterm·key of the four Big Dataterm·dom domainsterm·var.
Data Storageterm Data storage requirementsterm for all four domainsterm·var are projected to be enormous. Today, the largest astronomyterm·dom data center devotes ~100 petabytes to storageterm, and the completion of the Square Kilometre Array (SKA) project is expected to lead to a storageterm demand of 1 exabyte per year. YouTube currently requires from 100 petabytes to 1 exabyte for storageterm and may be projected to require between 1 and 2 exabytes additional storageterm per year by 2025date. Twitter’s storageterm needs today are estimated at 0.5 petabytes per year, which may increase to 1.5 petabytes in the next ten years. (Our estimates here ignore the “replication factor” that multiplies storageterm needs by ~4, for redundancy.) For genomicsterm·dom, we have determined more than 100 petabytes of storageterm are currently used by only 20 of the largest institutions ( S1 Table ).
Projections of storage requirementsterm for sequence data depend on the accuracy and application of the sequencingterm·mth. For every 3 billion bases of human genometerm·key sequence, 30-fold more data (~100 gigabases) must be collected because of errors in sequencingterm·mth, base calling, and genometerm·key alignment. This means that as much as 2–40 exabytes of storageterm capacity will be needed by 2025date just for the human genomesterm·key ( S3 Note ). These needs can be diminished with effective data compressionterm [ 27 ], but decompression times and fidelity are a major concern in compressive genomicsterm [ 28 ].
Are the emerging “third-generation” single-molecule sequencing technologiesterm with much longer reads, such as those from Pacific Biosciences and Oxford Nanopore, a computational panacea? Though error rates currently are higher and throughput lower than short-read technologies, as they mature, these technologies are starting to be used to sequence and assemble nearly entire chromosomes [ 29 ]. This will minimize the need to oversample as much, and eventually, the raw sequence dataterm may not need to be stored at all. However, eliminating the need to store raw sequence dataterm and only retaining complete genomesterm·key will have relatively little impact overall—perhaps one or two orders of magnitude less data storageterm. More significant reductions in storageterm demand will come when improvements in sequencingterm·mth accuracy and database comprehensiveness reach the point at which genometerm·key sequences themselves do not need to be stored, just the list of variants relative to a reference collectionterm (“delta encodingterm”) [ 30 ]. This works well for cataloging the simplest variants in a human genometerm·key, but it may not be as useful for complex samples, such as cancer genomesterm·key, that have many novel rearrangements and mutations. While certainly helpful, we thus do not expect long-read sequencingterm·mth technology or delta encodingterm to solve the storageterm challenges for genometerm·key sequencingterm·mth in 2025date.
In contrast, we do see great opportunities for data reductionterm and real-time analysisterm of other ‘omics analysisterm. For example, once sequencingterm·mth becomes fast enough and the methods mature enough to correctly infer transcript expression levels in real time, we anticipate that raw RNA-seq reads will no longer be stored, except for specific research purposes. Already several such “streaming” algorithmsterm have been published for this purpose, performing as well as or superior to their nonstreaming counterpartsterm [ 31 ]. For RNA-seq and other ‘omics applications, genomicsterm·dom will benefit greatly from the lessons learned in particle physicsterm·dom, in which in most cases raw data are discarded almost as fast as they are generated in favor of higher level and greatly compressed summaries.
Altogether, we anticipate the development of huge genomicsterm·dom archives used for storing millions of genomesterm·key along with the associated ‘omics measurements over time. Ideally, these archives will also collect or be linked to the patient phenotypic data, especially disease outcomes and treatments provided to support retrospective analysisterm as new relationships are discovered. To make it practical to search and query through such vast collections, the data will be stored in hierarchical systemsterm that make data and their statistical summaries available at different levels of compression and latency, as used in astronomyterm·dom [ 32 ] and text analysisterm [ 33 ]. Thus, although total genomicterm·dom data could far exceed the demands for the others, with the right new innovations the net requirements could be similar to the domainsterm·var of astronomyterm·dom and YouTube.
Data Distributionterm Astronomyterm·dom, YouTube, Twitter, and genomicsterm·dom also differ greatly in data distributionterm patterns. The major bandwidth requirement of the SKA project is to get data from its 3,000 antennae to a central server, requiring as much as 600 terabytes/second [ 34 ]. The bandwidth usage of YouTube is relatively small for a single download and well supported by the average consumer’s 10 Mbps connection, but aggregate needs worldwide are enormous, with estimates up to 240 petabytes/day ( S4 Note ) [ 35 ]. The distributionterm patterns of genomicsterm·dom data are much more heterogeneous, involving elements of both situations [ 36 ].
Genomicterm·dom data are distributed in units spanning a wide range of sizes, from comparisons of a few bases or gene sequences to large multiterabyte bulk downloads from central repositories. For large-scale analysisterm, cloud computingterm is particularly suited to decreasing the bandwidth for distribution of genomic dataterm [ 37 ] so that applications can run on remote machines that already have dataterm [ 38 ]. Only small segments of code are uploaded and highly processed outputs are downloaded, thus significantly reducing the computing resources necessary for distributionterm.
But in addition to tailoring genomicsterm·dom applications for the cloud, new methods of data reliabilityterm and securityterm are required to ensure privacyterm, much more so than for the other three domainsterm·var. A serious breach of medically sensitive genomicterm·dom data would have permanent consequences and could seriously hinder the development of genomicterm·dom medicine. Homomorphic encryption systemsterm, in which encrypted data can be analyzed and manipulated for certain controlled queries without disclosing the raw data, are currently too computationally expensive for widespread use, but these and related cryptographic techniques are promising areas of research [ 39 ].
Data Analysisterm Astronomyterm·dom, YouTube, Twitter, and genomicsterm·dom differ most in computational requirements for data analysisterm. Astronomyterm·dom data require extensive specialized analysisterm, but the bulk of this requirement is for in situ processing and reduction of data by computers located near the telescopes [ 40 ]. This initial analysisterm is daunting because of its real-time nature and huge data volumes but can often be effectively performed in parallel on thousands of cores. YouTube videos are primarily meant to be viewed, along with some automated analysisterm for advertisements or copyright infringements. Twitter data are the subject of intense research in the social sciences [ 41 ], especially for topic and sentiment mining, which is performed chiefly on textual “tweets” in the context of associated metadata (e.g., user demographics and temporal information).
Analysisterm of genomicterm·dom data involves a more diverse range of approaches because of the variety of steps involved in reading a genometerm·key sequence and deriving useful information from it. For population and medical genomicsterm·dom, identifying the genomicterm·dom variants in each individual genometerm·key is currently one of the most computationally complex phases. Variant callingterm on 2 billion genomesterm·key per year, with 100,000 CPUs in parallel, would require methods that process 2 genomesterm·key per CPU-hour, three-to-four orders of magnitude faster than current capabilities [ 42 ]. Whole genome alignmentterm is another important form of genomicterm·dom data analysisterm, used for a variety of goals, from phylogeny reconstruction to genometerm·key annotation via comparative methodologies. Just a single whole genome alignmentterm between human and mouse consumes ~100 CPU hours [ 43 ]. Aligning all pairs of the ~2.5 million species expected to be available by 2025date amounts to 50–100 trillion such whole genome alignmentsterm, which would need to be six orders of magnitude faster than possible today.
Improvements to CPU capabilitiesterm, as anticipated by Moore’s Lawterm·dom, should help close the gap, but trends in computing power are often geared towards floating point operations and do not necessarily provide improvements in genometerm·key analysisterm, in which string operations and memory management often pose the most significant challenges. Moreover, the bigger bottleneck of Big Dataterm·dom analysisterm in the future may not be in CPU capabilitiesterm but in the input/output (I/O) hardwareterm that shuttles data between storageterm and processors [ 44 ], a problem requiring research into new parallel I/O hardware and algorithmsterm that can effectively utilize them.
The Long Road Ahead Genomicsterm·dom clearly poses some of the most severe computational challengesterm·key facing us in the next decade. Genomicsterm·dom is a “four-headed beast”; considering the computational demands across the lifecycle of a dataset—acquisitionterm·mth, storageterm, distributionterm, and analysisterm—genomicsterm·dom is either on par with or the most demanding of the Big Dataterm·dom domainsterm·var. New integrative approachesterm need to be developed that take into account the challenges in all four aspects: it is unlikely that a single advance or technologyterm will solve the genomicsterm·dom data problem. Several key technologies that are most critically needed to support future solutions are discussed in Box 1 .
In human health, the major needs are driven by the realization that for precision medicine and similar efforts to be most effective, genomesterm·key and related ‘omics data need to be shared and compared in huge numbers. If we do not commit as a scientific community to sharing now, we run the risk of establishing thousands of isolated, private data collectionsterm, each too underpowered to allow subtle signals to be extracted. More than anything else, connecting these resources requires trust among institutions, scientists, and the publicterm to ensure the collections will be used for medical purposes and not to discriminate or penalize individuals because of their genetic makeup.
Finally, the exascale data and computing centers that are emerging today to meet Big Dataterm·dom challenges in several domainsterm·var (YouTube [ 50 ], Google [ 51 ], Facebook [ 51 ], and the National Securityterm Agency [ 52 ]), are the result of far-sighted planning and commitment by the respective organizations. Now is the time for concerted, community-wide planning for the “genomical” challenges of the next decade.
Box 1. Key Technological Needs for Big Dataterm·dom Genomicsterm·dom (1) Acquisitionterm·mth The most important need to sustain the explosive growth in genomicterm·dom data acquisitionterm·key is continued advances in sequencingterm·mth technologies to reduce costs, improve throughput, and achieve very high accuracy. The current costs of ~US$1,000 per human genometerm·key begin to make it practical to sequence human genomesterm·key in large numbers, especially for critical medical treatments, but to scale to populations of hundreds of millions to billions of genomesterm·key, costs must be reduced by at least another one to two orders of magnitude or more. For many medical applications, the time for sequencingterm·mth must also be reduced so that it can be completed in near real time, especially to rapidly diagnosis acute infections and conditions. Finally, to make a genometerm·key sequence most useful, it must be paired with automated methods to collect metadata and phenotype data, all according to appropriate standards so that data collected in one environment can be compared to those collected in another.
(2) Storageterm The community needs to start designing and constructing data centers with fast, tiered storageterm systems to query and aggregate over large collections of genomesterm·key and ‘omics data. There are new technologies on the horizon that will help support these needs, including 3-D memory, integrated computing technologies that overcome the I/O bottleneck, and networks that are two-to-five orders of magnitude faster because of optical switching [ 45 , 46 ]. Similarly, efficient compression and indexing systems are critical to make the best use out of each available byte while making the data highly accessible. We also expect algorithmic developments that can represent large collections of personal genomesterm·key as a compact graph, making it more efficient and robust to compare one genometerm·key to many others. Beyond these approaches, we see the rise of streaming approaches to make on-the-fly comparisons that will allow us to rapidly discard data, especially for sequencingterm·mth applications that use the sequence data as a means to infer abundances or other molecular activity.
(3) Distributionterm The most practical, and perhaps only, solution for distributing genometerm·key sequences at a population scale is to use cloud-computing systems that minimize data movement and maximize code federation [ 47 ]. New developments from companies such as Google, Amazon, and Facebook that include applications built to fit the frameworks of distributed computing efficient data centers and distributed storageterm and cloud computingterm paradigms will be part of the solution. Already, large cloud-based genomicterm·dom resources are being developed using these technologies, especially to support the needs of the largest sequencingterm·mth centers or to support the needs of large communities (BGI-cloud, TCGA, the International Cancer Genometerm·key Consortium [ICGC], etc.). To make these online systems most useful, the community needs to develop application programming interfaces (APIs)term for discovering and querying large datasets on remote systems. The Global Alliance for Genomicsterm·dom and Health [ 48 ] and others are beginning to develop such standards for human genomicterm·dom data, and we expect other communities to follow. Finally, authentication, encryption, and other securityterm safeguards must be developed to ensure that genomicterm·dom data remain private.
(4) Analysisterm Our ultimate goal is to be able to interpret genomicterm·dom sequences and answer how DNA mutations, expression changes, or other molecular measurements relate to disease, development, behavior, or evolution. Accomplishing this goal will clearly require integration of biological domainterm·var expertise, large-scale machine learning systemsterm, and a computing infrastructure that can support flexible and dynamic queries to search for patterns over very large collections in very high dimensions. A number of “data science technologies,” including R, Mahout, and other machine learning systems powered by Hadoop and other highly scalable systems, are a start, but the current offerings are still difficult and expensive to use. The community would also benefit from libraries of highly optimized algorithms within a simple interface that can be combined and reused in many contexts as the problems emerge. Data science companies as well as open-source initiatives are already starting to develop such components, such as Amazon’s recent “Amazon Machine Learning” prediction system. But because genomicsterm·dom poses unique challenges in terms of data acquisitionterm·key, distributionterm, storageterm, and especially analysisterm, waiting for innovations from outside our field is unlikely to be sufficient. We must face these challenges ourselves, starting with integrating data science into graduate, undergraduate, and high-school curriculaterm to train the next generations of quantitative biologists, bioinformaticians, and computer scientists and engineers [ 49 ].
Supporting Information S1 Fig. Growth of GenBank. The y -axis shows the total sequence in bp. (Blue = GenBank, red = whole genometerm·key shotgun [WGSterm·def] sequences.) Each line is double of the previous. The x -axis indicates time. Each line is 6 months after the previous. Source: http://www.ncbi.nlm.nih.gov/genbank/statistics .
(TIF) Click here for additional data file. (1.9MB, tif) S1 Note. YouTube data estimates. (DOCX) Click here for additional data file. (65.6KB, docx) S2 Note. Twitter data estimates. (DOCX) Click here for additional data file. (84.9KB, docx) S3 Note. Human genomicterm·dom data storageterm estimates for 2025date. (DOCX) Click here for additional data file. (39.9KB, docx) S4 Note. YouTube distributionterm statistics (current). (DOCX) Click here for additional data file. (37.8KB, docx) S1 Table. Capacities of 20 major genomicsterm·dom institutions. The number of sequencers as listed from OmicsMaps.com and their storageterm capacities from the listed citation. These 20 institutions alone collectively have more than 100 PB of storageterm available.
(DOCX) Click here for additional data file. (171.6KB, docx) Acknowledgments We thank V. Jongeneel, M. Baker, and the anonymous reviewers for comments that improved the manuscript and the Illinois CompGen Initiative for providing the collaborative structure that fostered the idea for this analysisterm. We would also like to thank all of the participants of the 2014 Keystone Symposium on “Big Dataterm·dom in Biology” (organized by Lincoln D. Stein, Doreen Ware, and MCS) for their inspiration for this analysisterm.
Abbreviations API application programming interface ASKAP Australian Square Kilometre Array Pathfinder CPU central processing unit ExAC Exome Aggregation Consortium ICGC International Cancer Genometerm·key Consortium I/O input/output NIH/NCBI National Institutes of Health National Center for Biotechnology Information SKA Square Kilometre Array SRA Sequence Read Archive TCGA The Cancer Genometerm·key Atlas WGSterm·def whole genometerm·key shotgun Funding Statement This research was supported, in part, by grant 1U54GM114838 awarded by NIGMS to SS through funds provided by the trans-NIH Big Dataterm·dom to Knowledge (BD2K) initiative and by National Institutes of Health award (R01-HG006677) to MCS. ZDS and RKI were supported by NSF grant MRI13-37732 (S.S. Lumetta, PI). The funders had no role in study design, data collection and analysisterm, decision to publish, or preparation of the manuscript.