standOff. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote shown on the same row, so the borrowing can be checked against the thing borrowed from without leaving this page. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.| relation to other work | claim_relations | how this paper stands to another: agrees, contradicts, extends, qualifies |
| claim type | claim_type | the shape of the claim: existence, mechanism, method, critique, prediction, null |
| concept edge | concept_edges | an asserted relation between two concepts, with the sentence asserting it |
| concept | concepts | an idea the paper works with, quoted where the paper uses it |
| content tag | content_tags | a finding, definition, quantity, statistic or variable named in the paper |
| contribution | contribution | what the paper claims to add |
| control | controls | an experiment run to eliminate a rival explanation |
| limitation | limitations | what the paper says its evidence cannot support |
| mechanism | mechanisms | how the reported effect is said to come about |
| method | method | how the paper produced its evidence |
| object of study | object_of_study | what the paper is about — the thing it investigates |
| phenomenon | phenomenon | the effect or regularity under study |
| research question | research_question | the question the paper sets itself |
| sense | senses | a distinct use of a term, quoted where the paper fixes it |
| unrecordable relation | unhoused_relations | a relation the paper asserts that this vocabulary cannot express, with the reason |
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
VSEARCH: a versatile open source tool for metagenomics
Abstract
object_of_studyclaim_typelimitationsconcepts↗concepts↗concepts↗concepts↗concepts↗sensesclaim_relationsconcept_edgesconcept_edgescontent_tagscontent_tagsVSEARCH is an open source and free of charge multithreaded 64-bit tool for processing and preparing metagenomics, genomics and population genomics nucleotide sequence data. It is designed as an alternative to the widely used USEARCH tool ( Edgar, 2010 ) for which the source code is not publicly available, algorithm details are only rudimentarily described, and only a memory-confined 32-bit version is freely available for academic use.
mechanisms↗concepts↗concepts↗concepts↗concepts↗concepts↗concepts↗claim_relationsconcept_edgescontent_tagscontent_tagsWhen searching nucleotide sequences, VSEARCH uses a fast heuristic based on words shared by the query and target sequences in order to quickly identify similar sequences, a similar strategy is probably used in USEARCH. VSEARCH then performs optimal global sequence alignment of the query against potential target sequences, using full dynamic programming instead of the seed-and-extend heuristic used by USEARCH. Pairwise alignments are computed in parallel using vectorisation and multiple threads.
contributioncontent_tagscontent_tagsVSEARCH includes most commands for analysing nucleotide sequences available in USEARCH version 7 and several of those available in USEARCH version 8, including searching (exact or based on global alignment), clustering by similarity (using length pre-sorting, abundance pre-sorting or a user-defined order), chimera detection (reference-based or de novo ), dereplication (full length or prefix), pairwise alignment, reverse complementation, sorting, and subsampling. VSEARCH also includes commands for FASTQ file processing, i.e., format detection, filtering, read quality statistics, and merging of paired reads. Furthermore, VSEARCH extends functionality with several new commands and improvements, including shuffling, rereplication, masking of low-complexity sequences with the well-known DUST algorithm, a choice among different similarity definitions, and FASTQ file format conversion. VSEARCH is here shown to be more accurate than USEARCH when performing searching, clustering, chimera detection and subsampling, while on a par with USEARCH for paired-ends read merging. VSEARCH is slower than USEARCH when performing clustering and chimera detection, but significantly faster when performing paired-end reads merging and dereplication. VSEARCH is available at https://github.com/torognes/vsearch under either the BSD 2-clause license or the GNU General Public License version 3.0.
VSEARCH has been shown to be a fast, accurate and full-fledged alternative to USEARCH. A free and open-source versatile tool for sequence analysis is now available to the metagenomics community.
Introduction
Rockström et al. (2009) and Steffen et al. (2015) presented biodiversity loss as a major threat for the short-term survival of humanity. Recent progress in sequencing technologies have made possible large scale studies of environmental genetic diversity, from deep sea hydrothermal vents to Antarctic lakes ( Karsenti et al., 2011 ), and from tropical forests to Siberian steppes ( Gilbert, Jansson & Knight, 2014 ). Recent clinical studies have shown the importance of the microbiomes of our bodies and daily environments for human health ( Human Microbiome Project Consortium, 2012 ). Usually focusing on universal markers (e.g., 16S rRNA, ITS, COI), these targeted metagenomics studies produce many millions of sequences, and require open-source, fast and memory efficient tools to facilitate their ecological interpretation.
Several pipelines have been developed for microbiome analysis, among which mothur ( Schloss et al., 2009 ), QIIME ( Caporaso et al., 2010 ), and UPARSE ( Edgar, 2013 ) are the most popular. QIIME and UPARSE are both based on USEARCH ( Edgar, 2010 ), a set of tools designed and implemented by Robert C. Edgar, and available at http://drive5.com/usearch/ . USEARCH offers a great number of commands and options to manipulate and analyse FASTQ and FASTA files. However, the source code of USEARCH is not publicly available, algorithm details are only rudimentarily described, and only a memory-confined 32-bit version is freely available for academic use.
research_questionWe believe that the existence of open-source solutions is beneficial for end-users and can invigorate research activities. For this reason, we have undertaken to offer a high quality open-source alternative to USEARCH, freely available to users without any memory limitation. VSEARCH includes most of the USEARCH functions in common use, and further development may add additional features. Here we describe the details of the VSEARCH implementation. To assess its performance in terms of speed and quality of results, we have evaluated some of the most important functions (searching, clustering, chimera detection and subsampling) and compared them to USEARCH. We find that VSEARCH delivers results that are better or on a par with USEARCH results.
Materials and Methods
Algorithms and implementation
Below is a brief description of the most important functions of VSEARCH and details of their implementation. VSEARCH command line options are shown in italics, and should be preceded by a single (-) or double dash (-TI:\,-) when used.
Reading FASTA and FASTQ files
content_tagsMost VSEARCH commands read files in FASTA or FASTQ format. The parser for FASTQ files in VSEARCH is compliant with the standard as described by Cock et al. (2010) and correctly parses all their tests files. FASTA and FASTQ files are automatically detected and many commands accept both as input. Files compressed with gzip or bzip2 are automatically detected and decompressed using the zlib library by Gailly & Adler (2016) or the bzip2 library by Seward (2016) , respectively. Data may also be piped into or out of VSEARCH, allowing for instance many separate FASTA files to be piped into VSEARCH for simultaneous dereplication, or allowing the creation of complex pipelines without ever having to write on slow disks.
concepts↗unhoused_relationscontent_tagscontent_tagsVSEARCH is a 64-bit program and allows very large datasets to be processed, essentially limited only by the amount of memory available. The free USEARCH versions are 32-bit programs that limit the available memory to somewhere less than 4GB, often seriously hampering the analysis of realistic datasets.
Writing result files
VSEARCH can output results in a variety of formats (FASTA, FASTQ, tables, alignments, SAM) depending on the input format and command used. When outputting FASTA files, the line width may be specified using the fasta_width option, where 0 means that line wrapping should be turned off. Similar controls are offered for pairwise or multiple sequence alignments.
Searching
method↗mechanisms↗concepts↗concepts↗sensesconcept_edgescontent_tags↗Global pairwise sequence comparison is a core functionality of VSEARCH. Several commands compare a query sequence against a database of sequences: all-vs-all alignment ( allpairs_global ), clustering ( cluster_fast , cluster_size , cluster_smallmem ), chimera detection ( uchime_denovo and uchime_ref ) and searching ( usearch_global ). This comparison function proceeds in two phases: an initial heuristic filtering based on shared words, followed by optimal alignment of the query with the most promising candidates.
mechanisms↗limitationsconcepts↗concepts↗concepts↗unhoused_relationssensescontent_tagscontent_tagscontent_tags↗content_tags↗The first phase is presumably quite similar to USEARCH ( Edgar, 2010 ). Heuristics are used to identify a small set of database sequences that have many words in common with the query sequence. Words (or k -mers) consist of a certain number k of consecutive nucleotides of a sequence (8 by default, adjustable with the wordlength option). All overlapping words are included. A sequence of length n then contains at most n − k + 1 unique words. VSEARCH counts the number of shared words between the query and each database sequence. Words that appear multiple times are counted only once. To count the words in the database sequences quickly, VSEARCH creates an index of all the 4 k possible distinct words and stores information about which database sequences they appear in. For extremely frequent words, the set of database sequences is represented by a bitmap; otherwise the set is stored as a list. A finer control of k -mer indexing is described for USEARCH by the pattern (binary string indicating which positions must match) and slots options. USEARCH has such options but seems to ignore them. Currently, VSEARCH ignores these two options too. The minimum number of shared words required may be specified with the minwordmatches option (10 by default), but a lower value is automatically used for short or simple query sequences with less than 10 unique words.
Comparing sequences based on statistics of shared words is a common method to quickly assess the similarity between two sequences without aligning them, which is often time-consuming. The D 2 statistic and related metrics for alignment-free sequence comparison have often been used for rapid and approximate sequence matching and their statistical properties have been well studied ( Song et al., 2014 ). The approach used here has similarities to the D 2 statistic, but multiple matches of the same word are ignored.
mechanisms↗mechanisms↗limitationslimitationsconcepts↗concepts↗unhoused_relationsunhoused_relationsunhoused_relationssensesclaim_relationscontent_tagscontent_tagscontent_tagscontent_tags↗In the second phase, searching proceeds by considering the database sequences in a specific order, starting with the sequence having the largest number of words in common with the query, and proceeding with a decreasing number of shared words. If two database sequences have the same number of words in common with the query, the shortest sequence is considered first. The query sequence is compared with each database sequence by computing the optimal global alignment. The alignment is performed using a multi-threaded and vectorised full dynamic programming algorithm ( Needleman & Wunsch, 1970 ) adapted from SWIPE ( Rognes, 2011 ). Due to the extreme memory requirements of this method when aligning two long sequences, an alternative algorithm described by Hirschberg (1975) and Myers & Miller (1988) is used when the product of the length of the sequences is greater than 25,000,000, corresponding to aligning two 5,000 bp sequences. This alternative algorithm uses only a linear amount of memory but is considerably slower. This second phase is probably where USEARCH and VSEARCH differ the most, as USEARCH by default presumably performs heuristic seed-and-extend alignment similar to BLAST ( Altschul et al., 1997 ), and only performs optimal alignment when the option fulldp (full dynamic programming) is used. Computing the optimal pairwise alignment in each case gives more accurate results but is also computationally more demanding. The efficient and vectorised full dynamic programming implementation in VSEARCH compensates that extra cost, at least for sequences that are not too long.
concepts↗concepts↗unhoused_relationscontent_tagscontent_tagscontent_tags↗If the resulting alignment indicates a similarity equal to or greater than the value specified with the id option, the database sequence is accepted. If the similarity is too low, it is rejected. Several other options may also be used to determine how similarity is computed ( iddef , as USEARCH used to offer up to version 6), and which sequences should be accepted and rejected, either before (e.g., self , minqsize ) or after alignment (e.g., maxgaps , maxsubs ). The search is terminated when either a certain number of sequences have been accepted (1 by default, adjustable with the maxaccepts option), or a certain number of sequences have been rejected (32 by default, adjustable with the maxrejects option). The accepted sequences are sorted by sequence similarity and presented as the search results.
VSEARCH also includes a search_exact command that only identifies exact matches to the query. It uses a hash table in a way similar to the full-length dereplication command described below.
Clustering
mechanisms↗concepts↗concepts↗concepts↗concepts↗unhoused_relationssensescontent_tagscontent_tags↗content_tags↗content_tagsVSEARCH includes commands to perform de novo clustering using a greedy and heuristic centroid-based algorithm with an adjustable sequence similarity threshold specified with the id option (e.g., 0.97). The input sequences are either processed in the user supplied order ( cluster_smallmem ) or pre-sorted based on length ( cluster_fast ) or abundance (the new cluster_size option). Each input sequence is then used as a query in a search against an initially empty database of centroid sequences. The query sequence is clustered with the first centroid sequence found with similarity equal to or above the threshold. The search is performed using the heuristic approach described above which generally finds the most similar sequences first. If no matches are found, the query sequence becomes the centroid of a new cluster and is added to the database. If maxaccepts is higher than 1, several centroids with sufficient sequence similarity may be found and considered. By default, the query is clustered with the centroid presenting the highest sequence similarity (distance-based greedy clustering, DGC), or, if the sizeorder option is turned on, the centroid with the highest abundance (abundance-based greedy clustering, AGC) ( He et al., 2015 ; Westcott & Schloss, 2015 ; Schloss, 2016 ). VSEARCH performs multi-threaded clustering by searching the database of centroid sequences with several query sequences in parallel. If there are any non-matching query sequences giving rise to new centroids, the required internal comparisons between the query sequences are subsequently performed to achieve correct results. For each cluster, VSEARCH can create a simple multiple sequence alignment using the center star method ( Gusfield, 1993 ) with the centroid as the center sequence, and then compute a consensus sequence and a sequence profile.
Dereplication and rereplication
mechanisms↗concepts↗concepts↗content_tags↗Full-length dereplication ( derep_fulllength ) is performed using a hash table with an open addressing and linear probing strategy based on the Google CityHash hash functions (written by Geoff Pike and Jyrki Alakuijala, and available at https://github.com/google/cityhash ). The hash table is initially empty. For each input sequence, the hash is computed and a lookup in the hash table is performed. If an identical sequence is found, the input sequence is clustered with the matching sequence; otherwise the input sequence is inserted into the hash table.
concepts↗sensescontent_tagscontent_tags↗Prefix dereplication ( derep_prefix ) is also implemented. As with full-length dereplication, identical sequences are clustered. In addition, sequences that are identical to prefixes of other sequences will also be clustered together. If a sequence is identical to the prefix of multiple sequences, it is generally not defined how prefix clustering should behave. VSEARCH resolves this ambiguity by clustering the sequence with the shortest of the candidate sequences. If they are equally long, priority will be given to the most abundant, the one with the lexicographically smaller identifier or the one with the earliest original position, in that order.
To perform prefix dereplication, VSEARCH first creates an initially empty hash table. It then sorts the input sequences by length and identifies the length s of the shortest sequence in the dataset. Each input sequence is then processed as follows, starting with the shortest: If an exact match to the full input sequence is found in the hash table, the input sequence is clustered with the matching hash table sequence. If no match to the full input sequence is found, the prefixes of the input sequence are considered, starting with the longest prefix and proceeding with shorter prefixes in order, down to prefixes of length s . If a match is now found in the hash table, the sequences are clustered, the matching sequence is deleted from the hash table and the full input sequence is inserted into the hash table instead. If no match is found for any prefix, the full sequence is inserted into the hash table. In the end, the remaining sequences in the hash table will be output with accumulated abundances for all sequences in each cluster.
mechanisms↗concepts↗In order to identify matches in the hash table during prefix dereplication, a hash is computed for each full-length input sequence and all its prefixes. The hash function used is the 64-bit Fowler–Noll–Vo 1a hash function ( Fowler, Noll & Vo, 1991 ), which is simple and quick to compute for such a series of sequences by adding one nucleotide at a time.
mechanisms↗concepts↗concepts↗The sequences resulting from dereplication and many other commands may be relabeled with a given prefix followed by a sequentially increasing number. VSEARCH exclusively also offers the possibility of relabelling each sequence with the SHA-1 ( Eastlake & Jones, 2001 ) or MD5 ( Rivest, 1992 ) message digest (hash) of the sequence. These are strings that are highly likely to be unique for each sequence. Before the digest is computed, the sequence is normalized by converting U’s to T’s and converting all symbols to upper case. VSEARCH includes public domain code for the MD5 algorithm written by Alexander Peslyak, and for SHA1 by Steve Reid and others.
limitationsconcepts↗VSEARCH also includes a new command ( rereplicate ) to perform rereplication that can be used to recreate datasets as they were before full-length dereplication, but of course original labels cannot be recreated.
Chimera detection
mechanisms↗concepts↗concepts↗claim_relationscontent_tagsChimeras are detected either de novo ( uchime_denovo command) or with a reference database ( uchime_ref command) using the UCHIME algorithm described by Edgar et al. (2011) . VSEARCH will divide each query sequence into four segments and look for similarity of each segment to sequences in the set of potential parents using the heuristic search function described earlier. It will consider the four best candidates for each segment using maxaccepts 4 and maxrejects 16, and an id threshold of 0.55. VSEARCH optionally outputs borderline sequences, that is, sequences having a high enough score (as specified with the minh option) but with too small a divergence from the closest parent (as specified with the mindiv option). Multi-threading is supported for reference-based chimera detection.
Low-complexity sequence masking
mechanisms↗concepts↗concepts↗concepts↗unhoused_relationscontent_tagsVSEARCH includes a highly optimized and parallelized implementation of the Dust algorithm by Tatusov and Lipman for masking of simple repeats and low-complexity nucleotide sequences. It is considerably faster than the implementation of the same algorithm in USEARCH. Their code available at ftp://ftp.ncbi.nlm.nih.gov/pub/tatusov/dust/version1/src/ is in the public domain. VSEARCH uses this algorithm by default, while USEARCH by default uses an undocumented rapid masking algorithm called fastnucleo . VSEARCH performs soft-masking automatically for the pairwise alignment, search, clustering and chimera detection commands. This behaviour can be controlled with the hardmask option to replace masked symbols with N’s instead of lower-casing them, and the dbmask and qmask options, which selects the masking algorithm (none, dust or soft) used for the database and query sequences, respectively. Masking may also be performed explicitly on an input file using the fastx_mask and maskfasta commands.
FASTQ file processing
concepts↗VSEARCH includes commands to detect the FASTQ file version and the range of quality scores used ( fastq_chars ), as well as two commands for computing sequence quality statistics ( fastq_stats and fastq_eestats ). It can also truncate and filter sequences in FASTQ files based on various criteria ( fastq_filter ). A new command is added to convert between different FASTQ file versions and quality encodings ( fastq_convert ), e.g., from the old Phred+64 encoded Illumina FASTQ files to the newer Phred+33 format.
Merging of paired-end reads
mechanisms↗mechanisms↗concepts↗concepts↗concepts↗Merging of paired-end reads is supported by VSEARCH using the fastq_mergepairs command. The method used has some similarity to PEAR ( Zhang et al., 2014 ) and recognises options similar to USEARCH. The algorithm computes the optimal ungapped alignment of the overlapping region of the forward sequence and the reverse-complemented reverse sequence. The alignment requires a minimum overlap length (specified with the fastq_minovlen option, default 10), a maximum number of mismatches ( fastq_maxdiffs option, default 5), and a minimum and maximum length of the merged sequence ( fastq_minmergelen option, default 1, and fastq_maxmergelen option, default infinite). Staggered read pairs, i.e., read pairs where the 3′ end of the reverse read has an overhang to the left of the 5′ end of the forward read, are not allowed by default, but may be turned on by the fastq_allowmergestagger option. VSEARCH uses a match score (alpha) of +4 and a mismatch score (beta) of −5 for perfect quality residues. These scores are weighted by the probability that these two residues really match or mismatch, respectively, taking quality scores into account. These probabilities are computed in a way similar to PEAR score method 2 described in ‘Algorithms and implementation’ of the PEAR paper ( Zhang et al., 2014 ), but VSEARCH assumes all nucleotide background frequencies are 0.25. When merging sequences, VSEARCH computes posterior quality scores for the overlapping regions as described by Edgar & Flyvbjerg (2015) . For speed, scores and probabilities are pre-computed for all possible quality scores.
Sorting and shuffling
mechanisms↗concepts↗VSEARCH can sort FASTA files by decreasing sequence length ( sortbylength ) or abundance ( sortbysize ). VSEARCH can also perform shuffling of FASTA files in random order ( shuffle ). A seed value for the pseudo random number generator may be provided by the randseed option to obtain replicable results.
Subsampling
mechanisms↗concepts↗Sequences in FASTA and FASTQ files can be subsampled ( fastx_subsample ) by randomly extracting a certain number ( sample_size ) or percentage ( sample_pct ) of the input sequences. Abundances may be taken into account, giving results as if the input sequences were rereplicated, subsampled and then dereplicated.
Results and Discussion
Supported commands and options
claim_relationsVSEARCH implements the following commands available in USEARCH version 7: allpairs_global , cluster_fast , cluster_smallmem , derep_fulllength , derep_prefix , fastq_chars , fastq_filter , fastq_mergepairs , fastq_stats , fastx_mask , maskfasta , sortbylength , sortbysize , uchime_denovo , uchime_ref and usearch_global . In addition, the following commands available in USEARCH version 8 have been implemented: fastq_eestats , fastx_revcomp , fastx_subsample and search_exact . VSEARCH additionally includes a few new commands that do not exist in USEARCH: cluster_size , fastq_convert , rereplicate and shuffle .
limitationsSome USEARCH version 7 commands have not yet been implemented in VSEARCH. We have not prioritized commands related to amino acid sequences ( findorfs ), local alignment ( allpairs_local , pairs_local , search_local , ublast ), brute-force search ( search_global , pairs_global ), UDB databases ( makeudb_ublast , makeudb_usearch , udb2fasta , udbinfo , udbstats ), and the UPARSE pipeline ( cluster_otus , uparse_ref ).
Almost all USEARCH 7 options are supported, except for those related to non-standard database indexing ( alpha , dbaccelpct , dbstep , pattern , slots ) as well as local alignments and alignment heuristics ( band , hspw , lext , lopen , matrix , minhsp , xdrop_g , xdrop_nw , xdrop_u ).
concepts↗content_tagsThe same command and option names as in USEARCH version 7 have generally been used in order to make VSEARCH an almost drop-in replacement. In fact, in QIIME many commands will run fine if an alias or link from usearch to vsearch is made. Detailed documentation of VSEARCH is available as a man page. We will consider adding further commands and options to VSEARCH in the future.
Performance assessment
phenomenonlimitationscontrolscontrolsThe performance of the most important functions of VSEARCH version 2.0.3 (64-bit) was evaluated and compared to the 32-bit versions of USEARCH version 7.0.1090 and 8.1.1861. All datasets used were small enough to fit comfortably in the memory allocated to a 32-bit process. Chimera detection was also compared to UCHIME version 4.2. All tests were run on GNU/Linux CentOS 6.7 compute nodes with 16 physical cores (Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60 GHz) and 64GB RAM. Programs were run with 8 threads, if possible. All times indicated are wall-clock times. All scripts and data necessary to perform the evaluations are available in the GitHub repository at https://github.com/torognes/vsearch-eval/ to enable independent replication.
Searching
controlsconcepts↗concepts↗content_tagscontent_tagscontent_tagscontent_tags↗Evaluation of search accuracy was carried out as described in the USEARCH paper ( Edgar, 2010 ), its supplementary, and on the website ( http://drive5.com/usearch/benchmark_rfam.html ), by assessing the ability of the programs to identify RNA sequences belonging to the same family in RFAM ( Burge et al., 2013 ). The 383,004 sequences in Rfam version 11 were randomly shuffled and then the first sequence from each of the 2,085 (out of 2,208) families that contained at least 2 members was selected as a representative and used as a query against the remaining 380,919 sequences. The programs were run with options id 0.0, minseqlength 1, maxaccepts 1, maxrejects 32, and strand plus. If the matching sequence found belonged to the same family, it was considered a true positive, otherwise it was considered as a false positive. We combined the results from 20 shufflings and plotted the results in the ROC-like curve shown in Fig. 1 . For a false discovery rate between 0.010 and 0.015, VSEARCH is more accurate than USEARCH’s latest version. For lower values, the three programs have similar accuracies. At higher false discovery rates, USEARCH version 8 has an advantage.
Search accuracy on the RFAM v11 dataset. USEARCH version 7 (blue), USEARCH version 8 (orange) and VSEARCH (black) was run using the usearch_global command on subsets of the RFAM dataset to identify members of the same families. The plot shows the true positive rate (also known as the recall or sensitivity) as a function of the false discovery rate at varying sequence similarity levels. This curve is based on data from 20 shufflings of the dataset.
controlscontent_tagsThe time to search the Rfam database as described above was measured. To avoid extremely short running times, 1,000 replicates of the datasets were used. USEARCH version 7 required on average 5 min 29 s for the search, USEARCH version 8 took 5 min 57 s, while VSEARCH took 5 min 26 s.
Clustering
Westcott & Schloss (2015) have already carried out an evaluation of the clustering performance of VSEARCH. They tested the ability of several tools to assign OTUs for 16S rRNA sequences and “demonstrated that for the greedy algorithms VSEARCH produced assignments that were comparable to those produced by USEARCH making VSEARCH a viable free and open source alternative to USEARCH”. Schloss (2016) also evaluated de novo clustering by VSEARCH.
limitationscontrolsconcepts↗concepts↗concepts↗concepts↗concepts↗sensessensessensesconcept_edgesconcept_edgescontent_tagscontent_tags↗content_tags↗content_tags↗content_tagscontent_tags↗content_tags↗content_tags↗We independently evaluated the clustering accuracy of USEARCH and VSEARCH as described for Swarm ( Mahé et al., 2014 ) using two mock datasets, one with an even and one with uneven composition of 57 archaea and bacteria. The datasets were first dereplicated. Then the taxonomy of the unique sequences was assigned by a search against the set of rRNA reference sequences representing the species in the mock datasets, carried out with the usearch_global command of USEARCH. The sequences were shuffled randomly 10 times and clustering was performed at 20 different similarity levels ranging from 80% to 99% in steps of 1%. Clustering was carried out in two ways, first using the cluster_fast command that pre-sorts the sequences by length, and then using the cluster_smallmem command that simply processes the sequences in the user-supplied order (in that case, we supplied sequences ordered by decreasing abundance using the sortbysize command). We then compared the clusters obtained to the assigned species and computed the recall, precision and the adjusted Rand index of the classifications. Recall measures to what extent amplicons assigned to the same species are grouped together in the same OTU (i.e., not over-splitting). Precision measures to what extent amplicons in an OTU are assigned to the same species (i.e., not over-grouping). The adjusted Rand index ( Rand, 1971 ; Hubert & Arabie, 1985 ) summarises both precision and recall using a measure of agreement between two clusterings while adjusting for expected values by chance. The average values over the all shufflings are presented in Figs. 2 and 3 for the even and uneven datasets, respectively. For abundance-sorted sequences, the difference between VSEARCH and USEARCH version 8 is negligible. The difference is larger for length-sorted sequences. When using length sorting, USEARCH 8 (as well as version 7 on the even dataset) shows better precision than VSEARCH for similarity levels below 93%. However, OTU delineation is usually performed at higher similarity values, typically 97%. In the case of our benchmark, overall accuracy as measured by the adjusted Rand index is maximised at 95–97% similarity. This is precisely the region where for length sorting at least VSEARCH outperforms USEARCH.
Clustering accuracy on the even dataset. USEARCH version 7 (blue) and 8 (orange) and VSEARCH (black) was run using abundance sorting ( cluster_smallmem ) (A, C, E) and length sorting ( cluster_fast ) (B, D, F) on the even dataset. The performance is indicated with the adjusted Rand index (A, B), recall (C, D) and precision (E, F) metrics.
Clustering accuracy on the uneven dataset. USEARCH version 7 (blue) and 8 (orange) and VSEARCH (black) was run using abundance sorting ( cluster_smallmem ) (A, C, E) and length sorting ( cluster_fast ) (B, D, F) on the uneven dataset. The performance is indicated with the adjusted Rand index (A, B), recall (C, D) and precision (E, F) metrics.
concepts↗concepts↗unhoused_relationscontent_tagscontent_tags↗The time used for clustering is shown in Fig. 4 . The time used depended on the dataset, algorithm and clustering threshold. The USEARCH programs were in general 2–3 times faster than VSEARCH. In general the difference in speed was smaller for higher thresholds, especially at 99% similarity. Clustering with USEARCH using the fulldp option is an order of magnitude slower than VSEARCH, but contrary to our expectations it does not seem to improve accuracy.
Clustering speed. Median wall time in seconds to cluster the even (A, B) and uneven (C, D) datasets using USEARCH version 7 (blue) and 8 (orange) and VSEARCH (black) using abundance sorting ( cluster_smallmem ) (A, C) and length sorting ( cluster_fast ) (B, D).
Dereplication
content_tagscontent_tagscontent_tagsMeasurements of dereplication speed were performed on the even and uneven datasets described earlier as well as on the BioMarKs dataset ( Logares et al., 2014 ). For full-length dereplication ( derep_fulllength ) VSEARCH was about 40–50% faster than USEARCH version 7 and 50–70% faster than version 8 on all three datasets. All programs were approximately equally fast on prefix dereplication ( derep_prefix ) of the even and uneven datasets. However, prefix dereplication of the BioMarKs dataset was extremely slow with USEARCH. USEARCH version 7 used more than 4 min and version 8 more than 27 min, while VSEARCH used less than 4 s. The prefix dereplication algorithm used in USEARCH appears ineffective when dealing with short sequences. Removing the 811 sequences shorter than 200 bp out of the 312,503 sequences of the BioMarKs dataset reduces the running time of USEARCH version 7 and 8 down to just 5 and 6 s, respectively.
Chimera detection
controlsWe evaluated the chimera detection accuracy of VSEARCH and USEARCH in two ways, first using a method similar to that performed for UCHIME, and then using a new chimera simulation procedure based on sequences from Greengenes ( DeSantis et al., 2006 ) and SILVA ( Quast et al., 2013 ) sequences.
Chimera detection performance with the SIMM dataset. UCHIME (UC), USEARCH version 7 (U7) and 8 (U8), and VSEARCH (V) was run using the uchime_ref algorithm on the SIMM dataset that was originally also used to evaluate the UCHIME algorithm. Divergence is the percentage of similarity to the original sequences. Noise is either zero (–) or the percentage of indels (i1–i5) or substitutions (m1-5) added. The number of chimeras detected out of 100 of each type is shown. The best results in each category are shaded.
concepts↗content_tagscontent_tagsFirst we repeated the evaluation of the uchime_ref command described in the UCHIME paper ( Edgar et al., 2011 ) using the SIMM dataset downloaded from http://drive5.com/uchime/uchime_download.html . The dataset consists of 900 simulated chimeras that are approximately 250 bp long. The chimeras were generated from 2, 3 or 4 segments selected randomly from 86 original sequences and have similarities in the ranges 90–95%, 95–97% and 97–99% to the original sequences. They were either used unmodified or with 1–5% indels or 1–5% substitutions. We assessed the performance of (i) the original open-source UCHIME version 4.2 program, (ii) USEARCH version 7, (iii) USEARCH version 8, and (iv) VSEARCH. The results are shown in Table 1 and indicate that VSEARCH is superior to the other tools in almost all cases, and in particular when indels were added. The original UCHIME program was found to be quite, but also considerably slower than all the other tools. USEARCH was better than VSEARCH in only 4 out of 99 cases.
limitationsconcepts↗concepts↗concepts↗concepts↗unhoused_relationssensesconcept_edgescontent_tagscontent_tags↗Next, we tested reference-based ( uchime_ref ) and de novo ( uchime_denovo ) chimera detection using sequences from the 2011 version of Greengenes downloaded from http://greengenes.lbl.gov/Download/Sequence_Data/Fasta_data_files/ and from version 106 (May 2011) of the SILVA database downloaded from https://www.arb-silva.de/no_cache/download/archive/release_106/Exports/ . Sequences from the 16S rRNA V4 region was computationally extracted using the 515F (5′-GTGNCAGCMGCCGCGGTAA-3′) and 806R (5′-GGACTACHVGGGTWTCTAAT-3′) primers, and 8,000 reads were randomly selected from each database. PCR was simulated using a new simulation algorithm known as Simera ( Nichols & Quince, 2016 ) (available at https://github.com/bnichols1979/Simera ) that includes amplification and creation of PCR artefacts like chimeras. We sampled 30,000 reads (-s 30,000) and generated 20,000 potential chimeras (-c 20,000). Defaults were used for other options to Simera. The output sequences were then fed into an Illumina MiSeq noise simulator ( Schirmer et al., 2015 ) ending up with 14,966 reads based on Greengenes and 14,952 reads based on SILVA, of which 1,262 and 1,640 reads contain chimeric sequences, respectively. Next, the sequences were either clustered using the cluster_fast command at 97% identity or dereplicated. VSEARCH and USEARCH version 7 and 8 were run using the uchime_denovo command and then using the uchime_ref command with the Gold database downloaded from http://drive5.com/uchime/uchime_download.html as the reference database. To assess the performance, the results were sorted based on the chimera score, and then the ability to classify individual sequences correctly into chimeric and non-chimeric was plotted as ROC curves. The curves reflect the accuracy of classifying individual reads, not clusters, as abundances were taken into account. The plots in Figs. 5 and 6 show that de novo chimera detection performs better than reference-based detection, with the SILVA dataset in particular, but it does of course depend on the reference database used. VSEARCH performs better than both versions of USEARCH for de novo chimera detection. For reference-based detection VSEARCH also performs better for the Greengenes dataset, while none of the programs work well with the SILVA dataset. Clustering at 97% appears to be more appropriate than dereplication. In this test, the USEARCH programs were about twice as fast as VSEARCH for de novo detection, while they were about 10–30% faster than VSEARCH for reference-based detection.
Chimera detection performance with the Greengenes dataset shown with ROC curves. USEARCH version 7 (blue) and 8 (orange) and VSEARCH (black) was run using the uchime_denovo (A, B) and the uchime_ref (C, D) commands on simulated Illumina data based on the Greengenes database that has either been clustered with a 97% identity threshold (using the cluster_fast command in VSEARCH) (A, C) or dereplicated (using the derep_fulllength command in VSEARCH) (B, D).
Chimera detection performance on the SILVA dataset shown with ROC curves. USEARCH version 7 (blue) and 8 (orange) and VSEARCH (black) was run using the uchime_denovo (A, B) and the uchime_ref (C, D) commands on simulated Illumina data based on the SILVA database that has either been clustered with a 97% identity threshold (using the cluster_fast command in USEARCH) (A, C) or dereplicated (using the derep_fulllength command in VSEARCH) (B, D).
Merging of paired-end reads
limitationscontrolsconcepts↗sensescontent_tagscontent_tags↗content_tags↗content_tags↗Evaluation of paired-end reads merging performance was carried out in a manner similar to that described for the evaluation of PEAR ( Zhang et al., 2014 ). We used whole genome sequencing data from Staphylococcus aureus subspecies aureus strain USA 300 TCH 1516 sequenced by MacCallum et al. (2009) and retrieved from the GAGE-B repository ( http://ccb.jhu.edu/gage_b/ ). The S.aureus reads were 101 bp long from on average 180 bp long fragments, giving a 45X coverage of the genome. We also used Methylococcus capsulatus strain Bath 16S rRNA V3 region amplicon reads sequenced by Masella et al. (2012) . These reads were 108 bp long and the pairs should have an overlap of exactly 18 bp. Merging options were set to allow a minimum overlap of 10 bp and a maximum of 5 mismatches (USEARCH 7 and 8 have different default values for those), while other options were left at defaults. All programs were run with 8 threads. Merged sequences that could be perfectly aligned to their respective reference sequences (either the entire genome or the specific rRNA region) using BWA MEM ( Li & Durbin, 2009 ) were considered correctly merged. The results are shown in Table 2 . The numbers indicate that USEARCH version 7 merges the most reads for both bacteria, but also has the lowest percentage of correctly merged pairs of those merged. USEARCH version 8 merges the fewest reads, but has the highest percentage of correctly merged reads of those merged. VSEARCH is in the middle by merging more reads than USEARCH 8 with only a small decrease in the percentage of correct merges. VSEARCH is about twice as fast as USEARCH 8 and 4–5 times faster than USEARCH version 7.
Paired-end reads merging performance. The number of sequence pairs, merged pairs, and correctly merged pairs are shown for each bacterium and program. The percentage of reads merged, as well as the percentage of correctly merged reads both of the merged reads and of all reads are also shown. Times are in seconds using 8 threads.
Subsampling performance. The observed distribution of the maximum amplicon abundance in 10,000 random subsamplings of 5% of the TARA V9 dataset using VSEARCH (A, black) and USEARCH version 8 (B, orange) is shown. The expected mean abundance is 782,133.5 (blue dashed line).
Subsampling
controlsconcepts↗concepts↗concepts↗unhoused_relationsunhoused_relationsunhoused_relationssensescontent_tagscontent_tagscontent_tagsWe evaluated the subsampling commands of USEARCH version 8 and VSEARCH to check if the results obtained correspond to those expected. We performed 10,000 random subsamplings of 5% of the 9.5 million unique sequences in the TARA V9 dataset ( Karsenti et al., 2011 ). To make this possible with the 32-bit USEARCH, we first downsampled the dataset once to 10% using VSEARCH and then randomly subsampled it again at 50% with either USEARCH or VSEARCH. Plots of the distribution of the abundance of the most abundant sequence in each subsampling are shown in Fig. 7 . The highest amplicon abundance in the original dataset is 15,638,316. After the initial 10% subsampling, the highest abundance was 1,564,267. After the second subsampling, the top abundances should therefore have a distribution centred on a value of 782,133.5. As can be seen from the figure, the USEARCH distribution has a mean that is about 2,000 too small, while the VSEARCH distribution is correctly centred on the expected value. Subsampling experiments were also performed at 2.5%, 1.5% and 0.5% with similar results, although the errors were of decreasing size. USEARCH seems to under-sample abundant amplicons and to over-sample rare amplicons.
Conclusions
limitationsVSEARCH supports almost all of the commands and options for nucleotide sequence analysis in USEARCH version 7 as well as several new features. It has a 64-bit design and handles large datasets virtually only limited by the amount of available memory. We have demonstrated that VSEARCH is in general more accurate than USEARCH when performing searching, clustering, chimera detection and subsampling. The accuracy is on a par with USEARCH for paired-end reads merging. VSEARCH is faster than USEARCH when performing dereplication and merging of paired-end reads, but slower for clustering and chimera detection. We will continue to improve the accuracy, speed and robustness of VSEARCH in the future, as well as adding new features.
Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 7 of 7 relations here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.
| facet | tag | field | conf. | whose words |
|---|---|---|---|---|
| object_of_study | open-source, memory-unrestricted nucleotide sequence processing for metagenomics, genomics, and population genomics | object_of_study | the paper's words | |
| research_question | whether a transparent 64-bit implementation can reproduce and extend USEARCH functionality with competitive accuracy, speed, and robustness | research_question | the paper's words | |
| contribution | VSEARCH, a multithreaded sequence-analysis suite with heuristic candidate search, optimal alignment, clustering, dereplication, chimera detection, FASTQ processing, paired-read merging, and subsampling | contribution | the paper's words | |
| claim_type | method | claim_type | the paper's words | |
| phenomenon | accuracy, runtime, memory capacity, interoperability, and algorithmic behavior across nucleotide searching, clustering, dereplication, chimera detection, merging, and subsampling | phenomenon | the paper's words | |
| method | system-or-tool-building also in: system-or-tool-building | method | the paper's words | |
| mechanisms | K-mer overlap selects a small candidate set without alignment. also in: shared-word filtering | mechanisms[0] | 0.95 | the tagger's words |
| mechanisms | An index of all possible k-mers maps words to database sequences. also in: word index | mechanisms[1] | 0.95 | the tagger's words |
| mechanisms | Optimal query-candidate global alignment determines final similarity. also in: full dynamic programming | mechanisms[2] | 0.95 | the tagger's words |
| mechanisms | Long sequence pairs switch to Hirschberg/Myers-Miller alignment to avoid extreme memory use. also in: linear-memory alternative algorithm | mechanisms[3] | 0.95 | the tagger's words |
| mechanisms | Parallel SIMD computation offsets optimal-alignment cost. also in: vectorisation and multiple threads | mechanisms[4] | 0.95 | the tagger's words |
| mechanisms | Queries join a qualifying centroid or create a new cluster. also in: greedy centroid algorithm | mechanisms[5] | 0.95 | the tagger's words |
| mechanisms | Open addressing and linear probing identify identical full sequences. also in: hash table | mechanisms[6] | 0.95 | the tagger's words |
| mechanisms | Incremental 64-bit prefix hashing supports prefix dereplication. also in: Fowler–Noll–Vo 1a | mechanisms[7] | 0.95 | the tagger's words |
| mechanisms | Sequence digests provide highly likely unique relabeling. also in: SHA-1 or MD5 | mechanisms[8] | 0.95 | the tagger's words |
| mechanisms | Four sequence segments are compared with candidate parents for chimera scoring. also in: UCHIME algorithm | mechanisms[9] | 0.95 | the tagger's words |
| mechanisms | Parallelized masking suppresses simple repeats and low-complexity regions. also in: Dust algorithm | mechanisms[10] | 0.95 | the tagger's words |
| mechanisms | Forward and reverse-complemented reads are aligned across their overlap. also in: optimal ungapped alignment | mechanisms[11] | 0.95 | the tagger's words |
| mechanisms | Match and mismatch scores are weighted by residue correctness probabilities. also in: quality scores | mechanisms[12] | 0.95 | the tagger's words |
| mechanisms | An optional seed makes shuffled FASTA output reproducible. also in: pseudo random number generator | mechanisms[13] | 0.95 | the tagger's words |
| mechanisms | A count or fraction of sequences is sampled, optionally respecting abundance. also in: randomly extracting | mechanisms[14] | 0.95 | the tagger's words |
| limitations | VSEARCH optimal alignment is more computationally demanding than seed-and-extend search. | limitations[0] | 0.95 | the tagger's words |
| limitations | The linear-memory fallback for long sequences is considerably slower. | limitations[1] | 0.95 | the tagger's words |
| limitations | VSEARCH and USEARCH both ignore some advanced k-mer-index options. | limitations[2] | 0.95 | the tagger's words |
| limitations | Original labels cannot be reconstructed after rereplication. | limitations[3] | 0.95 | the tagger's words |
| limitations | Some USEARCH functions for amino acids, local alignment, UDB databases, and UPARSE were not implemented. | limitations[4] | 0.95 | the tagger's words |
| limitations | The evaluation used small datasets that fit a 32-bit process, so it did not benchmark the main large-memory advantage. | limitations[5] | 0.95 | the tagger's words |
| limitations | Clustering and chimera detection were slower than USEARCH despite generally higher accuracy. | limitations[6] | 0.95 | the tagger's words |
| limitations | Clustering performance depends on input order, algorithm, identity threshold, and dataset composition. | limitations[7] | 0.95 | the tagger's words |
| limitations | Reference-based chimera detection depends strongly on the reference database and all tools failed on one SILVA condition. | limitations[8] | 0.95 | the tagger's words |
| limitations | VSEARCH paired-read merging trades slightly fewer correct percentages than USEARCH 8 for greater merge yield. | limitations[9] | 0.95 | the tagger's words |
| limitations | The free USEARCH comparator is a 32-bit build, making feature parity and large-data comparison structurally asymmetric. | limitations[10] | 0.95 | the tagger's words |
| controls | all performance datasets constrained to fit a 32-bit process eliminates: memory failure of USEARCH from ordinary task comparisons · outcome: held | controls[0] | 0.95 | the tagger's words |
| controls | same 16-core nodes, 64 GB RAM, and eight threads when possible eliminates: hardware and thread-count differences · outcome: held | controls[1] | 0.95 | the tagger's words |
| controls | 20 random RFAM shufflings eliminates: one sequence-order realization driving search accuracy · outcome: held | controls[2] | 0.95 | the tagger's words |
| controls | 1,000 replicated RFAM datasets eliminates: unstable timing from extremely short searches · outcome: held | controls[3] | 0.95 | the tagger's words |
| controls | even and uneven mock communities, ten shufflings, two orderings, and twenty thresholds eliminates: one composition, order, or identity threshold determining clustering conclusions · outcome: held | controls[4] | 0.95 | the tagger's words |
| controls | repeated UCHIME SIMM benchmark plus new Greengenes and SILVA simulations eliminates: chimera performance depending on one legacy benchmark · outcome: held | controls[5] | 0.95 | the tagger's words |
| controls | perfect alignment of merged reads to known references eliminates: counting all merged read pairs as correct · outcome: held | controls[6] | 0.95 | the tagger's words |
| controls | 10,000 random subsamplings compared with an analytically expected center eliminates: judging subsampling only from apparent randomness · outcome: held | controls[7] | 0.95 | the tagger's words |
| concepts | VSEARCH also in: VSEARCH | concepts[0] | 0.95 | the paper's words |
| concepts | open source and free of charge multithreaded 64-bit tool | concepts[1] | 0.95 | the paper's words |
| concepts | metagenomics also in: metagenomics | concepts[2] | 0.95 | the paper's words |
| concepts | nucleotide sequence data also in: nucleotide sequence data | concepts[3] | 0.95 | the paper's words |
| concepts | USEARCH also in: USEARCH | concepts[4] | 0.95 | the paper's words |
| concepts | words shared by the query and target sequences | concepts[5] | 0.95 | the paper's words |
| concepts | optimal global sequence alignment also in: optimal global sequence alignment | concepts[6] | 0.95 | the paper's words |
| concepts | full dynamic programming also in: full dynamic programming | concepts[7] | 0.95 | the paper's words |
| concepts | seed-and-extend heuristic also in: seed-and-extend heuristic | concepts[8] | 0.95 | the paper's words |
| concepts | vectorisation also in: vectorisation | concepts[9] | 0.95 | the paper's words |
| concepts | multiple threads also in: multiple threads | concepts[10] | 0.95 | the paper's words |
| concepts | global pairwise sequence comparison also in: global pairwise sequence comparison | concepts[11] | 0.95 | the paper's words |
| concepts | core functionality also in: core functionality | concepts[12] | 0.95 | the paper's words |
| concepts | shared words also in: shared words | concepts[13] | 0.95 | the paper's words |
| concepts | Words also in: Words | concepts[14] | 0.95 | the paper's words |
| concepts | bitmap also in: bitmap | concepts[15] | 0.95 | the paper's words |
| concepts | optimal global alignment also in: optimal global alignment | concepts[16] | 0.95 | the paper's words |
| concepts | alternative algorithm also in: alternative algorithm | concepts[17] | 0.95 | the paper's words |
| concepts | database sequence also in: database sequence | concepts[18] | 0.95 | the paper's words |
| concepts | similarity equal to or greater than the value specified with the id option | concepts[19] | 0.95 | the paper's words |
| concepts | centroid-based algorithm also in: centroid-based algorithm | concepts[20] | 0.95 | the paper's words |
| concepts | sequence similarity threshold also in: sequence similarity threshold | concepts[21] | 0.95 | the paper's words |
| concepts | distance-based greedy clustering also in: distance-based greedy clustering | concepts[22] | 0.95 | the paper's words |
| concepts | abundance-based greedy clustering also in: abundance-based greedy clustering | concepts[23] | 0.95 | the paper's words |
| concepts | Full-length dereplication also in: Full-length dereplication | concepts[24] | 0.95 | the paper's words |
| concepts | hash table also in: hash table | concepts[25] | 0.95 | the paper's words |
| concepts | Prefix dereplication also in: Prefix dereplication | concepts[26] | 0.95 | the paper's words |
| concepts | Fowler–Noll–Vo 1a hash function also in: Fowler–Noll–Vo 1a hash function | concepts[27] | 0.95 | the paper's words |
| concepts | SHA-1 also in: SHA-1 | concepts[28] | 0.95 | the paper's words |
| concepts | MD5 also in: MD5 | concepts[29] | 0.95 | the paper's words |
| concepts | rereplication also in: rereplication | concepts[30] | 0.95 | the paper's words |
| concepts | Chimeras also in: Chimeras | concepts[31] | 0.95 | the paper's words |
| concepts | UCHIME algorithm also in: UCHIME algorithm | concepts[32] | 0.95 | the paper's words |
| concepts | Dust algorithm also in: Dust algorithm | concepts[33] | 0.95 | the paper's words |
| concepts | soft-masking also in: soft-masking | concepts[34] | 0.95 | the paper's words |
| concepts | FASTQ also in: FASTQ | concepts[35] | 0.95 | the paper's words |
| concepts | paired-end reads also in: paired-end reads | concepts[36] | 0.95 | the paper's words |
| concepts | optimal ungapped alignment also in: optimal ungapped alignment | concepts[37] | 0.95 | the paper's words |
| concepts | posterior quality scores also in: posterior quality scores | concepts[38] | 0.95 | the paper's words |
| concepts | pseudo random number generator also in: pseudo random number generator | concepts[39] | 0.95 | the paper's words |
| concepts | subsampled also in: subsampled | concepts[40] | 0.95 | the paper's words |
| concepts | drop-in replacement also in: drop-in replacement | concepts[41] | 0.95 | the paper's words |
| concepts | RFAM also in: RFAM | concepts[42] | 0.95 | the paper's words |
| concepts | false discovery rate also in: false discovery rate | concepts[43] | 0.95 | the paper's words |
| concepts | recall also in: recall | concepts[44] | 0.95 | the paper's words |
| concepts | precision also in: precision | concepts[45] | 0.95 | the paper's words |
| concepts | adjusted Rand index also in: adjusted Rand index | concepts[46] | 0.95 | the paper's words |
| concepts | SIMM dataset also in: SIMM dataset | concepts[47] | 0.95 | the paper's words |
| concepts | Greengenes also in: Greengenes | concepts[48] | 0.95 | the paper's words |
| concepts | SILVA also in: SILVA | concepts[49] | 0.95 | the paper's words |
| concepts | reference-based detection also in: reference-based detection | concepts[50] | 0.95 | the paper's words |
| concepts | de novo chimera detection also in: de novo chimera detection | concepts[51] | 0.95 | the paper's words |
| concepts | correctly merged also in: correctly merged | concepts[52] | 0.95 | the paper's words |
| concepts | expected value also in: expected value | concepts[53] | 0.95 | the paper's words |
| concepts | abundant amplicons also in: abundant amplicons | concepts[54] | 0.95 | the paper's words |
| concepts | rare amplicons also in: rare amplicons | concepts[55] | 0.95 | the paper's words |
| concepts | memory also in: memory | concepts[56] | 0.95 | the paper's words |
| concepts | low-complexity nucleotide sequences also in: low-complexity nucleotide sequences | concepts[57] | 0.95 | the paper's words |
| concepts | clustering threshold also in: clustering threshold | concepts[58] | 0.95 | the paper's words |
| concepts | time used also in: time used | concepts[59] | 0.95 | the paper's words |
| concepts | grouped together in the same OTU also in: grouped together in the same OTU | concepts[60] | 0.95 | the paper's words |
| concepts | assigned to the same species also in: assigned to the same species | concepts[61] | 0.95 | the paper's words |
| unhoused_relations | database sequence → similarity equal to or greater than the value specified with the id option A database sequence is accepted when similarity meets or exceeds the id threshold. | unhoused_relations[0] | 0.95 | the tagger's words |
| unhoused_relations | shared words → database sequence Shared-word counts identify promising database candidates before alignment. | unhoused_relations[1] | 0.95 | the tagger's words |
| unhoused_relations | full dynamic programming → optimal global alignment Full dynamic programming computes optimal pairwise alignments. | unhoused_relations[2] | 0.95 | the tagger's words |
| unhoused_relations | full dynamic programming → memory Exact alignment of long sequences can require extreme memory. | unhoused_relations[3] | 0.95 | the tagger's words |
| unhoused_relations | vectorisation → full dynamic programming Vectorisation compensates part of exact alignment's computational cost. | unhoused_relations[4] | 0.95 | the tagger's words |
| unhoused_relations | sequence similarity threshold → centroid-based algorithm The threshold determines whether a query joins a centroid or creates a new cluster. | unhoused_relations[5] | 0.95 | the tagger's words |
| unhoused_relations | Dust algorithm → low-complexity nucleotide sequences DUST masks low-complexity sequence regions before comparison. | unhoused_relations[6] | 0.95 | the tagger's words |
| unhoused_relations | USEARCH → memory The free 32-bit USEARCH build limits memory to under 4 GB. | unhoused_relations[7] | 0.95 | the tagger's words |
| unhoused_relations | clustering threshold → time used Clustering runtime depends on dataset, algorithm, and threshold. | unhoused_relations[8] | 0.95 | the tagger's words |
| unhoused_relations | reference-based detection → SILVA Reference-based chimera detection performs poorly on the SILVA simulation. | unhoused_relations[9] | 0.95 | the tagger's words |
| unhoused_relations | VSEARCH → abundant amplicons VSEARCH subsampling preserves the expected abundance center. | unhoused_relations[10] | 0.95 | the tagger's words |
| unhoused_relations | USEARCH → abundant amplicons USEARCH appears to under-sample abundant amplicons. | unhoused_relations[11] | 0.95 | the tagger's words |
| unhoused_relations | USEARCH → rare amplicons USEARCH appears to over-sample rare amplicons. | unhoused_relations[12] | 0.95 | the tagger's words |
| senses | open source as inspectable unrestricted implementation rather than free academic binary access | senses[0] | 0.95 | the tagger's words |
| senses | search as heuristic candidate filtering followed by exact alignment | senses[1] | 0.95 | the tagger's words |
| senses | word as a unique fixed-length nucleotide substring counted once per sequence | senses[2] | 0.95 | the tagger's words |
| senses | global alignment as accuracy-oriented full dynamic programming | senses[3] | 0.95 | the tagger's words |
| senses | clustering as greedy assignment to a centroid under an identity threshold | senses[4] | 0.95 | the tagger's words |
| senses | dereplication as collapsing exact full or prefix-redundant sequences | senses[5] | 0.95 | the tagger's words |
| senses | recall as avoidance of splitting one species across OTUs | senses[6] | 0.95 | the tagger's words |
| senses | precision as avoidance of grouping multiple species in one OTU | senses[7] | 0.95 | the tagger's words |
| senses | adjusted Rand index as combined chance-corrected clustering agreement | senses[8] | 0.95 | the tagger's words |
| senses | chimera accuracy as classification of individual reads rather than clusters | senses[9] | 0.95 | the tagger's words |
| senses | correct merge as perfect alignment to a known reference | senses[10] | 0.95 | the tagger's words |
| senses | subsampling correctness as centering on analytically expected abundance | senses[11] | 0.95 | the tagger's words |
| claim_relations | contrasts-with → USEARCH note: VSEARCH is explicitly designed as an open 64-bit alternative. | claim_relations[0] | 0.95 | the tagger's words |
| claim_relations | extends → USEARCH version 7 command-line interface note: It retains most commands while adding functions absent from USEARCH. | claim_relations[1] | 0.95 | the tagger's words |
| claim_relations | builds-on → UCHIME chimera detection note: Both reference and de novo detection implement the published UCHIME algorithm. | claim_relations[2] | 0.95 | the tagger's words |
| claim_relations | builds-on → Needleman-Wunsch, SWIPE, Hirschberg, and Myers-Miller alignment methods note: The search phase adapts these established exact-alignment methods. | claim_relations[3] | 0.95 | the tagger's words |
| claim_relations | contrasts-with → seed-and-extend alignment note: VSEARCH uses full dynamic programming where USEARCH is described as heuristic. | claim_relations[4] | 0.95 | the tagger's words |
| concept_edges | VSEARCH —is-a→ open source and free of charge multithreaded 64-bit tool the relation itself is separately evidenced · strength of the relation: 1 · note: Direct categorical statement. · the relation stated: VSEARCH is an open source and free of charge multithreaded 64-bit tool. | concept_edges[0] | 0.95 | the tagger's words |
| concept_edges | global pairwise sequence comparison —is-a→ core functionality the relation itself is separately evidenced · strength of the relation: 1 · note: Direct categorical statement. · the relation stated: Global pairwise sequence comparison is a core functionality of VSEARCH. | concept_edges[1] | 0.95 | the tagger's words |
| concept_edges | VSEARCH —contrasted-with→ USEARCH the relation itself is separately evidenced · strength of the relation: 0.95 · note: Explicit alternative-to contrast. · the relation stated: VSEARCH is designed as an alternative to USEARCH with public source and unrestricted memory. | concept_edges[2] | 0.95 | the tagger's words |
| concept_edges | full dynamic programming —contrasted-with→ seed-and-extend heuristic the relation itself is separately evidenced · strength of the relation: 0.95 · note: Explicit instead-of contrast. · the relation stated: VSEARCH uses full dynamic programming instead of USEARCH's seed-and-extend heuristic. | concept_edges[3] | 0.95 | the tagger's words |
| concept_edges | recall —operationalised-as→ grouped together in the same OTU the relation itself is separately evidenced · strength of the relation: 1 · note: Direct measurement definition. · the relation stated: Recall measures grouping of same-species amplicons without over-splitting. also in: recall · grouped together in the same OTU | concept_edges[4] | 0.95 | the tagger's words |
| concept_edges | precision —operationalised-as→ assigned to the same species the relation itself is separately evidenced · strength of the relation: 1 · note: Direct measurement definition. · the relation stated: Precision measures keeping each OTU to one species without over-grouping. also in: precision · assigned to the same species | concept_edges[5] | 0.95 | the tagger's words |
| concept_edges | de novo chimera detection —contrasted-with→ reference-based detection the relation itself is separately evidenced · strength of the relation: 0.95 · note: Direct comparative result. · the relation stated: De novo chimera detection performs better than reference-based detection in these simulations. | concept_edges[6] | 0.95 | the tagger's words |
| content_tags | VSEARCH provides a free open-source 64-bit alternative to memory-limited closed-source USEARCH. | content_tags.findings[0] | 0.95 | the tagger's words |
| content_tags | Shared-word filtering narrows candidate sequences before optimal global alignment. | content_tags.findings[1] | 0.95 | the tagger's words |
| content_tags | Vectorisation and multithreading parallelize pairwise full-dynamic-programming alignments. | content_tags.findings[2] | 0.95 | the tagger's words |
| content_tags | Across evaluated tasks VSEARCH was more accurate for searching, clustering, chimera detection, and subsampling and comparable for paired-read merging. | content_tags.findings[3] | 0.95 | the tagger's words |
| content_tags | VSEARCH was slower for clustering and chimera detection but faster for merging and dereplication. | content_tags.findings[4] | 0.95 | the tagger's words |
| content_tags | The 64-bit design removes the under-4-GB address-space restriction of freely available USEARCH builds. | content_tags.findings[5] | 0.95 | the tagger's words |
| content_tags | VSEARCH handles compressed FASTA and FASTQ automatically and supports pipe-based workflows without slow disk writes. | content_tags.findings[6] | 0.95 | the tagger's words |
| content_tags | Full dynamic programming improves alignment accuracy at greater computational cost, partly offset by vectorisation. | content_tags.findings[7] | 0.95 | the tagger's words |
| content_tags | A linear-memory alternative is used for sequence-length products above 25 million. | content_tags.findings[8] | 0.95 | the tagger's words |
| content_tags | Centroid clustering supports input, length, or abundance ordering and configurable similarity thresholds. | content_tags.findings[9] | 0.95 | the tagger's words |
| content_tags | Prefix dereplication resolves ambiguous prefix matches using length, abundance, identifier, and original-order priorities. | content_tags.findings[10] | 0.95 | the tagger's words |
| content_tags | VSEARCH implements reference-based and de novo UCHIME chimera detection and parallelizes the reference mode. | content_tags.findings[11] | 0.95 | the tagger's words |
| content_tags | Its optimized DUST implementation is considerably faster than the corresponding USEARCH implementation. | content_tags.findings[12] | 0.95 | the tagger's words |
| content_tags | Nearly all USEARCH 7 command and option names are retained, allowing many QIIME commands to use VSEARCH through an alias. | content_tags.findings[13] | 0.95 | the tagger's words |
| content_tags | In RFAM searching VSEARCH was most accurate at false discovery rates between 0.010 and 0.015 and had similar runtime. | content_tags.findings[14] | 0.95 | the tagger's words |
| content_tags | Abundance-sorted clustering differences were negligible, while VSEARCH led near the 95 to 97 percent identity region under length sorting. | content_tags.findings[15] | 0.95 | the tagger's words |
| content_tags | USEARCH clustering was generally two to three times faster, but full-dynamic-programming USEARCH was much slower without improving accuracy. | content_tags.findings[16] | 0.95 | the tagger's words |
| content_tags | VSEARCH full-length dereplication was 40 to 70 percent faster across the tested datasets. | content_tags.findings[17] | 0.95 | the tagger's words |
| content_tags | VSEARCH prefix dereplicated BioMarKs in under four seconds where USEARCH took minutes. | content_tags.findings[18] | 0.95 | the tagger's words |
| content_tags | On SIMM chimeras VSEARCH was superior in almost all cases and USEARCH won only four of 99 cases. | content_tags.findings[19] | 0.95 | the tagger's words |
| content_tags | De novo chimera detection outperformed reference-based detection in the simulated microbial tests and depended on the reference database. | content_tags.findings[20] | 0.95 | the tagger's words |
| content_tags | VSEARCH paired-read merging balanced the number and correctness of merges and ran two to five times faster than USEARCH. | content_tags.findings[21] | 0.95 | the tagger's words |
| content_tags | VSEARCH subsampling centered abundant amplicons on the expected value while USEARCH under-sampled abundant and over-sampled rare amplicons. | content_tags.findings[22] | 0.95 | the tagger's words |
| content_tags | shared-word filtering followed by optimal query-to-candidate alignment also in: global pairwise sequence comparison | content_tags.definitions[1] | 0.95 | the tagger's words |
| content_tags | assigning a query to the sufficiently similar centroid with greatest similarity also in: distance-based greedy clustering | content_tags.definitions[2] | 0.95 | the tagger's words |
| content_tags | assigning a query to the sufficiently similar centroid with greatest abundance also in: abundance-based greedy clustering | content_tags.definitions[3] | 0.95 | the tagger's words |
| content_tags | hash-based grouping of identical entire input sequences also in: full-length dereplication | content_tags.definitions[4] | 0.95 | the tagger's words |
| content_tags | grouping sequences identical in full or as prefixes also in: prefix dereplication | content_tags.definitions[5] | 0.95 | the tagger's words |
| content_tags | the extent to which same-species amplicons stay in one OTU without over-splitting also in: recall | content_tags.definitions[6] | 0.95 | the tagger's words |
| content_tags | the extent to which an OTU contains amplicons from one species without over-grouping also in: precision | content_tags.definitions[7] | 0.95 | the tagger's words |
| content_tags | chance-adjusted agreement between two clusterings summarizing precision and recall also in: adjusted Rand index | content_tags.definitions[8] | 0.95 | the tagger's words |
| content_tags | a merged sequence perfectly alignable to its reference also in: correctly merged | content_tags.definitions[9] | 0.95 | the tagger's words |
| content_tags | 64-bit — VSEARCH process and data-addressing design | content_tags.quantities[0] | 0.95 | the tagger's words |
| content_tags | less than 4GB — memory ceiling of free 32-bit USEARCH versions | content_tags.quantities[1] | 0.95 | the tagger's words |
| content_tags | 8 — default shared-word length | content_tags.quantities[2] | 0.95 | the tagger's words |
| content_tags | 10 — default minimum shared-word matches | content_tags.quantities[3] | 0.95 | the tagger's words |
| content_tags | 25,000,000 — sequence-length product triggering linear-memory alignment | content_tags.quantities[4] | 0.95 | the tagger's words |
| content_tags | 0.97 — example sequence-similarity threshold for clustering | content_tags.quantities[5] | 0.95 | the tagger's words |
| content_tags | 1 — default maximum accepted search hits | content_tags.quantities[6] | 0.95 | the tagger's words |
| content_tags | 32 — default maximum rejected search candidates | content_tags.quantities[7] | 0.95 | the tagger's words |
| content_tags | 383,004 — RFAM v11 sequences in search evaluation | content_tags.quantities[8] | 0.95 | the tagger's words |
| content_tags | 20 shufflings — search evaluation repetitions combined | content_tags.quantities[9] | 0.95 | the tagger's words |
| content_tags | 5 min 26 s — VSEARCH RFAM search time across 1,000 replicate datasets | content_tags.quantities[10] | 0.95 | the tagger's words |
| content_tags | 80% to 99% — similarity-threshold range in clustering evaluation | content_tags.quantities[11] | 0.95 | the tagger's words |
| content_tags | 40–70% faster — range of VSEARCH full-length dereplication advantage across USEARCH versions | content_tags.quantities[12] | 0.95 | the tagger's words |
| content_tags | 900 — simulated chimeras in the repeated UCHIME evaluation | content_tags.quantities[13] | 0.95 | the tagger's words |
| content_tags | 10,000 — random subsampling repetitions | content_tags.quantities[14] | 0.95 | the tagger's words |
| content_tags | 782,133.5 — expected center of maximum abundance after two-stage 5-percent subsampling | content_tags.quantities[15] | 0.95 | the tagger's words |
| content_tags | shared words also in: shared words | content_tags.variables[0] | 0.95 | the tagger's words |
| content_tags | sequence similarity also in: sequence similarity | content_tags.variables[1] | 0.95 | the tagger's words |
| content_tags | wordlength also in: wordlength | content_tags.variables[2] | 0.95 | the tagger's words |
| content_tags | sequence length also in: sequence length | content_tags.variables[3] | 0.95 | the tagger's words |
| content_tags | clustering threshold also in: clustering threshold | content_tags.variables[4] | 0.95 | the tagger's words |
| content_tags | false discovery rate also in: false discovery rate | content_tags.variables[6] | 0.95 | the tagger's words |
| content_tags | recall also in: recall | content_tags.variables[7] | 0.95 | the tagger's words |
| content_tags | precision also in: precision | content_tags.variables[8] | 0.95 | the tagger's words |
| content_tags | adjusted Rand index also in: adjusted Rand index | content_tags.variables[9] | 0.95 | the tagger's words |
| content_tags | chimera score also in: chimera score | content_tags.variables[10] | 0.95 | the tagger's words |
| content_tags | merged pairs also in: merged pairs | content_tags.variables[11] | 0.95 | the tagger's words |
| content_tags | correct merges also in: correct merges | content_tags.variables[12] | 0.95 | the tagger's words |
Tags that could not be placed in the text:
What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.