Damian SzklarczykpersName, Rebecca KirschpersName, Mikaela KoutroulipersName, Katerina NastoupersName, Farrokh MehryarypersName, Radja HachilifpersName, Annika L GablepersName, Tao FangpersName, Nadezhda T DonchevapersName, Sampo PyysalopersName, Peer BorkpersName, Lars J JensenpersName, Christian von MeringpersName (2022date). “The STRING database in 2023: protein–protein association networks and functional enrichment analyses for any sequenced genome of interest.” Nucleic Acids Researchtitle·j.
Object of study — the STRINGterm database of protein-protein associations: how it collects, scores and integrates physical and functional interaction evidence from many sources and transfers it across organisms, and the developments in its version 12.0
Research question — how interaction evidence that is scattered across resources, experimental modalities and levels of detail, each with its own biases and noise, can be assessed and integrated into organism-wide networks that stay current as new genomes are sequenced
Contribution — STRINGterm 12.0: annotation of any user-uploaded proteometerm through hierarchical orthologyterm, a co-expressionterm channel rebuilt on variational autoencodersterm with single-cell RNA-seqterm and proteomics sources, experiment-level confidence scoring shown as three-tier grades, and a text-mining model that detects cross-sentence physical interactionsterm
Cross-disciplinary analysis
content-tags & facets:
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | the same interaction detection methods earn far lower confidence when used in high-throughput experiments than in low-throughput ones: high-throughput associations sit at or below a score of about 0.25, while low-throughput experiments with the same methods score more than twice as high, around 0.6 — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | |
| claim-type | method — “Much of the complexity within cells arises from functional and regulatory interactions among proteins. The core of these interactions is increasingly known, but novel interactions continue to be discovered, and the information remains scattered across different database resources, experimental modalities and levels of mechanistic detail. The STRING database ( https://string-db.org/ ) systematically collects and integrates protein–protein interactions—both physical interactions as well as functional associations. The data originate from a number of sources: automated text mining of the scientific literature, computational interaction predictions from co-expression, conserved genomic context, databases of interaction experiments and known complexes/pathways from curated sources. All of these interactions are critically assessed, scored, and subsequently automatically transferred to less well-studied organisms using hierarchical orthology information. The data can be accessed via the website, but also programmatically and via bulk downloads. The most recent developments in STRING (version 12.0) are: (i) it is now possible to create, browse and analyze a full interaction network for any novel genome of interest, by submitting its complement of encoded proteins, (ii) the co-expression channel now uses variational auto-encoders to predict interactions, and it covers two new sources, single-cell RNA-seq and experimental proteomics data and (iii) the confidence in each experimentally derived interaction is now estimated based on the detection method used, and communicated to the user in the web-interface. Furthermore, STRING continues to enhance its facilities for functional enrichment analysis, which are now fully available also for user-submitted genomes.” | |
| method — evidence | — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | |
| object of study — evidence | — “Much of the complexity within cells arises from functional and regulatory interactions among proteins. The core of these interactions is increasingly known, but novel interactions continue to be discovered, and the information remains scattered across different database resources, experimental modalities and levels of mechanistic detail. The STRING database ( https://string-db.org/ ) systematically collects and integrates protein–protein interactions—both physical interactions as well as functional associations. The data originate from a number of sources: automated text mining of the scientific literature, computational interaction predictions from co-expression, conserved genomic context, databases of interaction experiments and known complexes/pathways from curated sources. All of these interactions are critically assessed, scored, and subsequently automatically transferred to less well-studied organisms using hierarchical orthology information. The data can be accessed via the website, but also programmatically and via bulk downloads. The most recent developments in STRING (version 12.0) are: (i) it is now possible to create, browse and analyze a full interaction network for any novel genome of interest, by submitting its complement of encoded proteins, (ii) the co-expression channel now uses variational auto-encoders to predict interactions, and it covers two new sources, single-cell RNA-seq and experimental proteomics data and (iii) the confidence in each experimentally derived interaction is now estimated based on the detection method used, and communicated to the user in the web-interface. Furthermore, STRING continues to enhance its facilities for functional enrichment analysis, which are now fully available also for user-submitted genomes.” | |
| research question — evidence | — “A dense network of functional connections among proteins has evolved to support cellular life, forming a multitude of pathways, protein complexes and cellular structures ( 1 , 2 ). Recent developments have further improved our ability to unravel this connectivity, through techniques such as high-throughput genetic screens ( 3–5 ), systematic co-fractionation of proteomes ( 6 , 7 ), in-vivo proteome-wide cross-linking of proteins ( 8–10 ), and deep learning-based computational prediction frameworks such as AlphaFold ( 11 , 12 ). These efforts complement earlier results based on focused, small-scale laboratory studies, yeast two-hybrid screens, affinity purifications, co-crystallization and computational prediction algorithms (reviewed in ( 13 )). Together, the many approaches have begun to reveal a large part of the interaction landscape of cellular proteins, but they each have strengths and weaknesses, including potential biases, false negatives and noise. Furthermore, the resulting interaction data is scattered over a number of online resources, and available in varying namespaces and data-formats as well as varying levels of detail.” | |
| contribution — evidence | — “Here, we provide an update on the current features of the STRING database and describe some novel developments in more detail. The latter include a complete redesign of the co-expression based interaction prediction pipeline, newly exposed sub-scores for experimental dataset confidence, as well as novel facilities allowing users to upload and analyze any newly sequenced genome of interest.” | |
| mechanism | interolog transfer Associations found in one organism are carried to others on the assumption that orthologs of associated proteins are likewise associated, using eggNOG hierarchical orthology at several taxonomic levels. — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | mechanisms[0] |
| mechanism | autoencoder compression of expression data Encoding sparse, high-dimensional, redundant expression matrices into lower-dimensional latent spaces both reduces sparsity and limits redundancy before associations are scored. — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | mechanisms[1] |
| mechanism | shared-interactor raw scoring Within a high-throughput experiment each protein pair gets a raw score that rises with shared interactors and falls with non-shared ones; pairs are ranked by it and calibrated against KEGG per experiment. — “For interactions detected by HT experiments, benchmarking against KEGG molecular pathway maps and scoring is performed separately for each experiment: each pair is assigned a raw score based on the number of shared and non-shared interactors for both proteins within this experiment. The more shared and the fewer non-shared interactors, the higher the raw quality score of the protein pair from this experiment. Based on the raw score, all pairs detected in the experiment are ranked and benchmarked against KEGG. A typical STRING calibration function is derived for each HT experiment, which is then used to assign a functional association score to each protein pair identified in the experiment ( 33 ).” | mechanisms[2] |
| mechanism | climbing the orthology hierarchy for pathway consensus A newly submitted protein takes the consensus pathway annotation of its most specific orthologous group; if that group carries none, the consensus is tried one level up, repeating until it succeeds or reaches the root. — “The network prediction based on orthology is then performed as described previously ( 46 ). In parallel with the network prediction, STRING also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRING from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGG pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRING proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthology hierarchy; this is repeated until it is successful or until the root level is reached. STRING does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestor of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRING database, bulk download files are generated, and the user is given a unique STRING proteome identifier. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.” | mechanisms[3] |
| mechanism | cross-sentence relation extraction A roughly fortyfold larger set of manually annotated relations added many cross-sentence pairs to training, which let the text-mining model learn interactions stated across sentence boundaries. — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | mechanisms[4] |
| builds-on | KEGG pathway maps as the gold standard against which every confidence score is calibrated — “Improved interaction prediction based on co-expression. Interaction scores based on co-expression have been ranked and benchmarked against common KEGG pathway membership as ground truth. ( A ) Performance comparison of co-expression network between STRING version 11.5 and STRING version 12.0. ( B ) Overview of the performance of all expression datasets contributing to the STRING version 12.0 co-expression channel.” | claim_relations[0] |
| builds-on | the eggNOG hierarchical orthology database used for interolog transfer and for placing submitted proteins — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | claim_relations[1] |
| builds-on | FAVA (Functional Associations using Variational Autoencoders) for co-expression — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | claim_relations[2] |
| builds-on | a biomedical RoBERTa-large language model replacing BioBERT for relation extraction — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | claim_relations[3] |
| builds-on | the DIAMOND aligner for mapping user-uploaded proteomes onto the database — “To improve this situation, STRING now allows its users to upload any fully sequenced proteome (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRING database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysis, bulk download, and REST API access. The outline of the proteome annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteome begins by choosing ‘Annotate your proteome’ on the STRING input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteome, as well as the taxonomic name of the species or clade that the uploaded proteome belongs to. Along with the protein sequences from the file, if provided, STRING will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRING automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteome is uploaded, STRING directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthology database eggNOG ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestor between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthology groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteome of an organism already included in the database, the proteins of the matching proteomes will then directly map to each other, without a need for mapping to any of the existing orthologous groups.” | claim_relations[4] |
| builds-on | the PSI-MI Molecular Interaction Controlled Vocabulary for classifying detection methods — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | claim_relations[5] |
| extends | STRING version 11.5 and the earlier STRING releases — “Here, we provide an update on the current features of the STRING database and describe some novel developments in more detail. The latter include a complete redesign of the co-expression based interaction prediction pipeline, newly exposed sub-scores for experimental dataset confidence, as well as novel facilities allowing users to upload and analyze any newly sequenced genome of interest.” | claim_relations[6] |
| sense | association - deliberately wider than interaction: it covers physical binding, indirect cooperation in a pathway, regulation through intermediaries, shared structure, and even proteins that act antagonistically in the same pathway; the paper itself writes 'assocation' in the sentence that stretches the word furthest — “The scope of protein–protein links in STRING is that of a ‘functional association’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complex or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGG ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.” | senses[0] |
| sense | experiment - not a laboratory session but a bookkeeping unit: all protein pairs sharing one PubMed identifier and one detection method, then split into high- and low-throughput by a 25-interaction threshold — “For the experiments channel, STRING integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experiment’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughput (HT) experiment if at least 25 unique interactions are reported, otherwise as a low-throughput (LT) experiment.” | senses[1] |
| sense | confidence - a calibrated probability between zero and one benchmarked against KEGG, and separately a three-grade label (high, medium, exploratory) shown to users; one word for a number and for a verbal band — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | senses[2] |
| sense | organism name - the slot that names a species on the input page now also holds an arbitrary proteome identifier minted for a user upload, so an identifier functions as an organism name — “The network prediction based on orthology is then performed as described previously ( 46 ). In parallel with the network prediction, STRING also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRING from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGG pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRING proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthology hierarchy; this is repeated until it is successful or until the root level is reached. STRING does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestor of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRING database, bulk download files are generated, and the user is given a unique STRING proteome identifier. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.” | senses[3] |
| note | KEGG pathway maps as the gold standard against which every confidence score is calibrated — Every channel, including the new co-expression channel, is ranked and benchmarked against shared KEGG pathway membership. — “Improved interaction prediction based on co-expression. Interaction scores based on co-expression have been ranked and benchmarked against common KEGG pathway membership as ground truth. ( A ) Performance comparison of co-expression network between STRING version 11.5 and STRING version 12.0. ( B ) Overview of the performance of all expression datasets contributing to the STRING version 12.0 co-expression channel.” | claim_relations[0] |
| weight | KEGG pathway maps as the gold standard against which every confidence score is calibrated — 0.95 — “Improved interaction prediction based on co-expression. Interaction scores based on co-expression have been ranked and benchmarked against common KEGG pathway membership as ground truth. ( A ) Performance comparison of co-expression network between STRING version 11.5 and STRING version 12.0. ( B ) Overview of the performance of all expression datasets contributing to the STRING version 12.0 co-expression channel.” | claim_relations[0] |
| note | the eggNOG hierarchical orthology database used for interolog transfer and for placing submitted proteins — Orthology relationships at several taxonomic levels come from eggNOG. — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | claim_relations[1] |
| weight | the eggNOG hierarchical orthology database used for interolog transfer and for placing submitted proteins — 0.93 — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | claim_relations[1] |
| note | FAVA (Functional Associations using Variational Autoencoders) for co-expression — The co-expression channel is rebuilt on this separately published method. — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | claim_relations[2] |
| weight | FAVA (Functional Associations using Variational Autoencoders) for co-expression — 0.92 — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | claim_relations[2] |
| note | a biomedical RoBERTa-large language model replacing BioBERT for relation extraction — The text-mining channel swaps its language representation model. — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | claim_relations[3] |
| weight | a biomedical RoBERTa-large language model replacing BioBERT for relation extraction — 0.9 — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | claim_relations[3] |
| note | the DIAMOND aligner for mapping user-uploaded proteomes onto the database — Submitted proteomes are aligned with DIAMOND before orthology assignment. — “To improve this situation, STRING now allows its users to upload any fully sequenced proteome (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRING database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysis, bulk download, and REST API access. The outline of the proteome annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteome begins by choosing ‘Annotate your proteome’ on the STRING input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteome, as well as the taxonomic name of the species or clade that the uploaded proteome belongs to. Along with the protein sequences from the file, if provided, STRING will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRING automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteome is uploaded, STRING directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthology database eggNOG ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestor between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthology groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteome of an organism already included in the database, the proteins of the matching proteomes will then directly map to each other, without a need for mapping to any of the existing orthologous groups.” | claim_relations[4] |
| weight | the DIAMOND aligner for mapping user-uploaded proteomes onto the database — 0.88 — “To improve this situation, STRING now allows its users to upload any fully sequenced proteome (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRING database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysis, bulk download, and REST API access. The outline of the proteome annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteome begins by choosing ‘Annotate your proteome’ on the STRING input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteome, as well as the taxonomic name of the species or clade that the uploaded proteome belongs to. Along with the protein sequences from the file, if provided, STRING will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRING automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteome is uploaded, STRING directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthology database eggNOG ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestor between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthology groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteome of an organism already included in the database, the proteins of the matching proteomes will then directly map to each other, without a need for mapping to any of the existing orthologous groups.” | claim_relations[4] |
| note | the PSI-MI Molecular Interaction Controlled Vocabulary for classifying detection methods — Which methods count as experimental detection is read from PSI-MI terms. — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | claim_relations[5] |
| weight | the PSI-MI Molecular Interaction Controlled Vocabulary for classifying detection methods — 0.88 — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | claim_relations[5] |
| note | STRING version 11.5 and the earlier STRING releases — The paper is the version 12.0 update of a long-running resource. — “Here, we provide an update on the current features of the STRING database and describe some novel developments in more detail. The latter include a complete redesign of the co-expression based interaction prediction pipeline, newly exposed sub-scores for experimental dataset confidence, as well as novel facilities allowing users to upload and analyze any newly sequenced genome of interest.” | claim_relations[6] |
| weight | STRING version 11.5 and the earlier STRING releases — 0.93 — “Here, we provide an update on the current features of the STRING database and describe some novel developments in more detail. The latter include a complete redesign of the co-expression based interaction prediction pipeline, newly exposed sub-scores for experimental dataset confidence, as well as novel facilities allowing users to upload and analyze any newly sequenced genome of interest.” | claim_relations[6] |
| note | high-throughput —contrasted-with→ low-throughput — Explicit contrast cue 'in contrast' between the two experiment classes, with the measured direction stated. — “Reliability estimates for protein–protein interaction detection assays. The top three most prolific experimental interaction assay types are shown, ranked by the number of protein pairs to which they contribute in STRING. Benchmarked on KEGG pathways, they yield (on average) lower confidence scores when derived from the high-throughput assays ( A ), in contrast, for the equivalent low-throughput experiments (shown in color), the predicted confidence is substantially higher ( B ). The distributions shown in (A) encompass all HT interactions of a given assay type (for simplicity); in the actual scoring computations in STRING, each HT interaction datasets is scored separately.” | concept_edges[0] |
| relation | high-throughput —contrasted-with→ low-throughput — High-throughput assays yield lower confidence scores; in contrast, equivalent low-throughput experiments give substantially higher confidence. — “Reliability estimates for protein–protein interaction detection assays. The top three most prolific experimental interaction assay types are shown, ranked by the number of protein pairs to which they contribute in STRING. Benchmarked on KEGG pathways, they yield (on average) lower confidence scores when derived from the high-throughput assays ( A ), in contrast, for the equivalent low-throughput experiments (shown in color), the predicted confidence is substantially higher ( B ). The distributions shown in (A) encompass all HT interactions of a given assay type (for simplicity); in the actual scoring computations in STRING, each HT interaction datasets is scored separately.” | concept_edges[0] |
| weight | high-throughput —contrasted-with→ low-throughput — 0.9 — “Reliability estimates for protein–protein interaction detection assays. The top three most prolific experimental interaction assay types are shown, ranked by the number of protein pairs to which they contribute in STRING. Benchmarked on KEGG pathways, they yield (on average) lower confidence scores when derived from the high-throughput assays ( A ), in contrast, for the equivalent low-throughput experiments (shown in color), the predicted confidence is substantially higher ( B ). The distributions shown in (A) encompass all HT interactions of a given assay type (for simplicity); in the actual scoring computations in STRING, each HT interaction datasets is scored separately.” | concept_edges[0] |
| relation | physical interactions → functional association — All other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | unhoused_relations[0] |
| weight | physical interactions → functional association — 0.92 — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | unhoused_relations[0] |
| relation | orthology → functional association — Associations are transferred across organisms based on orthology relationships, assuming orthologs of associated proteins are likewise associated. — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | unhoused_relations[1] |
| weight | orthology → functional association — 0.85 — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | unhoused_relations[1] |
| relation | variational autoencoders → co-expression — The benefit of encoding the matrices into fewer dimensions is that it reduces sparsity and limits redundancy. — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | unhoused_relations[2] |
| weight | variational autoencoders → co-expression — 0.88 — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | unhoused_relations[2] |
| relation | KEGG → confidence score — Association evidence is benchmarked against common membership in KEGG pathway maps, yielding a confidence score. — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | unhoused_relations[3] |
| weight | KEGG → confidence score — 0.93 — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | unhoused_relations[3] |
| relation | RNA-Seq → STRING — RNA-Seq sensitivity combined with high-throughput data reduces the inherent literature bias of the STRING network. — “RNA-Seq datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughput nature of the experimental data reduces the inherent literature bias of the STRING network.” | unhoused_relations[4] |
| weight | RNA-Seq → STRING — 0.75 — “RNA-Seq datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughput nature of the experimental data reduces the inherent literature bias of the STRING network.” | unhoused_relations[4] |
| no type for this | physical interactions → functional association — Physical interaction evidence counts as functional association evidence too, while genetic interference evidence counts only as functional: the physical layer sits inside the functional one. — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | unhoused_relations[0] |
| no type for this | orthology → functional association — Orthology licenses transferring an association from studied proteins to their orthologs in other organisms, under the stated assumption that orthologs of associated proteins are also associated. — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | unhoused_relations[1] |
| no type for this | variational autoencoders → co-expression — Compressing expression matrices with variational autoencoders reduces sparsity and redundancy, and with the new sources improves co-expression performance over the previous version. — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | unhoused_relations[2] |
| no type for this | KEGG → confidence score — KEGG pathway co-membership is the ground truth that turns raw evidence of every kind into a calibrated confidence score. — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | unhoused_relations[3] |
| no type for this | RNA-Seq → STRING — RNA-Seq's sensitivity for lowly expressed genes, with high-throughput data, reduces the literature bias that text-derived evidence brings into the network. — “RNA-Seq datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughput nature of the experimental data reduces the inherent literature bias of the STRING network.” | unhoused_relations[4] |
| limitation | What counts as a common function is not formally defined; it is only operationally matched to the specificity of pathway maps in KEGG or Reactome. — “The scope of protein–protein links in STRING is that of a ‘functional association’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complex or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGG ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.” | limitations[0] |
| limitation | A slow release cycle means new genomes and improved gene sets are missed, while a fast one would cost heavy resources and complicate reproducibility and archiving. — “However, new genomes are sequenced and assembled daily, and existing genomes are re-sequenced or re-annotated; overall, the number of taxonomically distinct species that are being sequenced has been doubling roughly every 3 years ( 56 ). The UniProtKB database is updated with the new proteomes on an eight-week cycle, and projects like Ensembl Rapid Release ( 53 ) aim to provide annotation for the newly released genomes on a 2-week cycle. For STRING, such frequent update cycles would require heavy resources and may create complexities with data reproducibility/data archiving when the new updates supersede older datasets. On the other hand, a slow update cycle implies that the database will not incorporate newly sequenced genomes or any improvements to the gene sets of existing genomes.” | limitations[1] |
| limitation | Low-throughput experiments concentrate on particular pathways and known proteins, so no curve-based calibration is possible for them and each detection method receives a single score. — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | limitations[2] |
| limitation | Removing redundant expression matrices in earlier versions did not fully solve the sparsity and redundancy of expression data. — “The expression data tends to be sparse, high-dimensional and highly redundant. These attributes decrease the performance of correlations using Pearson Correlation Coefficient. For previous versions of STRING we have reduced the redundancy by removing highly correlated expression matrices before correlating the gene expression which, in turn, increased the recovery of the functional associations derived from these matrices. However, this did not fully address all of the challenges of such datasets.” | limitations[3] |
| limitation | Every source approach carries its own potential biases, false negatives and noise. — “A dense network of functional connections among proteins has evolved to support cellular life, forming a multitude of pathways, protein complexes and cellular structures ( 1 , 2 ). Recent developments have further improved our ability to unravel this connectivity, through techniques such as high-throughput genetic screens ( 3–5 ), systematic co-fractionation of proteomes ( 6 , 7 ), in-vivo proteome-wide cross-linking of proteins ( 8–10 ), and deep learning-based computational prediction frameworks such as AlphaFold ( 11 , 12 ). These efforts complement earlier results based on focused, small-scale laboratory studies, yeast two-hybrid screens, affinity purifications, co-crystallization and computational prediction algorithms (reviewed in ( 13 )). Together, the many approaches have begun to reveal a large part of the interaction landscape of cellular proteins, but they each have strengths and weaknesses, including potential biases, false negatives and noise. Furthermore, the resulting interaction data is scattered over a number of online resources, and available in varying namespaces and data-formats as well as varying levels of detail.” | limitations[4] |
| limitation | Pathways are never assigned to submitted proteins outside the taxonomic scope in which they were previously annotated. — “The network prediction based on orthology is then performed as described previously ( 46 ). In parallel with the network prediction, STRING also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRING from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGG pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRING proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthology hierarchy; this is repeated until it is successful or until the root level is reached. STRING does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestor of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRING database, bulk download files are generated, and the user is given a unique STRING proteome identifier. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.” | limitations[5] |
Content tags
content-observations:
finding · quantity · variable · definition · key-term — grounded in the full text| finding | The upgraded text-mining model and larger literature corpus doubled the number of physical protein-protein interactions above the low, medium and high confidence cut-offs. — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | content_tags.findings[0] |
| finding | High-throughput affinity chromatography, genetic inference and transcriptional complementation associations average at or below a confidence score of about 0.25. — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.findings[1] |
| finding | Low-throughput experiments with the same detection methods score more than twice as high, with a medium confidence around 0.6. — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.findings[2] |
| finding | Affinity chromatography supplies 40% of experimentally detected associations, genetic inference 34% and transcriptional complementation 11%, and 82% of these come from high-throughput experiments. — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.findings[3] |
| finding | Among low-throughput methods, assays that determine the 3D structure of protein complexes give the highest confidence scores. — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.findings[4] |
| finding | The version 12.0 co-expression network performs significantly better than the version 11.5 network. — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | content_tags.findings[5] |
| finding | All three recent SARS-CoV-2 host-factor screens used STRING to interpret their raw hit lists. — “Researchers employ protein network meta-resources for a wide variety of purposes, broadly falling into three categories: (i) facilitating individual, small-scale molecular discoveries, (ii) facilitating large-scale data analysis and (iii) contributing to new methodologies and workflows. With respect to STRING, it has, for example, proven useful in interpreting and reducing newly acquired genetic screening datasets—these are sometimes noisy and unwieldy, and STRING can be used to distill such data into more manageable sets of observations and hypotheses. As a case in point, consider three recent screens for SARS-CoV-2 human host factors ( 23–25 ). All three studies used STRING to interpret their initial, raw lists of screening hits—placing them into network contexts and searching them for functionally enriched processes/pathways. STRING and its competitors are also often used as data providers in novel methodologies and computational resources, be it new databases, new algorithms, or community-wide competitions. As an example, consider two recent uses of STRING networks in deep-learning frameworks ( 26 , 27 ). Both studies use deep learning to predict protein function (i.e. Gene Ontology terms), based on amino acid sequences and protein–protein interaction network topologies derived from STRING.” | content_tags.findings[6] |
| finding | The number of taxonomically distinct species being sequenced has been doubling roughly every three years. — “However, new genomes are sequenced and assembled daily, and existing genomes are re-sequenced or re-annotated; overall, the number of taxonomically distinct species that are being sequenced has been doubling roughly every 3 years ( 56 ). The UniProtKB database is updated with the new proteomes on an eight-week cycle, and projects like Ensembl Rapid Release ( 53 ) aim to provide annotation for the newly released genomes on a 2-week cycle. For STRING, such frequent update cycles would require heavy resources and may create complexities with data reproducibility/data archiving when the new updates supersede older datasets. On the other hand, a slow update cycle implies that the database will not incorporate newly sequenced genomes or any improvements to the gene sets of existing genomes.” | content_tags.findings[7] |
| finding | RNA-Seq sensitivity to lowly expressed genes, together with high-throughput data, reduces the inherent literature bias of the network. — “RNA-Seq datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughput nature of the experimental data reduces the inherent literature bias of the STRING network.” | content_tags.findings[8] |
| quantity | grew from 6,145 to 243,831, two orders of magnitude — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | content_tags.quantities[0] |
| quantity | on average below or around 0.25 — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.quantities[1] |
| quantity | medium confidence of around 0.6, more than twice the high-throughput level — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.quantities[2] |
| quantity | affinity chromatography 40%, genetic inference 34%, transcriptional complementation 11%; 82% from high-throughput experiments — “The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experiments. The associations derived from these three HT experiments, on average, place below or around a confidence score of 0.25 (Figure 4A ), while, in comparison, the LT experiments for the same methods score more than twice as high with a medium confidence score of around 0.6 (Figure 4B ). The top three methods by confidence score for LT-derived associations are assays determining the 3D structure of protein complexes, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.” | content_tags.quantities[3] |
| quantity | at least 25 unique interactions reported in one experiment — “For the experiments channel, STRING integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experiment’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughput (HT) experiment if at least 25 unique interactions are reported, otherwise as a low-throughput (LT) experiment.” | content_tags.quantities[4] |
| quantity | 12,535 high-quality genomes in version 12.0 — “Version 12.0 of STRING covers a phylogenetically diverse collection of 12 535 high-quality genomes. Beyond these, the system will record which genomes are frequently submitted by users—and these will then be prioritized for inclusion into subsequent releases. In addition, the results of an ongoing online user survey will be analyzed (350 users have already participated). This way, STRING will keep concentrating its resources on those areas that are of most interest to its users.” | content_tags.quantities[5] |
| quantity | more than 10,000 distinct genomes with networks and pathway annotations — “One of the unique features of STRING has always been its support for a large selection of non-model organisms: the current version of the database contains protein–protein interaction networks (and protein functional pathway annotations) for >10 000 distinct genomes. When selecting which genomes to include in a STRING update, key factors are the organism's research prominence, genome quality, and completeness. Subsequently, STRING utilizes genomes from authoritative sources only, including Ensembl ( 53 ), UniProtKB Reference Proteomes ( 54 ) and the ‘representative genomes’ set in the proGenomes database ( 55 ).” | content_tags.quantities[6] |
| quantity | UniProtKB updates proteomes on an eight-week cycle and Ensembl Rapid Release aims at two weeks — “However, new genomes are sequenced and assembled daily, and existing genomes are re-sequenced or re-annotated; overall, the number of taxonomically distinct species that are being sequenced has been doubling roughly every 3 years ( 56 ). The UniProtKB database is updated with the new proteomes on an eight-week cycle, and projects like Ensembl Rapid Release ( 53 ) aim to provide annotation for the newly released genomes on a 2-week cycle. For STRING, such frequent update cycles would require heavy resources and may create complexities with data reproducibility/data archiving when the new updates supersede older datasets. On the other hand, a slow update cycle implies that the database will not incorporate newly sequenced genomes or any improvements to the gene sets of existing genomes.” | content_tags.quantities[7] |
| quantity | 350 users had already taken part in the online user survey — “Version 12.0 of STRING covers a phylogenetically diverse collection of 12 535 high-quality genomes. Beyond these, the system will record which genomes are frequently submitted by users—and these will then be prioritized for inclusion into subsequent releases. In addition, the results of an ongoing online user survey will be analyzed (350 users have already participated). This way, STRING will keep concentrating its resources on those areas that are of most interest to its users.” | content_tags.quantities[8] |
| variable | Pearson Correlation Coefficient — “The expression data tends to be sparse, high-dimensional and highly redundant. These attributes decrease the performance of correlations using Pearson Correlation Coefficient. For previous versions of STRING we have reduced the redundancy by removing highly correlated expression matrices before correlating the gene expression which, in turn, increased the recovery of the functional associations derived from these matrices. However, this did not fully address all of the challenges of such datasets.” | content_tags.variables[1] |
| variable | raw quality score — “For interactions detected by HT experiments, benchmarking against KEGG molecular pathway maps and scoring is performed separately for each experiment: each pair is assigned a raw score based on the number of shared and non-shared interactors for both proteins within this experiment. The more shared and the fewer non-shared interactors, the higher the raw quality score of the protein pair from this experiment. Based on the raw score, all pairs detected in the experiment are ranked and benchmarked against KEGG. A typical STRING calibration function is derived for each HT experiment, which is then used to assign a functional association score to each protein pair identified in the experiment ( 33 ).” | content_tags.variables[2] |
| variable | true-positive rate — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | content_tags.variables[3] |
| definition | functional association — “The scope of protein–protein links in STRING is that of a ‘functional association’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complex or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGG ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.” | content_tags.definitions[0] |
| definition | confidence score — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | content_tags.definitions[1] |
| definition | evidence channels — “All confidence scores in STRING are pre-computed and freely available for download, under a Creative Commons Attribution license (CC BY 4.0). The various evidence types are grouped into seven distinct ‘evidence channels’, with separate sub-scores available for each channel. These channels can also be individually viewed on the STRING user interface, together with their underlying evidence, and can be enabled or disabled separately as desired. Of the seven channels, the first three (‘neighborhood’, ‘fusion’ and ‘co-occurrence’) are dealing with association evidence that can be gleaned from genome sequences alone. These so-called ‘genomic context’ channels (reviewed in ( 35 , 36 )) are based on detecting evolutionary constraints arising from functional gene-gene partnerships, and are best applicable in prokaryotic genomes. Another channel (‘co-expression’) is dealing with functional genomics measurements (transcripts or proteins) across a multitude of conditions, searching for evidence of common expression regulation (see also below). Next, the ‘experiments’ channel deals with laboratory experiments that were conducted with the expressed goal to uncover protein–protein association evidence. They are imported from primary database repositories: BioGRID ( 37 ), DIP ( 38 ), PDB ( 39 ), as well as IntAct and its partner databases in the IMEx consortium ( 40 ). The final two evidence channels are concerned with protein–protein associations that are already known. The ‘database’ channel imports well-established knowledge (‘textbook knowledge’) about protein complexes, pathways and other functional connections from dedicated knowledge resources: KEGG ( 31 ), Reactome ( 32 ), MetaCyc ( 41 ), EBI Complex Portal ( 42 ), and Gene Ontology Complexes ( 43 ). Lastly, the ‘textmining’ channel is the result of parsing full-text articles from the PMC Open Access Subset (up to April 2022), PubMed abstracts (up to August 2022), as well as summary texts from OMIM ( 44 ) and SGD ( 45 ) entry descriptions. These texts are all parsed for co-mentions of protein pairs and assessed against the frequencies of all separate mentions of the respective proteins, as described in ( 46 ). On top of functional associations, the three latter channels (i.e. experiments, database and textmining) provide also the interactions for the physical sub-network of STRING. The calculation of the confidence scores for the protein pairs in this network differs from that of the functional association network and is described in detail in ( 22 ).” | content_tags.definitions[2] |
| definition | genomic context — “All confidence scores in STRING are pre-computed and freely available for download, under a Creative Commons Attribution license (CC BY 4.0). The various evidence types are grouped into seven distinct ‘evidence channels’, with separate sub-scores available for each channel. These channels can also be individually viewed on the STRING user interface, together with their underlying evidence, and can be enabled or disabled separately as desired. Of the seven channels, the first three (‘neighborhood’, ‘fusion’ and ‘co-occurrence’) are dealing with association evidence that can be gleaned from genome sequences alone. These so-called ‘genomic context’ channels (reviewed in ( 35 , 36 )) are based on detecting evolutionary constraints arising from functional gene-gene partnerships, and are best applicable in prokaryotic genomes. Another channel (‘co-expression’) is dealing with functional genomics measurements (transcripts or proteins) across a multitude of conditions, searching for evidence of common expression regulation (see also below). Next, the ‘experiments’ channel deals with laboratory experiments that were conducted with the expressed goal to uncover protein–protein association evidence. They are imported from primary database repositories: BioGRID ( 37 ), DIP ( 38 ), PDB ( 39 ), as well as IntAct and its partner databases in the IMEx consortium ( 40 ). The final two evidence channels are concerned with protein–protein associations that are already known. The ‘database’ channel imports well-established knowledge (‘textbook knowledge’) about protein complexes, pathways and other functional connections from dedicated knowledge resources: KEGG ( 31 ), Reactome ( 32 ), MetaCyc ( 41 ), EBI Complex Portal ( 42 ), and Gene Ontology Complexes ( 43 ). Lastly, the ‘textmining’ channel is the result of parsing full-text articles from the PMC Open Access Subset (up to April 2022), PubMed abstracts (up to August 2022), as well as summary texts from OMIM ( 44 ) and SGD ( 45 ) entry descriptions. These texts are all parsed for co-mentions of protein pairs and assessed against the frequencies of all separate mentions of the respective proteins, as described in ( 46 ). On top of functional associations, the three latter channels (i.e. experiments, database and textmining) provide also the interactions for the physical sub-network of STRING. The calculation of the confidence scores for the protein pairs in this network differs from that of the functional association network and is described in detail in ( 22 ).” | content_tags.definitions[3] |
| definition | interologs — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | content_tags.definitions[4] |
| definition | experiment — “For the experiments channel, STRING integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experiment’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughput (HT) experiment if at least 25 unique interactions are reported, otherwise as a low-throughput (LT) experiment.” | content_tags.definitions[5] |
| definition | FAVA — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | content_tags.definitions[6] |
| definition | interacting unit — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | content_tags.definitions[7] |
| definition | STRING proteome identifier — “The network prediction based on orthology is then performed as described previously ( 46 ). In parallel with the network prediction, STRING also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRING from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGG pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRING proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthology hierarchy; this is repeated until it is successful or until the root level is reached. STRING does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestor of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRING database, bulk download files are generated, and the user is given a unique STRING proteome identifier. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.” | content_tags.definitions[9] |
Concepts
terms this paper carries, each with the sentence it was cut from — the layer that joins one paper to another. Terms with no quote yet are on CONCEPTS.html.
| concept | STRING — not defined in this paper's own words; the protein association network meta-resource — “Given this, a number of meta-resources dedicated to data integration in the protein network context have been developed. These resources aim to collect and critically assess protein–protein functional linkage data, integrate it, connect it to previous knowledge, and allow users to browse, compare and retrieve organism-wide protein networks. Among frequently used and actively maintained frameworks are ConsensusPathDB ( 14 ), FunCoup ( 15 ), GeneMANIA ( 16 ), HumanBase ( 17 ), HumanNet ( 18 ), IID ( 19 ) and STRING ( 20–22 ). Within this group, STRING places its focus on comprehensiveness and ease of use—it covers >10 000 organisms, draws from a wide diversity of data sources including text mining and computational predictions, and offers many intuitive interface features including personalization, enrichment detection and programmatic access.” | concepts[0] |
| concept | functional association — not defined in this paper's own words; an evidenced, evolved functional partnership between two proteins, broader than physical binding — “The scope of protein–protein links in STRING is that of a ‘functional association’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complex or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGG ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.” | concepts[1] |
| concept | physical interactions — not defined in this paper's own words; direct physical binding between proteins, one kind of evidence STRING integrates — “Much of the complexity within cells arises from functional and regulatory interactions among proteins. The core of these interactions is increasingly known, but novel interactions continue to be discovered, and the information remains scattered across different database resources, experimental modalities and levels of mechanistic detail. The STRING database ( https://string-db.org/ ) systematically collects and integrates protein–protein interactions—both physical interactions as well as functional associations. The data originate from a number of sources: automated text mining of the scientific literature, computational interaction predictions from co-expression, conserved genomic context, databases of interaction experiments and known complexes/pathways from curated sources. All of these interactions are critically assessed, scored, and subsequently automatically transferred to less well-studied organisms using hierarchical orthology information. The data can be accessed via the website, but also programmatically and via bulk downloads. The most recent developments in STRING (version 12.0) are: (i) it is now possible to create, browse and analyze a full interaction network for any novel genome of interest, by submitting its complement of encoded proteins, (ii) the co-expression channel now uses variational auto-encoders to predict interactions, and it covers two new sources, single-cell RNA-seq and experimental proteomics data and (iii) the confidence in each experimentally derived interaction is now estimated based on the detection method used, and communicated to the user in the web-interface. Furthermore, STRING continues to enhance its facilities for functional enrichment analysis, which are now fully available also for user-submitted genomes.” | concepts[2] |
| concept | confidence score — not defined in this paper's own words; the calibrated likelihood that an association is true — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | concepts[3] |
| concept | evidence channels — not defined in this paper's own words; the seven separately scored groups of evidence types — “All confidence scores in STRING are pre-computed and freely available for download, under a Creative Commons Attribution license (CC BY 4.0). The various evidence types are grouped into seven distinct ‘evidence channels’, with separate sub-scores available for each channel. These channels can also be individually viewed on the STRING user interface, together with their underlying evidence, and can be enabled or disabled separately as desired. Of the seven channels, the first three (‘neighborhood’, ‘fusion’ and ‘co-occurrence’) are dealing with association evidence that can be gleaned from genome sequences alone. These so-called ‘genomic context’ channels (reviewed in ( 35 , 36 )) are based on detecting evolutionary constraints arising from functional gene-gene partnerships, and are best applicable in prokaryotic genomes. Another channel (‘co-expression’) is dealing with functional genomics measurements (transcripts or proteins) across a multitude of conditions, searching for evidence of common expression regulation (see also below). Next, the ‘experiments’ channel deals with laboratory experiments that were conducted with the expressed goal to uncover protein–protein association evidence. They are imported from primary database repositories: BioGRID ( 37 ), DIP ( 38 ), PDB ( 39 ), as well as IntAct and its partner databases in the IMEx consortium ( 40 ). The final two evidence channels are concerned with protein–protein associations that are already known. The ‘database’ channel imports well-established knowledge (‘textbook knowledge’) about protein complexes, pathways and other functional connections from dedicated knowledge resources: KEGG ( 31 ), Reactome ( 32 ), MetaCyc ( 41 ), EBI Complex Portal ( 42 ), and Gene Ontology Complexes ( 43 ). Lastly, the ‘textmining’ channel is the result of parsing full-text articles from the PMC Open Access Subset (up to April 2022), PubMed abstracts (up to August 2022), as well as summary texts from OMIM ( 44 ) and SGD ( 45 ) entry descriptions. These texts are all parsed for co-mentions of protein pairs and assessed against the frequencies of all separate mentions of the respective proteins, as described in ( 46 ). On top of functional associations, the three latter channels (i.e. experiments, database and textmining) provide also the interactions for the physical sub-network of STRING. The calculation of the confidence scores for the protein pairs in this network differs from that of the functional association network and is described in detail in ( 22 ).” | concepts[4] |
| concept | genomic context — not defined in this paper's own words; sequence-only channels detecting evolutionary constraints between genes — “All confidence scores in STRING are pre-computed and freely available for download, under a Creative Commons Attribution license (CC BY 4.0). The various evidence types are grouped into seven distinct ‘evidence channels’, with separate sub-scores available for each channel. These channels can also be individually viewed on the STRING user interface, together with their underlying evidence, and can be enabled or disabled separately as desired. Of the seven channels, the first three (‘neighborhood’, ‘fusion’ and ‘co-occurrence’) are dealing with association evidence that can be gleaned from genome sequences alone. These so-called ‘genomic context’ channels (reviewed in ( 35 , 36 )) are based on detecting evolutionary constraints arising from functional gene-gene partnerships, and are best applicable in prokaryotic genomes. Another channel (‘co-expression’) is dealing with functional genomics measurements (transcripts or proteins) across a multitude of conditions, searching for evidence of common expression regulation (see also below). Next, the ‘experiments’ channel deals with laboratory experiments that were conducted with the expressed goal to uncover protein–protein association evidence. They are imported from primary database repositories: BioGRID ( 37 ), DIP ( 38 ), PDB ( 39 ), as well as IntAct and its partner databases in the IMEx consortium ( 40 ). The final two evidence channels are concerned with protein–protein associations that are already known. The ‘database’ channel imports well-established knowledge (‘textbook knowledge’) about protein complexes, pathways and other functional connections from dedicated knowledge resources: KEGG ( 31 ), Reactome ( 32 ), MetaCyc ( 41 ), EBI Complex Portal ( 42 ), and Gene Ontology Complexes ( 43 ). Lastly, the ‘textmining’ channel is the result of parsing full-text articles from the PMC Open Access Subset (up to April 2022), PubMed abstracts (up to August 2022), as well as summary texts from OMIM ( 44 ) and SGD ( 45 ) entry descriptions. These texts are all parsed for co-mentions of protein pairs and assessed against the frequencies of all separate mentions of the respective proteins, as described in ( 46 ). On top of functional associations, the three latter channels (i.e. experiments, database and textmining) provide also the interactions for the physical sub-network of STRING. The calculation of the confidence scores for the protein pairs in this network differs from that of the functional association network and is described in detail in ( 22 ).” | concepts[5] |
| concept | co-expression — not defined in this paper's own words; shared expression across conditions, read as evidence of association — “The degree of co-expression between RNAs (or between proteins) across different conditions provides an essential insight into the functional protein–protein interaction network of a cell ( 61–64 ). STRING collects gene expression evidence from RNA expression arrays and RNA-Seq datasets processed by the GEO database ( 65 ) as well as co-regulation evidence from the ProteomeHD database ( 7 ). In version 12.0 of STRING, the co-expression network is being extended with evidence from two novel sources: single-cell RNA-Seq data from the Human Protein Atlas ( 66 ) and proteomics datasets from the PRIDE database ( 67 ).” | concepts[6] |
| concept | text mining — not defined in this paper's own words; extraction of protein relations from literature — “The deep learning-based relation extraction text mining model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactions that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRING version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence score cut-offs in the text mining channel (Table 1 ).” | concepts[7] |
| concept | interologs — not defined in this paper's own words; associations transferred to orthologs of associated proteins — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | concepts[8] |
| concept | orthology — not defined in this paper's own words; the evolutionary correspondence of genes across species used for transfer — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | concepts[9] |
| concept | eggNOG — not defined in this paper's own words; the hierarchical orthology database STRING imports — “All protein–protein associations assembled for the STRING database are then transferred across organisms, where applicable, based on orthology relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologs’, ( 50 )). For this, hierarchical orthology relationships, at various levels of taxonomic resolution, are imported from the eggNOG database ( 51 ). After interolog transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRING allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRING via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).” | concepts[10] |
| concept | KEGG — not defined in this paper's own words; the pathway knowledgebase used as benchmark ground truth — “All protein–protein association evidence in STRING is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGG pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRING ‘confidence score’ for each association; confidence scores are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scores are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence score. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unit in STRING networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).” | concepts[11] |
| concept | variational autoencoders — not defined in this paper's own words; deep generative models compressing expression data into latent spaces — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | concepts[12] |
| concept | FAVA — not defined in this paper's own words; the autoencoder-based co-expression association method — “To address that, in the latest version we have utilized a novel method called FAVA (Functional Associations using Variational Autoencoders) ( 68 ) to build STRING’s co-expression network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencoders (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRING network. New methods together with the additional sources result in significant improvement in the performance of STRING version 12.0 over STRING version 11.5 co-expression network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRING interface now reports the associated fractional score and the source of each piece of evidence in the co-expression network.” | concepts[13] |
| concept | high-throughput — not defined in this paper's own words; experiments reporting at least 25 unique interactions — “For the experiments channel, STRING integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experiment’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughput (HT) experiment if at least 25 unique interactions are reported, otherwise as a low-throughput (LT) experiment.” | concepts[14] |
| concept | low-throughput — not defined in this paper's own words; experiments reporting fewer than 25 unique interactions — “For the experiments channel, STRING integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experiment’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughput (HT) experiment if at least 25 unique interactions are reported, otherwise as a low-throughput (LT) experiment.” | concepts[15] |
| concept | protein complex — not defined in this paper's own words; a physical assembly of proteins, one form of functional association — “The scope of protein–protein links in STRING is that of a ‘functional association’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complex or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGG ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.” | concepts[16] |
| concept | proteome — not defined in this paper's own words; the full set of proteins a user can now upload for annotation — “To improve this situation, STRING now allows its users to upload any fully sequenced proteome (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRING database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysis, bulk download, and REST API access. The outline of the proteome annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteome begins by choosing ‘Annotate your proteome’ on the STRING input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteome, as well as the taxonomic name of the species or clade that the uploaded proteome belongs to. Along with the protein sequences from the file, if provided, STRING will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRING automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteome is uploaded, STRING directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthology database eggNOG ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestor between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthology groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteome of an organism already included in the database, the proteins of the matching proteomes will then directly map to each other, without a need for mapping to any of the existing orthologous groups.” | concepts[17] |
| concept | gene set enrichment analysis — not defined in this paper's own words; statistical search for functionally enriched sets offered on networks — “To improve this situation, STRING now allows its users to upload any fully sequenced proteome (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRING database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysis, bulk download, and REST API access. The outline of the proteome annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteome begins by choosing ‘Annotate your proteome’ on the STRING input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteome, as well as the taxonomic name of the species or clade that the uploaded proteome belongs to. Along with the protein sequences from the file, if provided, STRING will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRING automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteome is uploaded, STRING directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthology database eggNOG ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestor between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthology groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteome of an organism already included in the database, the proteins of the matching proteomes will then directly map to each other, without a need for mapping to any of the existing orthologous groups.” | concepts[18] |
| concept | single-cell RNA-Seq — not defined in this paper's own words; a new expression source for the co-expression channel — “The degree of co-expression between RNAs (or between proteins) across different conditions provides an essential insight into the functional protein–protein interaction network of a cell ( 61–64 ). STRING collects gene expression evidence from RNA expression arrays and RNA-Seq datasets processed by the GEO database ( 65 ) as well as co-regulation evidence from the ProteomeHD database ( 7 ). In version 12.0 of STRING, the co-expression network is being extended with evidence from two novel sources: single-cell RNA-Seq data from the Human Protein Atlas ( 66 ) and proteomics datasets from the PRIDE database ( 67 ).” | concepts[19] |
| concept | RNA-Seq — not defined in this paper's own words; sequencing-based expression data sensitive to lowly expressed genes — “RNA-Seq datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughput nature of the experimental data reduces the inherent literature bias of the STRING network.” | concepts[20] |
| concept | Pearson Correlation Coefficient — not defined in this paper's own words; the correlation measure degraded by sparse redundant data — “The expression data tends to be sparse, high-dimensional and highly redundant. These attributes decrease the performance of correlations using Pearson Correlation Coefficient. For previous versions of STRING we have reduced the redundancy by removing highly correlated expression matrices before correlating the gene expression which, in turn, increased the recovery of the functional associations derived from these matrices. However, this did not fully address all of the challenges of such datasets.” | concepts[21] |
| concept | PSI-MI CV — not defined in this paper's own words; the controlled vocabulary of interaction detection methods — “For the benchmarking and scoring of LT experiments, STRING makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CV, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRING considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associations, while all other methods provide physical interaction evidence, which inherently is evidence for functional associations as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experiments tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRING directly translates the overall true-positive rate of each interaction detection method into the confidence score for that method. Each protein pair detected in an LT experiment by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experiments, one experimental functional association score for that pair is derived.” | concepts[22] |
| concept | last common ancestor — not defined in this paper's own words; the taxon bounding where orthologs are placed and pathways assigned — “The network prediction based on orthology is then performed as described previously ( 46 ). In parallel with the network prediction, STRING also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRING from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGG pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRING proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthology hierarchy; this is repeated until it is successful or until the root level is reached. STRING does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestor of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRING database, bulk download files are generated, and the user is given a unique STRING proteome identifier. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.” | concepts[23] |
Full text — tagged
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
The STRINGterm database in 2023: protein–protein association networks and functional enrichment analyses for any sequenced genome of interest Abstract Much of the complexity within cells arises from functional and regulatory interactions among proteins. The core of these interactions is increasingly known, but novel interactions continue to be discovered, and the information remains scattered across different database resources, experimental modalities and levels of mechanistic detail. The STRINGterm database ( https://string-db.org/ ) systematically collects and integrates protein–protein interactions—both physical interactionsterm as well as functional associationsterm. The data originate from a number of sources: automated text miningterm of the scientific literature, computational interaction predictions from co-expressionterm, conserved genomic contextterm, databases of interaction experimentsterm·def and known complexes/pathways from curated sources. All of these interactions are critically assessed, scored, and subsequently automatically transferred to less well-studied organisms using hierarchical orthologyterm information. The data can be accessed via the website, but also programmatically and via bulk downloads. The most recent developments in STRINGterm (version 12.0) are: (i) it is now possible to create, browse and analyze a full interaction network for any novel genome of interest, by submitting its complement of encoded proteins, (ii) the co-expressionterm channel now uses variational auto-encoders to predict interactions, and it covers two new sources, single-cell RNA-seqterm and experimental proteomics data and (iii) the confidence in each experimentally derived interaction is now estimated based on the detection method used, and communicated to the user in the web-interface. Furthermore, STRINGterm continues to enhance its facilities for functional enrichment analysis, which are now fully available also for user-submitted genomes.
INTRODUCTION A dense network of functional connections among proteins has evolved to support cellular life, forming a multitude of pathways, protein complexesterm and cellular structures ( 1 , 2 ). Recent developments have further improved our ability to unravel this connectivity, through techniques such as high-throughputterm genetic screens ( 3–5 ), systematic co-fractionation of proteomesterm ( 6 , 7 ), in-vivo proteome-wide cross-linking of proteins ( 8–10 ), and deep learning-based computational prediction frameworks such as AlphaFold ( 11 , 12 ). These efforts complement earlier results based on focused, small-scale laboratory studies, yeast two-hybrid screens, affinity purifications, co-crystallization and computational prediction algorithms (reviewed in ( 13 )). Together, the many approaches have begun to reveal a large part of the interaction landscape of cellular proteins, but they each have strengths and weaknesses, including potential biases, false negatives and noise. Furthermore, the resulting interaction data is scattered over a number of online resources, and available in varying namespaces and data-formats as well as varying levels of detail.
Given this, a number of meta-resources dedicated to data integration in the protein network context have been developed. These resources aim to collect and critically assess protein–protein functional linkage data, integrate it, connect it to previous knowledge, and allow users to browse, compare and retrieve organism-wide protein networks. Among frequently used and actively maintained frameworks are ConsensusPathDB ( 14 ), FunCoup ( 15 ), GeneMANIA ( 16 ), HumanBase ( 17 ), HumanNet ( 18 ), IID ( 19 ) and STRINGterm ( 20–22 ). Within this group, STRINGterm places its focus on comprehensiveness and ease of use—it covers >10 000 organisms, draws from a wide diversity of data sources including text miningterm and computational predictions, and offers many intuitive interface features including personalization, enrichment detection and programmatic access.
Researchers employ protein network meta-resources for a wide variety of purposes, broadly falling into three categories: (i) facilitating individual, small-scale molecular discoveries, (ii) facilitating large-scale data analysis and (iii) contributing to new methodologies and workflows. With respect to STRINGterm, it has, for example, proven useful in interpreting and reducing newly acquired genetic screening datasets—these are sometimes noisy and unwieldy, and STRINGterm can be used to distill such data into more manageable sets of observations and hypotheses. As a case in point, consider three recent screens for SARS-CoV-2 human host factors ( 23–25 ). All three studies used STRINGterm to interpret their initial, raw lists of screening hits—placing them into network contexts and searching them for functionally enriched processes/pathways. STRINGterm and its competitors are also often used as data providers in novel methodologies and computational resources, be it new databases, new algorithms, or community-wide competitions. As an example, consider two recent uses of STRINGterm networks in deep-learning frameworks ( 26 , 27 ). Both studies use deep learning to predict protein function (i.e. Gene Ontology terms), based on amino acid sequences and protein–protein interaction network topologies derived from STRINGterm.
Here, we provide an update on the current features of the STRINGterm database and describe some novel developments in more detail. The latter include a complete redesign of the co-expressionterm based interaction prediction pipeline, newly exposed sub-scores for experimental dataset confidence, as well as novel facilities allowing users to upload and analyze any newly sequenced genome of interest.
DATABASE CONTENT The scope of protein–protein links in STRINGterm is that of a ‘functional associationterm’ ( 28–30 )—proteins are considered to be associated when there is evidence suggesting an evolved, specific functional partnership between the two. This definition includes proteins that are physically associated to each other in a protein complexterm or in a transient interaction, but also proteins that are more indirectly associated: they may work towards a common goal in a metabolic or signaling pathway, may regulate each other through intermediaries, or may jointly contribute to a common cellular structure. The granularity of what constitutes a ‘common function’ is not formally defined; it should not be understood too broadly, however, and operationally it roughly corresponds to the specificity of pathways ‘maps’ or ‘diagrams’ in knowledgebases such as KEGGterm ( 31 ) or Reactome ( 32 ). It should be noted that the definition of a functional assocation can include proteins that act antagonistically to each other, albeit in the same overall pathway.
All protein–protein association evidence in STRINGterm is assessed and quantified, and its correspondence to the above definition is benchmarked against common membership in KEGGterm pathway maps (excluding maps that are largely based on homology, such as ‘ABC transporters’). The result of this benchmarking/calibration is a STRINGterm ‘confidence scoreterm’ for each association; confidence scoresterm are scaled between zero and one, and correspond to the estimated likelihood of a given association being true, given the underlying evidence. Confidence scoresterm are first computed separately per evidence type (see (33 ) for an example), and then integrated into a final, ‘combined’ confidence scoreterm. All evidence collected for a given protein pair contributes to the score, irrespective of the exact nature of these proteins in terms of alternative splicing isoforms or post-translational modifications; correspondingly, the interacting unitterm·def in STRINGterm networks is the entire protein-coding locus, represented by its most canonical protein product ( 34 ).
All confidence scoresterm in STRINGterm are pre-computed and freely available for download, under a Creative Commons Attribution license (CC BY 4.0). The various evidence types are grouped into seven distinct ‘evidence channelsterm’, with separate sub-scores available for each channel. These channels can also be individually viewed on the STRINGterm user interface, together with their underlying evidence, and can be enabled or disabled separately as desired. Of the seven channels, the first three (‘neighborhood’, ‘fusion’ and ‘co-occurrence’) are dealing with association evidence that can be gleaned from genome sequences alone. These so-called ‘genomic contextterm’ channels (reviewed in ( 35 , 36 )) are based on detecting evolutionary constraints arising from functional gene-gene partnerships, and are best applicable in prokaryotic genomes. Another channel (‘co-expressionterm’) is dealing with functional genomics measurements (transcripts or proteins) across a multitude of conditions, searching for evidence of common expression regulation (see also below). Next, the ‘experimentsterm·def’ channel deals with laboratory experimentsterm·def that were conducted with the expressed goal to uncover protein–protein association evidence. They are imported from primary database repositories: BioGRID ( 37 ), DIP ( 38 ), PDB ( 39 ), as well as IntAct and its partner databases in the IMEx consortium ( 40 ). The final two evidence channelsterm are concerned with protein–protein associations that are already known. The ‘database’ channel imports well-established knowledge (‘textbook knowledge’) about protein complexesterm, pathways and other functional connections from dedicated knowledge resources: KEGGterm ( 31 ), Reactome ( 32 ), MetaCyc ( 41 ), EBI Complex Portal ( 42 ), and Gene Ontology Complexes ( 43 ). Lastly, the ‘textmining’ channel is the result of parsing full-text articles from the PMC Open Access Subset (up to April 2022), PubMed abstracts (up to August 2022), as well as summary texts from OMIM ( 44 ) and SGD ( 45 ) entry descriptions. These texts are all parsed for co-mentions of protein pairs and assessed against the frequencies of all separate mentions of the respective proteins, as described in ( 46 ). On top of functional associationsterm, the three latter channels (i.e. experimentsterm·def, database and textmining) provide also the interactions for the physical sub-network of STRINGterm. The calculation of the confidence scoresterm for the protein pairs in this network differs from that of the functional associationterm network and is described in detail in ( 22 ).
The deep learning-based relation extraction text miningterm model has received significant upgrades in the current version. Specifically, the language representation model that we use has changed from BioBERT ( 47 ) to a biomedical RoBERTa-large model ( 48 ), which has already shown better performance in relation extraction tasks ( 49 ). Moreover, the model can now detect physical interactionsterm that span across the boundaries of a single sentence. This is made possible mainly due to the two orders of magnitude increase of manually annotated relations in the training set (from 6145 to 243 831), which led to the addition of many cross-sentence pairs during training, thus allowing the model to learn to detect such associations. These changes in the deep learning model, in combination with the increase in the size of the literature corpus compared to the previous STRINGterm version, have led to a two-fold increase in the number of physical protein–protein interactions above the low, medium, and high confidence scoreterm cut-offs in the text miningterm channel (Table 1 ).
Counts and relative frequencies of physically interacting protein pairs obtained via text miningterm. Aggregating across all organisms in STRINGterm, the table shows counts for various frequently used score cutoff levels, for both STRINGterm version 11.5 and STRINGterm version 12.0. The lowest score cut-off has been used to determine what constitutes 100% for each dataset All protein–protein associations assembled for the STRINGterm database are then transferred across organisms, where applicable, based on orthologyterm relationships with the assumption that orthologs of associated proteins are likewise associated (‘interologsterm’, ( 50 )). For this, hierarchical orthologyterm relationships, at various levels of taxonomic resolution, are imported from the eggNOGterm database ( 51 ). After interologterm transfer and the final combined score integration, the resulting protein association networks can be accessed in a number of ways. Firstly, the interactive website of STRINGterm allows browsing and searching, including evidence inspection via dedicated viewers. Users can also submit larger queries there, allowing for the construction of dedicated networks and for statistical searches for functional enrichments. Apart from the website, scientists can access STRINGterm via a dedicated Cytoscape plugin ( 52 ), as well as via a Bioconductor package and a dedicated application programming interface (REST).
USER-UPLOADED GENOMES One of the unique features of STRINGterm has always been its support for a large selection of non-model organisms: the current version of the database contains protein–protein interaction networks (and protein functional pathway annotations) for >10 000 distinct genomes. When selecting which genomes to include in a STRINGterm update, key factors are the organism's research prominence, genome quality, and completeness. Subsequently, STRINGterm utilizes genomes from authoritative sources only, including Ensembl ( 53 ), UniProtKB Reference Proteomesterm ( 54 ) and the ‘representative genomes’ set in the proGenomes database ( 55 ).
However, new genomes are sequenced and assembled daily, and existing genomes are re-sequenced or re-annotated; overall, the number of taxonomically distinct species that are being sequenced has been doubling roughly every 3 years ( 56 ). The UniProtKB database is updated with the new proteomesterm on an eight-week cycle, and projects like Ensembl Rapid Release ( 53 ) aim to provide annotation for the newly released genomes on a 2-week cycle. For STRINGterm, such frequent update cycles would require heavy resources and may create complexities with data reproducibility/data archiving when the new updates supersede older datasets. On the other hand, a slow update cycle implies that the database will not incorporate newly sequenced genomes or any improvements to the gene sets of existing genomes.
To improve this situation, STRINGterm now allows its users to upload any fully sequenced proteometerm (including those that are already part of the database), enabling them to browse and query the predicted protein–protein interactions in an identical manner to the genomes already natively covered by the STRINGterm database. This includes access to the evidence viewers, homology viewer, network clustering methods, gene set enrichment analysisterm, bulk download, and REST API access. The outline of the proteometerm annotation pipeline is shown in Figure 1 . The procedure for uploading a new proteometerm begins by choosing ‘Annotate your proteometerm’ on the STRINGterm input page, after which the user is guided through a few simple steps of the process. All that is required for the submission is a simple FASTA-formatted proteometerm, as well as the taxonomic name of the species or clade that the uploaded proteometerm belongs to. Along with the protein sequences from the file, if provided, STRINGterm will extract from the FASTA definition lines (headers) any standard gene names, identifiers, and free-text protein descriptions; these will later be searchable from the input page and used throughout the webpage. For this, STRINGterm automatically recognizes several formats of FASTA headers including those from RefSeq ( 57 ), UniProtKB ( 54 ) and Ensembl ( 53 ), and checks for any apparent errors such as duplicate sequence identifiers. After the proteometerm is uploaded, STRINGterm directly aligns the sequences to all sequences in its database using DIAMOND ( 58 ) (with the --iterate option). Each protein is then assigned to its orthologs via the hierarchical orthologyterm database eggNOGterm ( 51 ) based on its highest scoring alignment (best hit). The taxonomic level at which the protein is placed in the group hierarchy is defined by the last common ancestorterm between the user-specified taxon and the taxon of the best-scoring hit. If a protein cannot be placed via the hierarchical orthologyterm groups, it is considered a direct one-to-one ortholog with its best-scoring hit. As the user can submit a proteometerm of an organism already included in the database, the proteins of the matching proteomesterm will then directly map to each other, without a need for mapping to any of the existing orthologous groups.
Extending STRINGterm with user-submitted genomes. Submitted genomes are first searched against the existing STRINGterm genomes, and orthologyterm is used to transfer all relevant information (interactions, annotations) from closely related organisms. The submitted genomes then become available on the web interface, via the programmatic Application Programming Interface (API), and for bulk downloads.
The network prediction based on orthologyterm is then performed as described previously ( 46 ). In parallel with the network prediction, STRINGterm also attempts to assign the submitted proteins to their corresponding pathways and functional subsystems, as imported for STRINGterm from the three Gene Ontology branches (Biological process, Molecular Function and Cellular Component) ( 43 ), KEGGterm pathways ( 31 ), UniProtKB Keywords ( 54 ), COMPARTMENTS ( 59 ) and TISSUES ( 60 ). This assignment is done based on the consensus of the pathway annotations of the pre-existing STRINGterm proteins in the most specific orthologous group that a new protein is assigned to. If these pre-existing proteins carry no pathway annotation, the consensus is attempted in the next-higher, parental orthologous group in the orthologyterm hierarchy; this is repeated until it is successful or until the root level is reached. STRINGterm does not assign pathways or functional subsystems outside their previously known taxonomic scope. A pathway's scope is defined by the last common ancestorterm of all organisms that have in the past been annotated with this pathway. At the end of the computation process, all predictions are automatically uploaded to the internal STRINGterm database, bulk download files are generated, and the user is given a unique STRING proteome identifierterm·def. This identifier functions as an organism name on the input page, making the newly submitted genome browsable and searchable. Submitters can share this identifier with other users, but if they choose not to, the submitted genomes remain private.
IMPROVED CO-EXPRESSIONterm ANALYSIS The degree of co-expressionterm between RNAs (or between proteins) across different conditions provides an essential insight into the functional protein–protein interaction network of a cell ( 61–64 ). STRINGterm collects gene expression evidence from RNA expression arrays and RNA-Seqterm datasets processed by the GEO database ( 65 ) as well as co-regulation evidence from the ProteomeHD database ( 7 ). In version 12.0 of STRINGterm, the co-expressionterm network is being extended with evidence from two novel sources: single-cell RNA-Seqterm data from the Human Protein Atlas ( 66 ) and proteomics datasets from the PRIDE database ( 67 ).
The expression data tends to be sparse, high-dimensional and highly redundant. These attributes decrease the performance of correlations using Pearson Correlation Coefficientterm. For previous versions of STRINGterm we have reduced the redundancy by removing highly correlated expression matrices before correlating the gene expression which, in turn, increased the recovery of the functional associationsterm derived from these matrices. However, this did not fully address all of the challenges of such datasets.
To address that, in the latest version we have utilized a novel method called FAVAterm (Functional Associationsterm using Variational Autoencodersterm) ( 68 ) to build STRINGterm’s co-expressionterm network. This deep-learning model reduces the dimensionality of the data into lower-dimensional latent spaces using variational autoencodersterm (VAE). The benefit of encoding the matrices into fewer dimensions is 2-fold: it reduces the overall sparsity of the data and limits redundancy by essentially compressing the data. The predictions obtained from all the sources are combined in a probabilistic fashion ( 46 ) and the resulting scores are recalibrated to negate the effect that the non-independence of the sources has on the STRINGterm network. New methods together with the additional sources result in significant improvement in the performance of STRINGterm version 12.0 over STRINGterm version 11.5 co-expressionterm network (Figure 2A ). The performance of each individual source contributing to the combined network is shown in Figure 2B . In addition, the STRINGterm interface now reports the associated fractional score and the source of each piece of evidence in the co-expressionterm network.
Improved interaction prediction based on co-expressionterm. Interaction scores based on co-expressionterm have been ranked and benchmarked against common KEGGterm pathway membership as ground truth. ( A ) Performance comparison of co-expressionterm network between STRINGterm version 11.5 and STRINGterm version 12.0. ( B ) Overview of the performance of all expression datasets contributing to the STRINGterm version 12.0 co-expressionterm channel.
RNA-Seqterm datasets have increased sensitivity for genes exhibiting low levels of expression ( 69 , 70 ). This combined with improved performance and the high-throughputterm nature of the experimental data reduces the inherent literature bias of the STRINGterm network.
EXPERIMENT-LEVEL CONFIDENCE SCORING For the experimentsterm·def channel, STRINGterm integrates pairwise experimental interaction evidence from BioGRID, IntAct, MINT, and others (( 37–40 ), see Figure 3 ). Each reported protein pair supported by a specific publication is considered an independent piece of evidence and scored individually. During import, duplicate records from different sources are removed. Each ‘experimentterm·def’, defined as all protein–protein pairs supported by a common PubMed identifier and a given detection method, is classified as a high-throughputterm (HT) experimentterm·def if at least 25 unique interactions are reported, otherwise as a low-throughputterm (LT) experimentterm·def.
Processing and scoring of experimental interaction evidence. Experimental evidence is retrieved from several public databases. Protein pairs from low-throughputterm (LT) experimentsterm·def are grouped by detection method and pairs coming from high-throughputterm (HT) experimentsterm·def are grouped by experimentterm·def. Within each group, pairs are benchmarked against the KEGGterm pathway database to assess the confidence of identifying functional associationsterm. All LT pairs are assigned the benchmark score derived for the particular detection method. HT pairs are scored based on calibration on the experimentterm·def level.
For interactions detected by HT experimentsterm·def, benchmarking against KEGGterm molecular pathway maps and scoring is performed separately for each experimentterm·def: each pair is assigned a raw score based on the number of shared and non-shared interactors for both proteins within this experimentterm·def. The more shared and the fewer non-shared interactors, the higher the raw quality scoreterm·var of the protein pair from this experimentterm·def. Based on the raw score, all pairs detected in the experimentterm·def are ranked and benchmarked against KEGGterm. A typical STRINGterm calibration function is derived for each HT experimentterm·def, which is then used to assign a functional associationterm score to each protein pair identified in the experimentterm·def ( 33 ).
For the benchmarking and scoring of LT experimentsterm·def, STRINGterm makes use of the Molecular Interaction Controlled Vocabulary (PSI-MI CVterm, https://www.ebi.ac.uk/ols/ontologies/mi , ( 71 )). Specifically, STRINGterm considers the methods annotated as ‘experimental interaction detection’ (MI:0045) and its children but excludes those that are inferred by the author/curator (MI:0363 and MI:0364) or predicted (MI:0063). Among the included methods, evidence generated by ‘genetic interference’ methods (MI:0254) is considered as strictly functional associationsterm, while all other methods provide physical interactionterm evidence, which inherently is evidence for functional associationsterm as well. Protein pairs are grouped by experimental interaction detection method, to obtain groups that are large enough for benchmarking. Because protein pairs detected by LT experimentsterm·def tend to focus on specific pathways or known proteins, no curve-based score calibration is done. Instead, STRINGterm directly translates the overall true-positive rateterm·var of each interaction detection method into the confidence scoreterm for that method. Each protein pair detected in an LT experimentterm·def by this method is then assigned the corresponding method's score. By aggregating all scores for a particular protein pair across experimentsterm·def, one experimental functional associationterm score for that pair is derived.
The highest number of experimentally detected associations has been derived by affinity chromatography technologies (40%), followed by genetic inference (34%) and transcriptional complementation assays (11%). Out of these, 82% came from HT experimentsterm·def. The associations derived from these three HT experimentsterm·def, on average, place below or around a confidence scoreterm of 0.25 (Figure 4A ), while, in comparison, the LT experimentsterm·def for the same methods score more than twice as high with a medium confidence scoreterm of around 0.6 (Figure 4B ). The top three methods by confidence scoreterm for LT-derived associations are assays determining the 3D structure of protein complexesterm, which not surprisingly are excellent predictors of functional and physical protein–protein interactions.
Reliability estimates for protein–protein interaction detection assays. The top three most prolific experimental interaction assay types are shown, ranked by the number of protein pairs to which they contribute in STRINGterm. Benchmarked on KEGGterm pathways, they yield (on average) lower confidence scoresterm when derived from the high-throughputterm assays ( A ), in contrast, for the equivalent low-throughputterm experimentsterm·def (shown in color), the predicted confidence is substantially higher ( B ). The distributions shown in (A) encompass all HT interactions of a given assay type (for simplicity); in the actual scoring computations in STRINGterm, each HT interaction datasets is scored separately.
In STRINGterm’s ‘experimental’ evidence viewer, the user can now better appreciate the reliability of each experimental prediction, as the individual confidence for every HT and LT dataset is now communicated to the user in a form of three-tier confidence grade (‘high’, ‘medium’ and ‘exploratory’).
OUTLOOK Version 12.0 of STRINGterm covers a phylogenetically diverse collection of 12 535 high-quality genomes. Beyond these, the system will record which genomes are frequently submitted by users—and these will then be prioritized for inclusion into subsequent releases. In addition, the results of an ongoing online user survey will be analyzed (350 users have already participated). This way, STRINGterm will keep concentrating its resources on those areas that are of most interest to its users.
DATA AVAILABILITY STRINGterm is freely available, under a Creative Commons Attribution license (CC BY 4.0).