Deep-learning microscopy image reconstruction with quality control reveals second-scale rearrangements in RNA polymerase II clusters — TEI-tagged view

Hamideh HajiabadipersName, Irina MamontovapersName, Roshan PrizakpersName, Agnieszka PancholipersName, Anne KoziolekpersName, Lennart HilbertpersName (2022date). “Deep-learning microscopy image reconstruction with quality control reveals second-scale rearrangements in RNA polymerase II clusters.” PNAS Nexustitle·j.

Object of study — quality controlterm·key of deep-learning (Noise2Voidterm·ai) reconstruction for fast fluorescence microscopyterm·key

Research question — how to use neural-network denoisingterm·ai while guarding against its reconstruction errors

Contribution — a two-phase acquisition work-flow with per-image quality controlterm·key, tripling acquisition speed

Full text — tagged

Deep-learning microscopy image reconstruction with quality controlterm·key reveals second-scale rearrangements in RNA polymerase IIterm·key clusters Abstract Fluorescence microscopyterm·key, a central tool of biological research, is subject to inherent trade-offs in experimentterm·mth design. For instance, image acquisition speed can only be increased in exchange for a lowered signal quality, or for an increased rate of photo-damage to the specimen. Computational denoisingterm·ai can recover some loss of signal, extending the trade-off margin for high-speed imaging. Recently proposed denoisingterm·ai on the basis of neural networks shows exceptional performance but raises concerns of errors typical of neural networks. Here, we present a work-flow that supports an empirically optimized reduction of exposure timesterm·mth, as well as per-image quality controlterm·key to exclude images with reconstruction errors. We implement this work-flow on the basis of the denoisingterm·ai tool Noise2Voidterm·ai and assess the molecular state and 3D shape of RNA polymerase IIterm·key (Pol II) clusters in live zebrafishterm·key embryos. Image acquisition speed could be tripled, achieving 2-s time resolution and 350-nm lateral image resolution. The obtained data reveal stereotyped events of approximately 10 s duration: initially, the molecular mark for recruited Pol II increases, then the mark for active Pol II increases, and finally Pol II clustersterm·key take on a stretched and unfolded shape. An independent analysis based on fixed sample images reproduces this sequence of events, and suggests that they are related to the transient association of genes with Pol II clustersterm·key. Our work-flow consists of procedures that can be implemented on commercial fluorescence microscopes without any hardware or software modification, and should, therefore, be transferable to many other applications.

Keywords: image processing, reliable deep learning, fluorescence microscopyterm·key, gene regulation, transcription Light microscopy is subject to unavoidable performance trade-offs. For observation of live biological samples, image acquisition speed can only be increased in exchange for a lowered signal quality, or an increased rate of photo-damage. These limitations can be partially compensated for by denoisingterm·ai after acquisition. Denoisingterm·ai based on deep learning performs especially well, but denoisingterm·ai errors have caused concern. We present a pragmatic work-flow that enables quality controlterm·key of denoised images. We illustrate the applicability of this work-flow by assessing RNA polymerase IIterm·key clusters in live zebrafishterm·key embryos, revealing coordinated changes in the molecular state and cluster shape. Our observations point toward the activation of genes over the course of 10 s during which they visit a polymerase cluster.

Introduction Light microscopy is one of the most central tools of biological research, whether a biologist aims to get the first glimpse of a given cellular process or to quantitatively test the validity of hypotheses ( 1 ). A specific area of application is the visualization of fluorescently labeled molecules. The design of such experimentsterm·mth is subject to inherent limitations ( 2 , 3 ), requiring a trade-off between acquisition speed, signal-to-noise ratioterm·mth (SNRterm·mth), and prevention of photo-damage to the specimen ( 4 ). These parameters cannot be optimized separately. For instance, to increase acquisition speed, exposure timeterm·mth must be reduced, leading to lower SNRterm·mth ( 5 , 6 ). SNRterm·mth can be recovered by, for example, increased power of the light used to excite fluorescence in the sample, resulting however in increased photo-damage.

While the experimental parameters during acquisition are subject to firm trade-off relationships, computational processing of images after acquisition can recover image quality. These approaches allow, for example, a further reduction of exposure timesterm·mth followed by computational reconstruction of low-SNR images. Conventional approaches for reconstruction of low-SNR images include projection methods ( 7 ), deconvolution filters ( 8 , 9 ), and denoisingterm·ai methods ( 10 , 11 ). In the past decade, deep-learning methods have become widely used in a variety of image processing applications, often outperforming conventional approaches ( 12 ). In biological microscopy, deep learning has been successfully used for image classification ( 13–15 ), segmentation ( 16 , 17 ), and restoration ( 18–21 ). Initial deep learning approaches used standard deep networks to restore fluorescence microscopyterm·key images, requiring training data sets of matched low-quality and high-quality images. For example, networks can be trained on a reference data set with high SNRterm·mth (“ground truth”), so as to restore matched images with low SNRterm·mth (“noisy data”) ( 22 ). One obstacle to the wide-spread application of such reconstruction approaches is the requirement for matched high-quality training data ( 23 , 24 ). These data are laborious or sometimes even impossible to obtain in a fashion that is sufficiently matched to noisy data. An alternative is provided by Noise2Noise (n2n) techniques, which enable the training of deep networks from matched pairs of noisy images ( 25 , 26 ). The requirement for any matched images is fully removed in the Noise2Voidterm·ai (n2vterm·ai) technique, where learning and removal of noise are carried out based on a single noisy image data set ( 26 , 27 ). Reconstruction based on a single noisy data set also allows per-image training, thus compensating for day-to-day variability of, for example, fluorescence labeling or fine-adjustment of optical parts.

A second obstacle to the wide application of deep learning methods is the possibility of errors in the reconstructed fluorescence images ( 23 , 24 ). These errors manifest as deviations between the high-quality ground truth images and the images reconstructed from low SNRterm·mth data. A dilemma arises, where the effective application of deep learning networks can only proceed without acquisition of ground truth data, but ground truth data are required to assure the experimenter that reconstruction is error-free. In this work, we develop a pragmatic work-flow for the quality-controlled adjustment and application of n2vterm·ai for denoisingterm·ai in high-speed fluorescence microscopyterm·key. In this work-flow, for every acquired view of a given sample, a small data set with high-quality data is recorded to control reconstruction quality, followed by full time-lapse acquisition of only compromised data. We demonstrate the applicability of this work-flow in the analysis of fluctuations in molecular clusters in live zebrafishterm·key embryos. Our analysis reveals a close coordination between post-translational modifications of RNA polymerase IIterm·key (Pol II) and changes in the 3D shape of these clusters on the scale of a few seconds. These observations are confirmed by an alternative experimental approach, where still images from chemically fixed cells are sorted based on an additional fluorescence marker for genes that transiently engage with the molecular clusters. Our approach provides a guideline for other microscopists interested in the quality-controlled application of ground-truth-free image reconstruction methods. The approach can be implemented on fluorescence microscopes with typical specifications used for time-lapse recordings without the need of software development or hardware control beyond the standard functionality.

Results Quantification of image reliabilityterm·mth and effective resolutionterm·mth in reconstructed microscopy images The structural reliabilityterm·mth and effective spatial resolution of reconstructed images can be assessed by a combination of widely used metrics. The structural reliabilityterm·mth can be assessed via the structural similarity index metricterm·mth (SSIMterm·mth). SSIMterm·mth quantifies the similarity between two images and returns a value between 0 and 1 ( 28 , 29 ). SSIMterm·mth values close to 1 indicate that two images are very similar, lower SSIMterm·mth values indicate images that are less similar. One application is the comparison of two images obtained with the same acquisition and postprocessing steps, providing a quantification of the reliability of the obtained images. Using SSIMterm·mth, we can, for example, demonstrate how changes in image acquisition settings, such as the reduction of exposure timeterm·mth, can compromise image reliabilityterm·mth ( Fig. S1, Supplementary Material ). Applying n2vterm·ai to pairs of super-resolution microscopy images, we can illustrate how denoisingterm·ai can increase image reliabilityterm·mth (Fig. 1A and B ). SSIMterm·mth can also be used to assess whether reconstructions of low-quality images obtained with, for example, low exposure timesterm·mth can approximate high-quality images (Fig. 1A and B ). The assessment of image reliabilityterm·mth via SSIMterm·mth is, however, not sensitive to localized differences between images, as are typically introduced at edges during denoisingterm·ai procedures. Such local occurrences of unreliable reconstruction are readily detected by the local SSIMterm·mth ( Fig. S2, Supplementary Material ) ( 30 ). The combination of SSIMterm·mth and local SSIMterm·mth, thus, allows an assessment of image reliabilityterm·mth based on paired images, as well as the similarity between a reconstructed and a corresponding high-quality image.

Metrics for the reliability and effective resolutionterm·mth in n2v-reconstructed images. (A) Representative micrographs of the DNA distribution in a nucleus in a fixed zebrafishterm·key embryo, recorded with a stimulated emission depletion (STED) super-resolution microscope. The same image plane was recorded twice at low quality, once at high quality, and two n2v-reconstructed images were prepared from the low-quality images. (B) SSIMterm·mth values for pair-wise comparison (image 1 vs. image 2) and comparison against the high-quality image (image 1 vs. high-quality and image 2 vs. high-quality) for the low-quality images and the reconstructed images. (C) FRCterm·mth curves calculated based on a low-quality image pair and the corresponding reconstructed image pair. (D) FRC-based effective resolutionterm·mth for four pairs of low-quality images and the corresponding pairs of reconstructed images.

A key aspect of performance in microscopy is the effective image resolution. The effective image resolution is determined by both the optical resolution of a given imaging instrument, and by the ratio of photons emitted by the structure of interest over polluting photons, often referred to as SNRterm·mth. This effective resolutionterm·mth can be quantified via Fourier ring correlationterm·mth (FRCterm·mth) ( 31 , 32 ). FRCterm·mth evaluates the similarity of a pair of images in frequency space, so as to determine the spatial frequency up to which the images are consistent with each other (Fig. 1C ). The inverse of this spatial frequency is then taken as the effective spatial resolution (Fig. 1C ). Applying the FRCterm·mth metric to our super-resolution microscopy data reveals that, indeed, n2v-denoising can recover effective resolutionterm·mth in low-quality images (Fig. 1D ; Fig. S3, Supplementary Material ). Taken together, SSIMterm·mth and FRCterm·mth can objectively assess image reliabilityterm·mth and effective resolutionterm·mth in matched pairs of reconstructed images.

Optimization of exposure timeterm·mth for high-speed time-lapse imaging While denoisingterm·ai with n2vterm·ai can, in principle, reconstruct images acquired with reduced exposure timeterm·mth ( t exp ), for a given experimentterm·mth it is not known a priori just how far t exp can be reduced while ensuring a sufficient image reconstruction. To demonstrate how SSIMterm·mth and FRCterm·mth can guide the choice of t exp , we carried out live sample microscopy of cells obtained from buccal smears (“human cheek cells”) for a range of exposure timesterm·mth, (Fig. 2A ). For each t exp , a n2v-network was separately trained on a pair of images and the effective resolutionterm·mth for these reconstructed images was assessed (Fig. 2B ). For or higher, an effective resolutionterm·mth of was attained for the reconstructed images (Fig. 2C ). This resolution was not further improved by longer exposure timesterm·mth, but could not be attained for shorter exposure timesterm·mth (Fig. 2C ). This FRC-based assessment suggests as an optimal exposure timeterm·mth. We controlled the structural reliabilityterm·mth of the reconstructed images by local SSIMterm·mth, finding reconstruction errors for ( Fig. S2A–G, Supplementary Material ). Considering both the FRCterm·mth and local SSIMterm·mth results, all seem structurally reliable, while only allow maximal effective image resolution after image reconstruction. In this setting, the experimenter can, therefore, choose between faster acquisition ( ) or higher effective resolutionterm·mth ( ), all while ensuring a high certainty of structural reliabilityterm·mth.

Metric-based estimation of how far image quality can be compromised while still allowing recovery of effective resolutionterm·mth by denoisingterm·ai. (A) Representative micrographs of nuclei of human cheek cells for different camera exposure timesterm·mth ( t exp , as indicated), all high-quality images were acquired at the same position but with an exposure timeterm·mth of 200 ms. Images are maximum-intensity projections, DNA was labeled by Hoechst 33342. (B) FRCterm·mth curves calculated from a pair of matched low-quality images, from a pair of reconstructed images, and a pair of high-quality images for the different t exp . (C) Effective resolutionterm·mth for the indicated t exp , n = 5 nuclei per t exp , values are shown with mean.

A two-phase acquisition protocol for quality-controlled denoisingterm·ai of time-lapse recordings To integrate the metric-based assessment of n2v-processed images with the recording of high-speed time-lapse data, we propose an acquisition protocol that contains two distinct phases and is carried out at every position in a given sample (Fig. 3A ). In the first phase (A, assessment), all image data required for the application of SSIMterm·mth and FRCterm·mth metrics are recorded (Fig. 3B ). In particular, for each of the image planes that make up the acquired 3D volume, the following images are obtained: one low-quality image ( t exp ), two high-quality images recorded with the longer reference exposure timeterm·mth ( t ref ), followed by two more low-quality test images ( t exp ). In the second phase (B, time-lapse), a sequence of 3D volumes is acquired with only a single low-quality image for each of the image planes, reducing the time spent for the acquisition of a 3D volume (Fig. 3B ).

A two-phase acquisition protocol to combine acquisition of quality controlterm·key images with high-speed time-lapse imaging. (A) Image data were acquired at multiple positions in a sample, thus obtaining multiple viewpoints containing several objects of interest (nuclei, indicated as circles). (B) For each position, a sequence of two acquisition phases is carried out. In phase A, for each z position, a low-quality image, two high-quality reference images, and two low-quality test images are recorded. Low-quality images are recorded at a shortened exposure timeterm·mth ( t exp ), high-quality images at a reference exposure timeterm·mth resulting in images of the desired quality ( t ref ). Acquisition phase A obtains the images required for n2vterm·ai model training as well as the assessment of effective image resolution and reconstruction errors. In phase B, only single low-quality images are recorded with the shortened exposure timeterm·mth ( t exp ), resulting in an increased rate of acquisition compared to acquisition with full exposure timeterm·mth ( t ref ). Acquisition phase B obtains only low-quality images, which are reconstructed after the experimentterm·mth is completed.

The data acquired by this two-phase acquisition protocol allow a comprehensive quality controlterm·key assessment for every recorded position. Specifically, we first train a n2v-network for each position, with which we reconstruct the low-quality test images 1 and 2. We can then assess the effective resolutionterm·mth using the FRCterm·mth metric and additionally control the reconstructed image for reconstruction errors using SSIMterm·mth and local SSIMterm·mth. For positions where a sufficient effective resolutionterm·mth is achieved by the reconstruction, and a sufficiently low level of reconstruction error is found, the trained n2v-network is then applied to the time-lapse data from phase B, thus providing n2v-reconstructed time-lapses with per-position quality controlterm·key.

High-speed imaging reveals coordinated changes of phosphorylation and shape of Pol II clustersterm·key To demonstrate the applicability of our proposed protocol for quality-controlled n2v-supported live imaging protocol, we attempted to visualize changes in the molecular state as well as the 3D shape of macromolecular clusters enriched in Pol II. To this end, we recorded microscopy images from live zebrafishterm·key embryos with an instant-SIM microscope ( 33 ). We visualized Pol II that is recruited to macromolecular clusters (Pol II Ser5P) or has transitioned toward production of RNA transcripts (Pol II Ser2P) by fluorescently labeled antibody fragments (Fabs). These Fabs have been validated to specifically and reliably detect changes in the Pol II Ser5P and Pol II Ser2P levels in zebrafishterm·key embryos, and do not perturb embryonic development in any obvious fashion ( 34–36 ). To establish exposure timesterm·mth, we first adjusted imaging parameters so as to obtain images that reveal cluster shape in the Pol II Ser5P channel on the microscope’s live display without any processing. We chose this reference exposure timeterm·mth as , resulting in an overall time of that is required to obtain a full 3D image stack. Using , we recorded image data in line with the two-phase acquisition protocol, with the phase B spanning a total time of 2 min. Specifically, we recorded data for four different exposure timesterm·mth ( ; Fig. S4A–C and Table S1, Supplementary Material ). For all t exp , we achieve an effective resolutionterm·mth of 400 nm (lateral) or better after n2v-based reconstruction, which compares favorably to an effective resolutionterm·mth of approximately 700 nm in the high-quality images ( Fig. S4D, Supplementary Material ). A comparison against conventional, state-of-the-art 3D deconvolution also indicates a stronger improvement of effective image resolution by n2vterm·ai ( Fig. S5, Supplementary Material ). Analysis by local SSIMterm·mth suggests that reconstructions for offer a reliability similar to a comparison between two high-quality images, reconstructions of images obtained with are prone to reconstruction errors ( Fig. S4E, Supplementary Material ). Accordingly, we selected images acquired with (effective lateral resolution ) and (effective lateral resolution ) for further analysis, which provided full 3D image stacks at a time resolution of and , respectively.

As previously observed, clusters seen in the Pol II Ser5P channel were persistent during the entire phase B acquisition period ( 36 ). The n2v-processed Pol II Ser5P time-lapse images were segmented to detect Pol II-enriched clusters, each cluster was then tracked over the whole time-lapse based on spatial proximity in consecutive time points (Fig. 4A ). Based on the Pol II Ser5P-derived segmentation masks, Pol II Ser5P and Ser2P intensities as well as shape quantifiers could be determined for each time point (Fig. 4B ). We used the two shape quantifiers elongationterm·mth, which indicates how much an object is stretched along its main axis, and solidityterm·mth, which indicates how convex (“rounded out”) the outline of a given shape is. The resulting time courses exhibit fluctuations, and the question arises whether a systematic relationship exists between the different quantities (Fig. 4C ). Indeed, a cross-correlation analysis that was anchored on cluster elongationterm·mth suggests a systematic relationship (Fig. 4D ). The cross-correlation analysis reveals an initial increase in Pol II Ser5P intensity, followed by a transient increase in Pol II Ser2P intensity ∼5 s later, and a transient decrease in Pol II Ser5P intensity another ∼5 s later. These changes are accompanied by an initial rounding up of clusters (solidityterm·mth increase), followed by transient unfolding (solidityterm·mth decrease) ∼10 s later. These cross-correlation analysis results are obtained at both (Fig. 4 ) and ( Fig. S6, Supplementary Material ), indicating that our findings are not mere coincidence. Our observations are representative of a stereotypical sequence of events, which occurs repeatedly and is, therefore, detected by the cross-correlation analysis: Pol II Ser5P intensity increases and the cluster rounds up via the rapid recruitment of Pol II to a given cluster, Pol II Ser5P intensity decreases and Pol II Ser2P intensity increases as some of the recruited Pol II proceeds into transcript production, while the cluster gets elongated and unfolded (Fig. 4E ). Previous work indicates that transcribing Pol II and the resulting nascent RNA transcripts induce distinct rearrangements in molecular clusters, providing a potential cause for the elongationterm·mth and unfolding ( 35–38 ). Notably, these works suggest that changes in Pol II state and cluster organization come about due to transient engagement of genes with Pol II-enriched clusters, leading to the induction of genes and to their release from these clusters.

n2v-accelerated imaging reveals coordinated changes in shape and phosphorylation levels of Pol II clustersterm·key on the scale of seconds. (A) Representative series of time-lapse images showing a single Pol II clusterterm·key in the Pol II Ser5P channel (single image plane from the middle z position of the cluster, exposure timeterm·mth , effective time resolution for full 3D volume acquisition of ). The Pol II Ser2P channel is not shown because only average intensity, not shape was quantified from this channel. (B) Example shapes to illustrate how elongationterm·mth and solidityterm·mth represent object shape. (C) Time courses of Pol II Ser5P intensity, Pol II Ser2P intensity, elongationterm·mth, and solidityterm·mth for the example time-lapse shown in panel A. (D) Cross-correlation analysis of the temporal coordination of Pol II Ser5P intensity, Pol II Ser2P intensity, and solidityterm·mth with elongationterm·mth. Gray lines indicate the time-shifted correlation for single cluster time courses, thick lines indicate the mean, and the gray region the 95% bootstrap CI. Analysis based on n = 30 tracked clusters, recorded from one sphere stage embryo. (E) Summary of the coordinated changes in phosphorylation and cluster shape suggested by the cross-correlation analysis. A stereotypical sequence of events can be seen: cluster Pol II Ser5P intensity transiently increases (red) and the cluster becomes rounder, then cluster Pol II Ser2P transiently intensity increases (blue), until finally the cluster transiently unfolds and becomes elongated.

Pseudo-timeterm·key analysis from fixed sample images also detects coordinated changes in phosphorylation and cluster shape To verify the conclusions obtained by the fluctuation analysis, we assessed changes in cluster state by an independent approach based on the interaction with a gene. Specifically, we fixed zebrafishterm·key embryos in the sphere stage, and fluorescently labeled a panel of eight genes as well as Pol II Ser5P and Pol II Ser2P (Fig. 5A ; Tables S2–S4, Supplementary Material ). In most nuclei, zero to four labeled foci representing the labeled genes could be detected, in line with the expected counts for two alleles that undergo replication before cell division ( Fig. S7, Supplementary Material ). Fixation of samples prevents live imaging, thus removing the temporal information from the images. In exchange, images with distinctly higher signal can be obtained without the need of n2v-processing, and the location of the labeled gene can be used as additional information that is not available in our live imaging data. The analysis of the obtained image data was, therefore, based on gene-Pol II cluster interaction pairs. An interaction pair is constructed by the detection of the location of a labeled gene, and by logical association of this gene with the Pol II Ser5P cluster that is closest in space ( Fig. S8A and B, Supplementary Material ). For each interaction pair, fluorescence intensities of the gene, fluorescence intensities of the Pol II clusterterm·key, distance between both objects, and shape properties of the Pol II clusterterm·key were combined into a vector representing the interaction pair. Principal component analysis of these pairs revealed a cyclical pattern, based on which a pseudo-timeterm·key coordinate was constructed (Fig. 5B ; Fig. S8C, Supplementary Material ). In particular, the two first principal component coordinates of each pair were transformed into an angle coordinate using the 2-argument arctangent (atan2) function. This angle coordinate was divided by 2π to obtain a pseudo-timeterm·key coordinate s in the range from 0 to 1. The assignment of a pseudo-temporal order to image data obtained from fixed samples has been used previously, for example for the nanoscale assessment of endocytosis ( 39 , 40 ). Ordering the interaction pairs along the pseudo-timeterm·key coordinate allowed the extraction of time-shifted correlations (Fig. 5C ), which directly mirrored those we obtained from our live imaging data (Fig. 4D ). We suspected that the location of a gene that interacts with Pol II Ser5P clusters provides the crucial information for successful pseudo-timeterm·key reconstruction (genes foxd5, klf2b , and zgc:64022 ; Fig. S9, Supplementary Material ). Indeed, when we attempted pseudo-timeterm·key reconstruction on the full panel consisting of eight genes, we found that for genes that only rarely come close to Pol II Ser5P clusters, the pseudo-timeterm·key approach failed to reproduce the correlation analysis results (genes vamp2, ripply1, drll.2, gadd45ga , and iscub ; Fig. S9, Supplementary Material ). In the case of successful pseudo-timeterm·key reconstruction, our results suggests that a gene visits a Pol II Ser5P cluster in close coordination with changes that occur in the Pol II clusterterm·key. Specifically, genes engage in close contact when cluster Pol II Ser5P intensity increases, and detach at a time when clusters undergo transient elongationterm·mth (genes foxd5, klf2b , and zgc:64022 ; Fig. 5A and B ; Fig. S9, Supplementary Material ). The time-scales of this interaction can be estimated by a comparison of the distance between the cross-correlation maximum and minimum in the cluster Pol II Ser5P signal (∼50 steps in pseudo-timeterm·key, corresponding to in the cross-correlation analysis based on live-imaging results) and the total number of observed interaction pairs (169, 186, and 191 for foxd5, klf2b , and zgc:64022 , respectively), implying an average duration of between two consecutive interaction events. To conclude, the correlation analysis based on pseudo-timeterm·key reconstruction provides an independent confirmation of the coordination between Pol II phosphorylation levels and cluster shape obtained by n2v-supported live imaging. This agreement suggests that these two approaches provide complementary views of the same, stereotyped sequence of changes in molecular properties and the shape of Pol II clustersterm·key.

Pseudo-timeterm·key analysis of data from fixed embryos relates transient engagement and activation of a gene to the phosphorylation and shape changes observed in live embryos. (A) Example images of Pol II Ser5P (magenta signal) clusters sorted by a pseudo-timeterm·key progress coordinate ( s , periodic, defined on the interval [0,1)), which is calculated based on interaction with the gene klf2b (green represents oligopaint fluorescence in situ hybridization signal for klf2b ). Center positions (weighted centroid) are indicated for the Pol II Ser5P cluster (white circle with black filling) and the gene (black circle with white filling) and connected with a white line for illustration. For details of the reconstruction, see Figure S8 (Supplementary Material) . For an overview containing all eight genes that were assessed, see Fig. S9B (Supplementary Material) . (B) Pol II Ser5P and Ser2P intensity, elongationterm·mth, and solidityterm·mth of Pol II Ser5P clusters sorted by pseudo-timeterm·key s . A total of n = 186 clusters from N = 4 independent samples, obtained in two independent experimentsterm·mth, were included in the analysis. (C) Cross-correlation analyses for different register shifts in the coordinate s , the register shift is in units of data points by which the coordinate s was shifted. Gray regions indicate 95% bootstrap CI.

Discussion In this study, we describe how the quality of images that are reconstructed by deep-learning algorithms can be controlled for, addressing the specific case of unsupervised denoisingterm·ai by n2vterm·ai (for an overview of the work-flow, see Fig. 6 ). We implemented our approach of quality controlterm·key toward the acceleration of high-speed imaging, where camera exposure timesterm·mth are reduced and the resulting loss of signal quality is recovered by n2v-denoising. We then apply our approach to the example of imaging the molecular state and the shape of Pol II clustersterm·key in live zebrafishterm·key embryos. Our work illustrates how, in a practical application setting, the performance improvements from deep-learning algorithms in fluorescence microscopyterm·key can be combined with a high level of confidence in the reconstructed images. The tools used in our study are designed for data sets consisting of ordered 2D images (“hyper-stacks”), limiting their immediate application to these types of image data.

Work-flow for n2vterm·ai reconstruction for time-lapse data with quality controlterm·key.

We specifically apply our quality controlterm·key approach to an unsupervised denoisingterm·ai technique, the deep learning-based tool n2vterm·ai ( 27 ). Currently, reconstructions that map from noisy to high-quality data on the basis of paired training image data offer the highest reconstruction performance ( 22 ). In many practical settings, such pairs of noisy and high-quality images cannot be obtained. An alternative is offered by reconstructions based on matched pairs composed of noisy images only (n2n) ( 26 ). Further developments now offer the possibility to reconstruct high-quality images directly from single noisy images (Noise2Self ( 41 ) and n2vterm·ai ( 27 )). Such a self-supervised approach seems ideally suited to reconstruction tasks where fluorescence labeling exhibits strong variability, optical components are changed between different experimentsterm·mth, or sample properties vary on a day-to-day basis. These characteristics are typical of biological microscopy applications, highlighting the applicability of self-supervised reconstruction methods in this area. A crucial assumption of self-supervised denoisingterm·ai approaches is that the noise in each pixel is an uncorrelated sample from the same probability distribution. Newer variants of these algorithms explicitly adjust the probability distribution of the noise to different parts of the image, thus improving the results where additional information on the noise characteristics is available ( 19 , 42 , 43 ). Yet, other variants model the structure of the signal itself ( 44 ). These newer variants of self-supervised denoisingterm·ai could provide further improvements in reconstruction performance, while retaining most of the pragmatic applicability of self-supervised reconstruction methods.

We base our assessment of image quality on two metrics, (local) SSIMterm·mth and FRCterm·mth. More generally, metrics for image quality assessment belong to three main groups of functionality. The first group includes methods assessing the quality of images against a corresponding reference image (high-quality image). These methods are called full-reference, emphasizing the need for high-quality reference data ( 28 , 45 ). SSIMterm·mth and consecutive similarity (CSS) metric, which is a variation of SSIMterm·mth ( 46 ), are in this category. We used (local) SSIMterm·mth, which provides an error map by structurally comparing the reconstructed image with the reference image, and based on that error map controlled for reconstruction defects. The second group, called reduced-reference, contains methods which are not using matched reference images, but rather general knowledge of properties and statistics that are typical of a set of reference images ( 47 ). Natural scene statistics (NSS) is one major method in this category ( 45 ). The underlying hypothesis of all NSS-based method is that all the original images are “natural” and that a distortion process introduces some unnaturalness that can be quantified by deviation from models of natural signals. Due to the day-to-day variability of the signals produced by fluorescence microscopyterm·key of biological samples, modeling natural signals appears challenging. The third category of image quality assessment methods is called no-referenceterm·mth, because quality assessment proceeds without a matched reference image or other prior knowledge ( 48 ). FRCterm·mth is in this category and we used it to assess the spatial resolution of the reconstructed images. Based on the achieved spatial resolution, we could decide how far exposure timesterm·mth could be reduced while still supporting successful denoisingterm·ai. One tool that implements several of these metrics for the assessment of local anomalies in super-resolution microscopy data is SQUIRREL ( 49 ). The quality scores and error mapping provided by SQUIRREL can, in principle, also be applied to images reconstructed by deep-learning methods.

The image acquisition protocol we propose consists of a phase during which all necessary data for quality controlterm·key are collected for a single time point (phase A), followed by high-speed time-lapse imaging with compromised image quality (phase B). This protocol seems appropriate for the acquisition of short bursts of images, where the main limitation lies in how many images can be acquired in a short amount of time. For other imaging challenges, different protocols could be developed. In a different situation where, for example, photo-bleaching limits the acquisition of long time courses, excitation light levels could be reduced, and the compromised signal could be recovered by denoisingterm·ai. In such an experimentterm·mth, quality controlterm·key points could be placed at regular intervals over the course of acquisition. In a setting where, for example, sample structure or the level of fluorescence labeling changes significantly over the course of recording, a quality controlterm·key phase at the beginning and at the end of the experimentterm·mth might be advisable. Besides the implementation of additional control points in the experimental procedure, such extensions of our simple two-phase protocol would need no further modification to the quality controlterm·key approach we used in our work.

Our live-sample microscopy recordings reveal a stereotypical sequence of events, where the Pol II recruitment and pause–release steps of transcriptional induction are closely coordinated with changes in the shape of Pol II clustersterm·key. While previous studies achieve high spatial or temporal resolution, our approach combines high resolution in time as well as in space. Our temporal resolution of 1 to 2 s for a full 3D stack is comparable to previous assessments of Pol II localization ( 50 , 51 ). These studies, however, do not monitor the specific phosphorylation states associated with Pol II regulation. Imaging of these phosphorylation states was previously performed with an effective time resolution of 1 min for a single gene ( 38 ) or 10 s for an engineered gene array ( 52 ) for the acquisition of full 3D volumes. By fitting of kinetic models of Pol II regulation, these studies suggest rates of pause release of 2 to 2.5 min and production of the first 1 kb of transcript length within 2.5 min (assuming an elongationterm·mth rate of 0.4 kb per min) ( 38 , 52 ). Photobleaching experimentsterm·mth assessing endogenous Pol II combined with computational modeling indicated 2.3 s for initiation and 42 s of pausing at the promoter, as well as an elongationterm·mth rate of 2 kb per min ( 53 ). These estimates for the elongationterm·mth rates fit well with estimates of approximately 1 kb per min from alternative sequencing-based approaches ( 54 ). Lastly, another study suggests that 6.3 s are sufficient for Pol II to loosely associate with an induced gene as well as proceed into elongationterm·mth ( 50 ). While these estimates for the duration of induction and pause–release imply a broad spectrum of kinetics, our estimates of 2 to 3 s for pause release and approximately 36 s for the duration of one complete gene–cluster interaction cycle fall within the previously estimated range for pause release and RNA production. Besides temporal coordination, also relative distances have been assessed, for example between Pol II clustersterm·key and nascent mRNA ( 38 ) and between enhancers, Pol II, and the transcription start site ( 55 , 56 ). In these studies, nascent mRNA is displaced 100 to several hundred nm relative to sites harboring transcriptional regulators and recruited Pol II. This displacement is in line with our observations that genes that undergo elongationterm·mth are located outside of Pol II Ser5P clusters. In contrast to previous work, our approach reveals the full shape of the Pol II Ser5P clusters. Taken together, the kinetics of single-gene induction suggested by our live-sample experimentsterm·mth seem in line with previous work, and the spatial organization of clusters and interacting genes directly correlates with previous work assessing relative distances of different components of the transcriptional machinery.

Our pseudo-timeterm·key reconstruction revealed that the changes in Pol II phosphorylation and cluster shapes are temporally coordinated with the visit of genes to the Pol II clustersterm·key. Previous work suggests that the Pol II clustersterm·key in early embryonic development form on regulatory chromatin regions, including superenhancers ( 36 , 57 , 58 ). Accordingly, our data seem to directly show single genes that undergo transcriptional activation during a visit to Pol II-enriched clusters that contain regulatory chromatin regions. Different models for such enhancer–promoter communication in transcriptional control were proposed ( 59–61 ). The stereotypical sequence suggested by our data fits most closely to a condensate hit-and-run model, where genes transiently interact with enhancer-associated condensates for transcription initiation, and leave from the condensate in association with the onset of transcriptional elongationterm·mth ( 61 ). A condensate hit-and-run model can also explain earlier observations suggesting cyclic interactions, where genes repeatedly engage with and depart from Pol II-enriched clusters ( 62 ). Such a model also could support the proximity-dependent activation of Shh by its enhancer ZRS ( 63 , 64 ). The activation of genes by enhancers was also found to not require direct contact, but can occur over a distance of 200 nm or more ( 65 , 66 ). These observations, together with evidence in support of the condensate hit-and-run model, allow speculations about a liquid-bridge model of enhancer–gene communication. In such a liquid-bridge model, genes transiently become embedded within an enhancer-associated condensate, allowing the transfer of transcriptional machinery, including Pol II, to the gene promoter ( 61 , 67 , 68 ). While previous work indicates that the onset of RNA production at newly activated genes results in their exclusion from the enhancer-associated condensates ( 37 , 69–72 ), the initial engagement with the enhancer-associated condensates is less well understood. Such an engagement would, however, be naturally explained by the formation of small condensates at promoters. Such condensates could emerge, for example, at CpG-rich regions that are placed directly upstream of promoter regions of many developmental genes and were found to contribute to gene–promoter contacts in 3D space ( 73 ).

Methods Summary Live imaging of primary cell culture of human cheek cells Short-term primary cell cultures of human cheek cells were obtained by a buccal smear, stained with Hoechst 33342, and transferred into an 8-well ibidi μ-Slide (#1.5 selected glass) for microscopy. Microscopy data were recorded using a commercial implementation of the instant-SIM high-speed super-resolution confocal microscopy principle (VisiTech iSIM) ( 33 ) built on a Nikon Ti2-E stand. A Nikon Silicone Immersion Objective (NA 1.35, CFI SR HP Plan Apochromat Lambda S 100XC Sil) was used. Detailed description see Supplementary Material Appendix .

Zebrafishterm·key husbandry, live imaging, and fixation Embryos were obtained through spontaneous mating and dechorionated with pronase. For STED microscopy of DNA in fixed zebrafishterm·key embryos protocols from our previous work were followed ( 74 ). STED microscopy was performed using a Leica TCS SP8 STED microscope with a 775-nm depletion line and a motorized-correction 93x NA 1.30 glycerol objective (HC PL APO 93X/1.30 GLYC motCORR). For live-imaging of Pol II CTD phosphorylation, embryos were microinjected with covalently labeled fragments of antibodies (Fab). Microscopy images were recorded by iSIM with the silicon immersion objective. Detailed description see Supplementary Material Appendix .

Oligopaint FISH and immunofluorescence Following protocols from our previous work ( 36 ), fixed sphere-stage zebrafishterm·key embryos were subjected to oligopaint FISH labeling of genomic regions surrounding zygotically expressed genes ( 75–78 ), followed by indirect immunofluorescence detection of Pol II CTD phosphorylation, and mounted in Vectashield H-1000 under #1.5 selected cover glass. Image data were recorded by iSIM using a Nikon Oil Immersion Objective (NA 1.49, CFI SR HP Apo TIRF 100XAC Oil). Detailed description see Supplementary Material Appendix .

Metrics for image assessment For SSIMterm·mth and local SSIMterm·mth analysis, we only used the structural term The FRCterm·mth analysis is based on the cross-correlation of two images in frequency space: where F 1 , F 2 are the Fourier transforms of two images and r i refers to all frequency space bins that fall within a given ring radius r . Images are considered reliably resolved up to the frequency F ( r ) for which FRCterm·mth ( r ) falls below the commonly used threshold value of 1/7. Details are described in Supplementary Material Appendix .

Image analysis The analysis of Pol II clustersterm·key in live-imaging data as well as the pseudo-timeterm·key reconstruction from fixed-sample data were implemented in the form of MatLab scripts. The analysis steps and script availability are described in SI Appendix.

Authors’ Contributions Designed research: H. H., I. M., R.P., A.P., A.K., L.H. Performed research: H.H., I.M., R.P., A.P., L.H. Analyzed data: H.H., R.P., L.H. Wrote the paper: H.H., I.M., A.K., L.H.

Supplementary Material Click here for additional data file.

ACKNOWLEDGEMENTS Antibody fragments (Fab) for live imaging were kindly provided by the Kimura laboratory (Tokyo Institute of Technology). STED microscopy was conducted at the Karlsruhe Center for Optics and Photonics (KCOP). We thank Ralf Mikut for comments on our manuscript.

Notes Competing Interest: The authors declare no competing interest.

Contributor Information Hamideh HajiabadipersName, HIDSS4Health - Helmholtz Information and Data Science School for Health, 76131, Karlsruhe/Heidelberg, Germany; Institute of Biological and Chemical Systems, Department of Biological Information Processing, Karlsruhe Institute of Technology, 76344, Eggenstein-Leopoldshafen, Germany; KASTEL – Institute of Information Security and Dependability, Karlsruhe Institute of Technology, 76131, Karlsruhe, Germany.

Irina MamontovapersName, Institute of Biological and Chemical Systems, Department of Biological Information Processing, Karlsruhe Institute of Technology, 76344, Eggenstein-Leopoldshafen, Germany.

Roshan PrizakpersName, Institute of Biological and Chemical Systems, Department of Biological Information Processing, Karlsruhe Institute of Technology, 76344, Eggenstein-Leopoldshafen, Germany.

Agnieszka PancholipersName, Institute of Biological and Chemical Systems, Department of Biological Information Processing, Karlsruhe Institute of Technology, 76344, Eggenstein-Leopoldshafen, Germany.

Anne KoziolekpersName, KASTEL – Institute of Information Security and Dependability, Karlsruhe Institute of Technology, 76131, Karlsruhe, Germany.

Lennart HilbertpersName, Institute of Biological and Chemical Systems, Department of Biological Information Processing, Karlsruhe Institute of Technology, 76344, Eggenstein-Leopoldshafen, Germany; Zoological Institute, Department of Systems Biology and Bioinformatics, Karlsruhe Institute of Technology, 76131, Karlsruhe, Germany.

Funding This work is supported by the Helmholtz Association under the joint research school “HIDSS4Health – Helmholtz Information and Data Science School for Health” and the Helmholtz program Natural, Artificial, and Cognitive Information Processing (NACIP). H.H. was supported by the HIDSS4Health, I.M. and L.H. were supported by the Priority Program Molecular Mechanisms of Functional Phase Separation of the German Science Foundation (DFG-SPP2191). We acknowledge support by the KIT-Publication Fund of the Karlsruhe Institute of Technology.

Data availability All raw data and analysis code are provided as supplementary materials or via publicly accessible repositories. All links to repositories are listed in the supplementary material. The raw data obtained from human cheek cells cannot be shared due to ethical restrictions.

References Associated Data Supplementary Materials Click here for additional data file.

Data Availability Statement All raw data and analysis code are provided as supplementary materials or via publicly accessible repositories. All links to repositories are listed in the supplementary material. The raw data obtained from human cheek cells cannot be shared due to ethical restrictions.