11 Outlook
What can AI enable for biological imaging?
With this book, we hope to have set you on a journey of exploration into the use of AI methods in microscopy. In this final chapter, we reflect on several early examples in which AI has already changed what can be observed, measured, and inferred from microscopy data, and then look ahead to the questions and capabilities now coming into view. Finally, we turn to the path before you — how you can build on what you have learned to not only answer specific biological questions, but contribute to the broader advancement of this rapidly evolving domain.
11.1 Early successes: AI across biological imaging
11.1.1 In situ cryo-Electron Tomography
Cryogenic electron tomography (cryo-ET) images biological samples preserved in vitreous ice by acquiring a series of transmission electron micrographs at different tilt angles and reconstructing them into a three-dimensional volume. Combined with computational post-processing1, cryo-ET offers the highest spatial resolution available for imaging cellular structures in their near-native context, and can resolve repeated macromolecular complexes at Ångström-level resolution2. The data is, however, not easy to analyze: the electron dose must be limited to avoid damaging the sample, resulting in a very low signal-to-noise ratio, while the restricted range of achievable tilt angles produces a characteristic missing-wedge artefact. The frontier application of cryo-ET to intact microorganisms and thin cellular lamellae adds a further challenge of macromolecule identification in an extremely crowded and heterogeneous environment3. AI is becoming integral across the workflow: helping to select suitable acquisition regions4, improve reconstructions5, segment cellular structures, and locate large numbers of macromolecular particles for structural averaging6–9. These capabilities are beginning to turn cellular tomograms into molecular maps in which complexes can be identified and compared in situ10–12, opening the way to studying their organisation, interactions, and structural states inside cells rather than in isolation. As acquisition and analysis become increasingly automated, the longer-term vision is a form of visual proteomics that can systematically chart the molecular architecture of cells in near-native conditions13–15.
11.1.2 Connectomics
The field of connectomics aims to reconstruct and analyze complete neural circuit diagrams. Microscopy for connectomics faces the combined challenge of requiring a very high resolution for individual synapse detection and a very large field of view for complete neuron tracing. Up until the very recent advances in Expansion Microscopy16, Electron Microscopy remained the only modality suitable for connectomics studies, routinely generating tens to hundreds to thousands of Terabytes of image data at nanoscale resolution17,18. As the infeasibility of fully manual analysis became obvious with the first significant volume acquisitions (the first connectome from19 required a decade of manual tracing!), neuroscientists sought collaboration with machine learning and computer vision experts to speed up and automate neuron segmentation and synapse detection. Since then, deep learning has enabled increasingly complex pipelines20–22, including AI-based error detection and correction23,24. More recent models are beginning to infer properties that were previously inaccessible from morphology alone, such as neurotransmitter identity prediction directly from electron microscopy25. As connectomics expands towards whole mammalian brains and substantial volumes of the human brain26,27, progress in AI-based analysis will remain inseparable from progress in the field itself28.
11.1.3 Image restoration and virtual staining
As discussed in Chapter 6, AI-based image restoration is becoming an increasingly important part of modern microscopy. Restoration models can suppress noise, compensate for optical degradation, improve spatial or temporal resolution, and reconstruct useful images from measurements acquired with fewer photons or less sampling29,30. This can enable faster, deeper, and gentler imaging, extending the duration of live-cell experiments and revealing structures or dynamics that would otherwise be difficult to observe31–33. Related virtual labelling approaches can predict fluorescence-like representations from label-free or sparsely labelled images34–36. However, since these models generate image content rather than simply measure it, their outputs may contain convincing-looking structures that are weakly supported by the acquired data37. Trustworthiness therefore depends on careful validation against independent measurements, appropriate controls and biological expectations38. Emerging approaches are beginning to complement generated images with calibrated, spatially resolved uncertainty information, helping researchers identify regions that require closer inspection39–41. Together, these developments are shifting AI to become an integrated — and increasingly transparent - part of the imaging process itself30,42,43.
11.1.4 Imaging development
Development transforms a single cell into a complex organism through coordinated cell divisions, movements, shape changes, and differentiation. Modern light-sheet microscopes can capture these processes across entire living embryos with high spatial and temporal resolution, producing volumetric time-lapse recordings in which tens of thousands of cells must be segmented and tracked over hundreds or thousands of frames44–46. Both tasks are challenging at this scale: cells divide, deform, and become densely packed, while tracking algorithms must preserve identities, detect divisions, and remain robust to segmentation errors that can propagate through complete lineage trees47. Continued progress in AI-based segmentation is improving developmental analysis pipelines. Increasingly sophisticated approaches are extending machine learning to tracking itself, with recent methods addressing learned cell association, segmentation uncertainty, and iterative error correction at whole-embryo scale48–51. Together, these advances bring us closer to turning whole-embryo movies into quantitative records of every cell’s behaviour and history48,52, ultimately enabling systematic comparisons across embryos, perturbations and species53,54.
11.1.5 Multimodal integration in large-scale experiments
High-throughput microscopy experiments are reaching an extraordinary scale, producing hundreds of terabytes of data and images of billions of cells across tens to hundreds of thousands of chemical and genetic perturbations55,56. At the same time, new experimental platforms are beginning to measure several complementary aspects of cellular state, pairing morphological imaging with readouts such as gene expression, protein abundance, and spatial organisation57–59. Integrating these modalities is challenging as they differ in dimensionality, resolution, and experimental biases, while each captures only a partial view of the biological response. Early AI-based studies have shown that imaging, transcriptomic, and chemical representations can be combined to improve the identification of related perturbations and mechanisms of action60,61, while newer models are beginning to predict one modality from another, including transcriptome-guided prediction of cellular morphology62–64. The longer-term ambition of these efforts is the development of Virtual Cells65: computational models that integrate diverse measurements of cellular state and predict how cells will respond to previously unseen perturbations66–68. Although such models remain at an early stage69, they point towards a future in which large parts of cellular experimentation can first be explored in silico, guiding which hypotheses and interventions should be tested in the laboratory.
Across these examples, AI succeeds not simply by replacing manual analysis, but by changing the scale, resolution, or complexity at which biological systems can be studied. In microscopy, AI does not act on images in isolation: it operates on measurements shaped by the sample, labels, optics and the acquisition process. Its greatest impact will therefore come when models, experiments, and biological questions are designed together rather than treated as separate stages.
11.2 Why is now a good time to start with deep learning for microscopy?
As the examples above illustrate, deep learning can provide a decisive advantage when analysing data that is large, volumetric, high-throughput, or time-resolved. In other applications (e.g. screening, image-based cell sorting, or targeted imaging), the accuracy of the analysis pipeline determines how reliably regions and events are identified, and therefore directly influences the success of the experiment. Many such closed-loop applications, in which analysis actively shapes data acquisition, are discussed further in Chapter 7 on smart microscopy. Automated quality control, detection of rare or transient phenotypes, adaptive acquisition focused on particular structures, classification of cells before downstream manipulation, and prioritisation of candidates for follow-up experiments: all of these can directly benefit and have already been made target of the development of AI for microscopy. The scalability of deep learning analysis also allows these workflows to be extended to datasets sufficiently large to reveal subtle phenotypes and dynamic patterns. More broadly, advances in accuracy, robustness, and computational speed do more than save time: they practically redefine the experiments that can be attempted and the biological questions that can be answered.
Over the years of the explosive growth of DL-based technology, the field has accumulated substantial practical knowledge. Established architectures discussed in Chapter 4 already perform well across many common tasks, including classification, segmentation, detection, tracking, and image restoration, while newer approaches continue to expand what is possible. Many microscopy problems no longer require inventing an entirely new method from scratch; instead, the challenge is often to select an appropriate model, adapt it to the data, and validate that its outputs are biologically meaningful. Such adaptation is more accessible than ever, and even developing a new method from the ground up is becoming easier. As detailed in Chapter 3, emerging conversational tools70,71 allow users to describe an analysis problem in natural language and receive guidance, executable code, or the beginnings of a complete workflow, making it possible to assemble useful pipelines with little or no initial coding. From there, general AI-assisted coding tools can support a gradual progression towards greater customisation, helping you understand, modify, and debug the code behind your analyses. Pretrained and increasingly general-purpose models also make it possible to transfer knowledge between datasets and tasks, often reducing the amount of annotation needed for a new experiment. Once validated, automated pipelines can apply the same analysis consistently across large studies, improving reproducibility as well as throughput. At the same time, microscopy-specific software, public datasets, tutorials, and educational resources — including this book — are making this growing body of expertise easier to find and reuse.
The existing tools discussed in Chapter 8 are already powerful, but they are most useful when applied with an understanding of how they work and where they can fail. It is tempting to treat a model as a black box and immediately pass its output to the next analysis step, but even small systematic errors can distort downstream biological conclusions. Here, you have a distinct advantage as a domain expert: you understand the experimental context, know which structures and behaviours are biologically plausible, and can recognise when an output scores well on standard benchmarks but remains scientifically implausible (as discussed in Chapter 10). Understanding the basic principles behind a method will help you combine this biological judgement with appropriate validation, recognise unreliable outputs, and design better experiments. Used in this way, deep learning becomes a powerful means of amplifying your expertise.
11.3 What’s next?
Deep learning is developing rapidly and in ways that remain difficult to predict. Few researchers asked in 2016 — the year before the Transformer architecture was introduced — would have anticipated today’s generative models, natural-language interfaces, and foundation models, or the accuracy and generality now achieved on established microscopy tasks such as cell and nuclei segmentation. Rather than trying to forecast individual architectures, let us consider the broader directions in which microscopy image analysis is beginning to move. Across all of them, we believe that continued progress will depend on shared datasets, rigorous evaluation standards, and close collaboration between biologists, microscopists, and AI researchers.
One such direction is the increasing transfer of models from natural image computer vision to microscopy. Foundation models trained on natural images, such as DINOv2 and the newer DINOv3, are beginning to provide useful representations for cellular morphology and microscopy tasks, sometimes with limited adaptation to the target data72–74. This is perhaps less surprising than it first appears: humans also approach microscopy using a visual system developed through experience with the natural world, and gradually learn to recognise domain-specific structures and patterns. Future workflows may similarly span a continuum, from using general-purpose vision models directly, through lightweight adaptation and more extensive supervised fine-tuning, to full self-supervised retraining on large collections of microscopy images75–78.
For data-driven generative models that produce microscopy images from acquired measurements, the next challenge will be to make uncertainty operational. A model may already indicate that several outputs are compatible with the available data, but this information is useful only if it can propagate down the analysis pipeline. Future segmentation, tracking, and quantification methods will need to handle uncertainty in their inputs and report how ambiguity in generated images affects their results. Uncertainty could also guide experimental decisions, e.g., by triggering reacquisition, directing higher-resolution imaging towards ambiguous regions, or identifying cases that require expert review. The goal is therefore to build workflows that respond appropriately when the data does not support a single confident interpretation.
Another emerging direction is the development of agentic analysis systems that move beyond generating individual pieces of code towards assembling and evaluating complete microscopy workflows, including additional experimental steps (Chapter 3). Their success will depend on strict validation at every stage, because an error introduced early in a workflow can propagate unnoticed into the final biological conclusion. Here again, your domain expertise is a critical asset: it provides the experimental context needed to define meaningful checks, recognise implausible outputs, and decide when closer human inspection is needed. Although current examples remain sparse, they point towards a future in which natural-language interaction becomes an interface not only to individual models, but to increasingly autonomous and reproducible experimental pipelines.
A deeper conceptual shift will be the expansion of AI from processing images to interpreting the biology measured by them. Representation-learning approaches are already being used to discover cell types and unusual phenotypes without predefined labels, or to organize proteome-scale patterns of cellular morphology and subcellular protein distribution79,80. Other models can infer properties that are not directly visible in the input image, including cellular forces and mechanical behaviour81–83, or reconstruct temporal processes from collections of static observations84. These early examples will give rise to analysis systems that identify phenotypes, infer hidden physical or molecular states, and propose relationships within complex biological data. The long-term opportunity is for AI to become not only a tool for extracting measurements from images, but a partner in turning those measurements into biological understanding.
11.4 How to continue from here?
One of the best ways to continue learning is to engage with the open-source and open-science ecosystem around bioimage analysis. Public model repositories such as the BioImage Model Zoo and — for deep learning more broadly — HuggingFace can show you how others preprocess data, build their models and evaluate intermediate and final performance. Reproducing or adapting an existing workflow is often more instructive than starting from an empty notebook, as it exposes the many practical decisions that sit between an architecture and a reliable biological result. Similarly, your own models can become useful long before your final biological results are ready, so consider contributing them to model repositories with complete metadata, descriptions of their intended use and limitations, and reproducible training and validation code. Sharing the full workflow rather than only the final weights makes it far easier for others to build upon your work. Open communities such as the Scientific Community Image Forum (Image.sc) give a further opportunity to learn directly from tool developers and experienced users. As your own expertise grows, answering questions and sharing solutions is a great way to strengthen your understanding and help others solve similar problems.
Finally, the central importance of data in deep learning cannot be emphasised enough. As a domain expert, you hold the key to the future capabilities of AI models for microscopy: architectures will continue to evolve, but the quality, diversity, and documentation of the data remain the foundation of what any model can learn. Following the FAIR principles — making data findable, accessible, interoperable and reusable — helps preserve the data value beyond the original study, allowing it to support new models and biological questions. AI-ready microscopy data should therefore be treated as a scientific product in its own right: organised in standard formats, accompanied by rich metadata, and linked to clear annotations and provenance. Researchers acquiring such data can contribute it to public repositories such as the BioImage Archive, which provides a permanent home for biological imaging datasets and makes them available for reuse by the wider community. Community resources such as this book aim to provide the experience with training AI models, enabling domain experts to structure their data into training, validation, and test sets that avoid hidden leakage and provide trustworthy estimates of model performance. Designed with both biological and computational expertise, such datasets can support not only individual experiments, but also future benchmarks that attract AI researchers to the field and help address its outstanding problems. You do not need to wait for a perfect model or a complete benchmark: a documented dataset, a reproducible notebook or a well-described failure case can already be valuable to others. Share your models, data and expertise generously! In doing so, you join the community shaping the future of AI in microscopy and help make the next biological discoveries possible.