1 Preface
AI at Every Stage of the Microscopy Workflow
Teng-Leong Chew 
Rachel M. Lee
Owen F. Puls
The rapid advancement of microscopy, further fueled by parallel developments in related technologies such as probe chemistry, molecular biology, and image analysis, has propelled bioimaging to the forefront of life science studies1. In fact, many life scientists now fully expect to visualize intricate biological processes in the context of a whole, living organism—a prospect deemed impossible a mere decade ago.
This seemingly limitless possibility has greatly amplified the complexity of the questions that can be probed. Likewise, it demands a significant degree of forethought in the design of biological experiments2,3 to ensure that the maximal amount of biological information can be extracted from the acquired data. At the same time, the wealth of information contained in modern bioimage data can now often exceed the grasp of a human observer. This thus requires computational means to aid in data interpretation. In other words, the ever-expanding and multifaceted field of artificial intelligence (AI) can no longer be separated from bioimaging. Yet the absence of a comprehensive resource to explore how AI can enhance the field prevents many in the community from taking advantage of this indispensable technology. This textbook therefore aims to demystify AI in its many forms and outline proper use cases thereof to facilitate biological discovery. We have specifically contextualized the textbook for two target audiences: microscopists and biologists. New AI and machine learning (ML) tools often fail to tailor explanations and tutorials for the very users that will ultimately effectuate scientific discovery. We hope this textbook will bridge that gap.
As the chapters of this book will collectively demonstrate, AI now plays an indispensable role in the four main stages of the microscopy experimental workflow (Figure 1.1): project design, tool-building, execution of experiments, and quantitative data interpretation. The microscopist must optimize each of these stages in concert to maximize the usefulness of their data2,3. That optimization can be enhanced by AI tools, which are capable of everything from generating new ideas for experiments to maximizing the information in noisy images.
To guide researchers just getting started with AI, the first section of this book focuses on introducing the core concepts of AI, large language models (LLMs), and the computational architectures that underpin these technologies. Chapter 2 will introduce the basic principles behind AI and ML as a gateway for those less familiar with this topic. In the experimental workflow, modern hypotheses can now drive a project design so complex that AI may be required to design the tools needed to tackle the experimental challenge. As will be discussed in Chapter 3, this can be accomplished by using LLMs as assistants. Chapter 4 will further explain the fundamental construction and mechanics of AI models.
The next section focuses on extending microscopy hardware with AI. To render a microscope “smart”, i.e., for it to perform an experiment with minimal or no human intervention, ML software first needs to be trained on extensive and high-quality training data. The collection and design of training data will be covered in Chapter 5. Chapter 6 will discuss the implementation of AI and ML to enhance the capabilities of existing microscopes denoising and other image restoration techniques. Likewise, AI and ML models can be designed to control event-driven image acquisition to visualize biological events that are rare and/or photosensitive (Chapter 7).
The final section will address processing and analyzing microscopy data with AI models in order to extract biological information beyond what is achievable through human observation alone. Chapter 8 will help readers find and use available open-source tools. When those existing tools are insufficient, Chapter 9 seeks to further widen the horizon by discussing the process of training new models for specific image processing and analysis tasks. Finally, Chapter 10 will address the question of when a model is performing well enough. This important step pinpoints potential weaknesses of the analysis approach, thereby placing results in meaningful context.
We conclude with Chapter 11, which summarizes the core concepts and aims to inspire readers to begin taking full advantage of the promise of the new era of AI and ML. To close the loop, we also hope that by equipping readers with the proper vocabulary, the book will facilitate more effective communication between biologists and AI tool developers.