Glossary

A-posteriori Analysis

An approach in which iamges are processed after the complete dataset has been acquired and stored.

AI agent

AI agents extend models into systems that can take actions (running code, calling APIs, controlling hardware), observe the results, and adapt their approach iteratively. At its core, an AI agent is a system built around a language model that operates in a loop: it observes the current state (an image, a dataset, an error message), reasons about what to do next, acts by invoking tools or executing code, and then observes the outcome — repeating until the task is complete. This observe-reason-act cycle is what distinguishes an agent from a chatbot.

Aberration

The spreading of light (also called ‘wavefront distortion’) due to imperfections in the optical path or variations in refractive index at the sample, which results in images that are blurrier than the ideal diffraction-limited image we would expect were aberrations absent.

Activation Function

An activation function is a mathematical formula that calculates the output of a node. These functions are nonlinear, which allows the network to solve nontrivial problems. They are so named because the output of the function decides to what extent each node in the network is “activated,” an analogy to the way in which biological neurons can be activated.

Activations

The values of elements in neural network units, often specifically the values of the output of an activation function.

Adaptive Optics

Technology that senses distortions in the wavefront of light and cancels them, thereby suppressing optical aberrations to enhance image clarity.

Agentic Workflow

Agentic workflows are hand-designed sequences of tool calls with human-specified decision points. A developer would define: “first call the segmentation tool, then if the count is above 100 switch to algorithm B.” These workflows are effective but rigid: every decision point must be anticipated in advance.

Autoencoder

A deep learning architecture used to learn efficient coding of unlabeled data. An autoencoder learns two functions: an encoding function that transforms the input data, and a decoding function that recreates the input data from the encoded representation. It is an unsupervised learning version of the more general encoder-decoder architecture.

Auxiliary Variables

Auxiliary variables are variables added to a linear programming problem to simplify mathematics, improve precision, or handle missing data. An example of an auxiliary variable is distance to the boundary. These or other shape descriptors can be used as part of of a loss function.

Average Precision

Average Precision (AP) summarizes the precision–recall trade-off across confidence thresholds. A predicted instance counts as a true positive only if its IoU with a ground-truth instance exceeds a threshold \(x\):

\[\text{AP}_x = \sum_{k} (R_k - R_{k-1})\, P_k\]

where \(P_k\) and \(R_k\) are precision and recall at the \(k\)-th confidence threshold after matching predictions to ground-truth at IoU \(\geq x\). Higher \(x\) demands more precise localization.

Backpropagation

The method used by neural networks to learn from its predictions. Once the prediction is done, it is compared with the ground truth through a training loss and the value of the comparison is used backwards to sequentially update the weights in the neural network, reward it when making a good prediction and punish it when making a bad prediction.

Batch

A small group of data that is processed together at the same time. For example, when training a machine learning model, a batch is a group of data that is given to the model for learning. Batches are commonly used to make the processes more efficient.

Bayesian Optimization

A strategy that allows the optimization of black-box functions such as deep neural networks. It creates a surrogate model, which is a probabilistic representation of the objective function, using only a few example points.

Bias Vector

The bias vector is a set of values (one for each node in the layer) that is added to the weighted input of the layer before the activation function is calculated. Note that the use of “bias” here is mathematical and not related to any potential biases in the model’s decision making.

Binary Cross-Entropy

Binary cross-entropy (BCE) penalizes the model based on how confident it is in the wrong answer. BCE requires the output to be interpretable as a probability — it asks “is this pixel foreground or background?” and treats all wrongly-classified pixels the same way, whether they are at a cell boundary, inside a hole, or belong to an entirely missing cell. For ground truth label \(y_i \in \{0,1\}\) and predicted probability \(\hat{y}_i \in [0,1]\): \[\text{BCE} = -\frac{1}{N} \sum_{i=1}^{N} \bigl[ y_i \log(\hat{y}_i) + (1-y_i)\log(1-\hat{y}_i) \bigr]\]

Binary Segmentation

A type of image segmentation where each pixel is classified into one of two categories — typically “foreground” (e.g., cell) or “background.” The output is a binary mask distinguishing objects (set to a value of 1) from their background (0).

Boosting algorithm

An ensemble learning method that builds a strong classifier by training many weak learners sequentially, with each new learner trained to correct the errors of the previous ones.

Bounding region

A geometric shape, typically a box, drawn around an object in an image to indicate its spatial extent.

CARE (Content-aware image restoration)

A deep learning-based method for image restoration that leverages content-specific features to enhance degraded images. See https://github.com/CSBDeep/CSBDeep for more information.

Capacity

A model’s capacity reflects how much information a network can store. It corresponds to a network’s ability to model an arbitrary function.

Centroid coordinate

An object’s geometric center point, used as a simple way to describe its location in an image.

Channels

In natural color images there are often three channels (red, green, and blue). In microscopy images, the number of channels can refer to the number of fluorescent labels or image settings used while imaging. The term is used similarly in neural networks where channels refers to the number of layers in an image. These channels can come from convolutional operations, leading to images with hundreds of channels corresponding to each filter used.

Class label

A descriptive name selected from a predefined list of categories, assigned to a detected object or event to describe its identity.

Classification

The process of assigning a data point to one of a set of predefined categories, based on its measured features.

Coding Agent

Coding agents are AI systems that operate directly in development environments with access to terminals, file systems, and web browsers. These agents go well beyond earlier LLM assistants that could only suggest code snippets.

Computer Vision

A field of computer science wherein computers extract information from images. It often involves object detection within images and can involve classification of the images and/or objects.

Confidence Score

A continuous value, often between 0 and 1, that reflects how certain a model is about a given prediction. Unlike a distance transform, which encodes geometric information such as position within an object, a confidence score expresses the model’s belief that a prediction is correct — for example, that a pixel belongs to the foreground class or that a detected object was correctly identified during object detection. Confidence scores are useful for filtering predictions, such as discarding low-confidence detections, but a high confidence score does not guarantee a correct prediction, particularly when a model is poorly calibrated.

Context Window

A context window is the size of the largest input that a model can consider at once. Making the context window larger allows the model to incorporate more information into the output.

Convolution

A mathematical process where a kernel (small matrix) slides over input data (e.g., images) to compute feature maps, highlighting patterns like edges or textures.

Convolutional Neural Networks (CNNs)

A deep learning architecture that applies convolutions to automatically learn features from images for computer vision tasks like classification and detection.

Cytoprofile

A numerical feature vector describing a biological cell’s phenotype, derived from measurements of shape, intensity, and texture extracted from microscopy images.

Data Augmentation

A strategy to artificially increase the diversity of a dataset prior to training by applying transformations such as rotation, flipping, or brightness adjustment. It helps improve model robustness and generalisation.

Decision tree

Decision trees are flow-chart-like models that make simple yes/no decisions in a series of conditional steps. The first root node splits the entire dataset into two groups, after which internal nodes and branches continue to split the data through conditional questions, finally terminating in a leaf node that provides the answer, class, or prediction.

Deconvolution

A mathematical process to partially reverse the blurring effect caused by the microscope’s PSF, increasing contrast and resolution over the raw image data if performed carefully.

Distance Transform

A transformation that assigns each foreground pixel a value representing its distance to the nearest background pixel or object boundary, producing an image that peaks at object centers and falls to zero at their edges. In instance segmentation, distance transforms are commonly used to help separate touching or overlapping objects—for example, as a target the model learns to regress during training, or as an input to watershed-based post-processing that splits a binary mask into individual instances.

Domain Randomization

Using simulations or synthetic training data, domain randomization applies random and exaggerated variations to background, lighting, shapes, or textures in the synthetic dataset. This strategy helps the model learn domain-invariant features and is usually used for pretraining a neural network or to enable simulation-to-real transfer.

Downsampling

Downsampling is a technique to reduce the sampling of input data. For an image, this involves descreasing the number of pixels, and thus decreasing the resolution of the image.

Effect Size

How “strong” a phenotype is, or how mathematically possible it is to distinguish a given population from the control population.

Embedding Space

Embedding space is also know as a latent space or latent feature space. This space is a set of variables that describe items such that similar items are positioned closer together in the space.

Encoder-Decoder

An encoder-decoder architecture refers to a network with two parts. The “encoder” component of the network is a series of layers that take the input and map it to an abstract representation. The second “decoder” component of the network is trained to recreate input data from the abstract representation. This type of architecture learns an efficient way to represent the input data and is often used for dimensionality reduction.

End-to-end learning

A deep learning approach in which all stages of a task (e.g., locating and identifying objects) are optimized jointly within a single model, rather than being handled as separate, independent steps.

Ensemble

A machine-learning approach that combines predictions from multiple individual models to produce a single result that can be more robust than any single model alone.

Epoch

One complete pass through the entire training dataset during the training process.

Event

A set of spatial features that change or transitions over time, temporally defined by a beginning and an end.

Event detection

The algorithmic identification of temporal changes or transitions in a system, requiring analysis of data across time.

F1 Score

A classification metric that gives the harmonic mean of precision and recall. This metric was originally designed for binary classification but can be adapted to multiclass classification by calculating the F1 score per class. \[ \text{F1} = \frac{2\,\text{TP}}{2\,\text{TP}+\text{FP}+\text{FN}}\] This metric is also known as the DICE coefficient.

False Negatives

In a scenario where you have two classes “positive” and “negative”, you try to predict cases as one of those classes. False negatives are the cases that you incorrectly predicted as negative and were really positive.

False Positives

In a scenario where you have two classes “positive” and “negative”, you try to predict cases as one of those classes. False positives are the cases that you incorrectly predicted as positive and were really negative.

Feature vector

A set of numerical measurements (such as size, shape, intensity, or texture) that together provide a quantitative description of an object.

Fiji

An image processing platform that comes bundled with many plugins for scientific image analysis. See https://imagej.net/software/fiji/ for more information.

Fine-Tuning

Fine-tuning takes a pre-trained model and adapts it to a new task by providing task-specific training data. In classic fine-tuning, all layers of the pre-trained model are retrained, with their weights initialised from the original training. Thus, you continue training the full model using the new data. This approach allows the model to adjust more comprehensively to new data while still preserving the advantages of pre-learned features.

Foundation Model

A foundation model is a model that was trained on a very large and very diverse dataset. Because of this diversity in training data, foundation models can be used for many downstream tasks or can be fine-tuned for specific purposes.

Frequency Domain

The representation of an image as a function of spatial frequency, obtained by transforming an image from the spatial domain using the Fourier transform.

Function Calling

Function calling is also called tool use. This is the ability for LLMs to invoke external tools, APIs, and software through structured interfaces rather than generating plain text.

Gaussian Process

A common surrogate model for optimization strategies such as Bayesian Optimization. A Gaussian Process is a non-parametric model that represents a probability distribution over possible functions. In the hyperparameter search scenario, the Gaussian Process models the probability of getting an objective function value based on some hyperparameters.

General Agency

General agency gives AI agents access to broad capabilities — a terminal, a file system, APIs, analysis libraries — and lets them determine the right sequence of actions for each task. Rather than pre-scripting workflows, the agent receives a goal (“segment and count the nuclei in this dataset”) and works out what tools to use, in what order, handling errors as they arise.

Generative Adversarial Networks (GANs)

A deep learning architecture where two neural networks, a generator and a discriminator, are trained in an adversarial process, enabling the generator to create synthetic data, such as realistic images, by learning to deceive the discriminator.

Genetic Algorithms

An optimisation method inspired by the principles of natural selection and genetics. It starts with a population of solutions. These solutions are combined through a process called crossover to produce new solutions (offspring). During this process, random changes or mutations may occur to introduce diversity. After crossover and mutation, a selection step chooses the best solutions from both the parent and offspring populations to form the next generation. This cycle repeats for a set number of generations or until a predefined goal or stopping criterion is met.

Gradient Descent

Gradient descent computes the gradient of the loss with respect to each weight. Moving the weights in the negative direction of the gradient reduces the loss for the given images or data points over which the loss is computed.

Ground Truth

Accurate data against which a model can be evaluated. Ground truth data is often manually annotated. The data type itself will vary depending on the task and evaluation. e.g. instance segmentation may be compared to ground truth object counts or masks.

Hallucinations

Outputs from a model that do not have a basis in the input data and may contain false or misleading information.

Harmonic Mean

A type of average of a set of numbers, calculated as the number of values divided by the sum of their reciprocals. Unlike the arithmetic mean, the harmonic mean is closer to the smaller of the values being averaged, so it penalizes cases where one value is much lower than the other. This makes it useful for combining two metrics, such as precision and recall, into a single score that only looks good when both metrics are good.

Hausdorff Distance

The Hausdorff distance measures how far apart two sets of boundary points are at their worst-case point. For boundary point sets \(A\) and \(B\):

\[d_H(A, B) = \max\!\left(\sup_{a \in A} \inf_{b \in B} d(a,b),\; \sup_{b \in B} \inf_{a \in A} d(a,b)\right)\]

A small Hausdorff distance means every point on one boundary has a close counterpart on the other.

Human-in-the-loop

Human-in-the-loop workflows use model predictions that are manually corrected for retraining. This iterative approach can be used to supervise the model output and improve accuracy.

Hyperparameters

The options you choose when training a machine learning model that affect the training process or the architecture of the model (e.g., learning rate, batch size, number of layers, training loss, etc.) are called hyperparameters. This term is used to differentiate them from the parameters (also known as weights) of the machine learning model.

Image Classification

A computer vision task where each image is associated with one class and the goal of this task is to correctly predict that class.

Image Restoration

The process of recovering clear, high-quality images from degraded raw data contaminated by blur, noise, or other distortions.

Image Translation

Image-to-image translation (or style transfer) is a generative AI technique which transforms an input image into the style of another image. In the microscopy context, this could manifest as a network which transforms an image of one modality (e.g. brightfield) into a different modality (e.g. DAPI staining of nuclei) in the same field of view.

Instance Segmentation

A segmentation task that not only separates objects from the background but also distinguishes between individual objects of the same type (e.g., separating touching cells one by one).

IoU

“Intersection over Union”. A segmentation metric that calculates the area of overlap between two segmentation masks divided by the area of their union.

Kernel

Convolutional kernels are also known as filters. They are small matrices that define the function for taking input pixels and creating an output image. They are often used for tasks like blurring or edge detection.

Large Language Model (LLM)

Large Language Models (LLMs) are a type of AI model that was trained on a vast amount of data. Users interact with LLMs using natural language to accomplish a variety of tasks.

Linear Layer

Linear layers are also known as fully-connected layers or dense layers. They are a set of input nodes (or neurons) that are each connected to every output node of the layer.

Localization

The task of determining the spatial position of an object or event within an image.

Logprobs

Logprobs is short for log probabilities. Taking the logarithm of a probability has practical computational advantages. In the AI context, logprobs refers to the probability of each token, given the previous token, and can be used as a measure of model confidence.

Long-short term memory (LSTM)

A type of recurrent neural network designed to retain information over longer sequences, addressing limitations of standard RNNs in capturing long-range temporal dependencies.

Loss function

A loss function (or cost function) is a formula for quantifying how much a model’s prediction deviates from the actual ground truth value. The loss function returns a single scalar number to quantify the loss. As training relies on gradient descent, the loss must be differentiable; the algorithm needs the slope of the loss with respect to every parameter to know which direction to step.

Manual Annotation

The process of manually labeling specific structures or objects in an image using drawing tools. Typically done in software like Fiji or Napari, this step is essential for creating ground truth data to train or evaluate machine learning models.

Mean Squared Error

Mean-squared error (MSE) penalizes the model proportionally to the squared difference between its prediction and the ground truth: \[\text{MSE} = \frac{1}{N} \sum_{i=1}^{N} (y_i - \hat{y}_i)^2\] MSE measures how far the prediction is from the target value, and does not require the output to be a probability — it works equally well when the model produces a continuous map such as a distance transform or a confidence score. Because it sums squared errors over all pixels, it is sensitive to the total extent of the error: predicting an entire missing cell as background accumulates far more MSE than a few wrongly-classified boundary pixels.

Metadata

Any data that provides additional information about other data. In bioimaging, examples include information about sample preparation, the imaging instrument, and image acquisition parameters.

Model Context Protocol (MCP)

The Model Context Protocol (MCP) is a standardized way for AI systems to connect to external tools and data sources. MCP gives any AI model a universal way to discover and use any compatible tool — analogous to how USB standardized device connectivity. For microscopy, MCP means that a single agent could potentially access a microscope control system, an image analysis library, a literature database, and a model repository through the same protocol.

Momentum

In the context of learning rate, momentum speeds up learning when the gradient is consistent over multiple iterations, which allows the training to avoid local minima.

Multilayer Perceptron

A sequence of fully-connected layers that are applied in series, each on the output of the previous one, is called a multilayer perceptron (MLP). This is the simplest example of a deep neural network.

N2N (Noise2Noise)

A supervised denoising method that trains a neural network on pairs of independently noisy images of the same scene, requiring no clean reference data but needing paired noisy inputs. See https://github.com/NVlabs/noise2noise for more information.

N2S (Noise2Self)

A self-supervised denoising method that trains a neural network assuming statistically independent noise across the image, requiring only single noisy images without paired clean data. See https://github.com/czbiohub-sf/noise2self for more information.

N2V (Noise2Void)

A self-supervised denoising method that trains a neural network to predict pixel values from noisy images by masking input pixels, requiring only single noisy images without paired clean data. See https://github.com/juglab/n2v for more information.

Natural Image

Natural images are images of the environment. This most commonly refers to color photographs of people, nature, objects, etc. Many machine learning researchers focus on natural images for tasks like self-driving cars, but microscopy images pose additional challenges for AI.

Network Architecture

The architecture of a machine learning model refers to the design and structure of the model, including choices in how individual components (such as preprocessing or feature extraction) are connected to each other.

Neural Network

Neural networks are a type of computational model that was inspired by the way that biological neural networks are structured.

Neural Network Unit

A neural unit is the processing element of a neural network. It also sometimes called a node or artificial neuron.

Nonlinear

If a system or function is non-linear, changes in its output are not proportional to the changes that are made to the input. Non-linear functions are incorporated into machine learning networks to allow for the solution of complicated problems because it allows for non-linear decision boundaries.

Nonlinear Problem

A mathematical problem where the governing equations or operations are nonlinear, meaning outputs are not linearly proportional to inputs.

Object

A discrete spatial feature within an image, spatially defined by properties such as size, shape, intensity, and location.

Object Detection

A computer vision task that identifies and locates individual objects within an image, typically by drawing bounding boxes around them. It provides both the category (what) and position (where) of each object.

Optimizer

An optimizer is a function that is used to adjust model parameters during training. By adjusting weights and biases, the optimizer attempts to minimize the loss function and thus make the model more accurate.

Padding

Padding involves adding extra layers of pixels around an image to avoid edge effects. Padding can be performed by simply setting the extra pixels to zero, or it can take into account the image itself (such as in mirror padding or replicate padding).

Panoptic Segmentation

A computer vision technique that is a combination of semantic segmentation and instance segmentation. It separates an image into regions while also detecting individual object instances within those regions.

Patches

In image processing, a patch refers to a small region or subset of pixels that are extracted from the larger image.

Perceptron

Perceptrons are linear weighted sums of inputs followed by a nonlinear activation function. They are used for binary classification, and can be combined to create more complex network architectures. Perceptrons are a specific type of an artificial neuron or node.

Pixel Classifiers

Machine learning models that classify each pixel in an image based on features such as intensity, texture, or local neighborhood. Commonly used in traditional workflows for segmentation or classification tasks.

Point Spread Function (PSF)

A mathematical function that describes how an imaging system blurs a point source.

Pooling Layer

A pooling layer aggregates information from many vectors into fewer vectors, i.e., it downsamples the image. The pooling layer does this using sliding windows across the image of maximum or average operations. This removes redundant information and reduces the size of the image for the following calculations.

Precision

A classification metric that measures the proportion of predicted positives that are correct: true positives divided by the sum of true positives and false positives. Precision answers “of everything the model called positive, how much actually was?” A model can achieve high precision by only making positive predictions it is very confident about, at the cost of missing some true positives (i.e., lower recall). \[ \text{Precision} = \frac{\text{TP}}{\text{TP}+\text{FP}}\]

Probability Vector

A probability vector is composed of values that indicate the probability of a particular variable (e.g., the probability that an image is a dog or cat). The values of a probability vector sum to one.

Quality Control Metric

Any metric that can be used to evaluate quality. It will vary depending on the task and data type. It can be binary (e.g. an image doesn’t have debris) or continuous (e.g. annotated object centroids are within 5 pixels of the ground truth centroids).

RCAN (residual channel attention network)

A deep learning-based method using residual learning and channel attention to improve image restoration tasks. See https://github.com/AiviaCommunity/3D-RCAN for more information.

ReLU

An activation function common in deep learning that outputs the input directly if it is positive, and outputs zero otherwise. That is, it is the ramp function where it equals x if x>0 and 0 if x<=0. This characteristic helps introduce non-linearity into the model and mitigate the vanishing gradient problem. ReLU stands for rectified linear unit.

Real-time analysis

An approach in which images are processed as the data is acquired. This is in contrast to a-posterior analysis, where the data is acquired before processing.

Recall

A classification metric that measures the proportion of actual positives that were correctly identified: true positives divided by the sum of true positives and false negatives. Recall answers “of everything that was actually positive, how much did the model find?” A model can achieve high recall by predicting positive more liberally, at the cost of more false positives (i.e., lower precision). Also known as sensitivity or the true positive rate. \[ \text{Recall} = \frac{\text{TP}}{\text{TP}+\text{FN}}\]

Receptive Field

The receptive field size of a convolutional layer is the total spatial extent of pixels that influence the activations of the layer. The receptive field of the entire network describes the size of the region in the input image that influences the output image.

Recurrent neural networks (RNNs)

Neural network architectures designed to process sequential data by maintaining an internal memory of previous inputs, used to model temporal dependencies.

Region of interest (ROI)

A defined area within an image or specimen selected for closer analysis, imaging, or targeted intervention.

Regularization

Regularization adds a penalty term to the loss function to prevent overfitting. This term discourages overly complex models.

ResNet

A residual neural network (ResNet) is a deep learning architecture in which skip connections between layers are used to learn residual mappings instead of direct transformations, increasing training efficacy and deeper networks.

Self-attention

Attention allows a model to determine the importance of each component of an input sequence. Self-attention is a type of attention mechanism that is commonly used in transformer architectures.

Self-supervised learning

A deep learning method where models generate their own supervisory signals from unlabeled data, often by using pretext tasks, to learn useful representations that can be applied to various downstream tasks.

Semantic Segmentation

A form of segmentation where each pixel in an image is assigned to a class (e.g., nucleus, cytoplasm, background), but it does not distinguish between separate instances of the same class.

Shallow decision trees

Simple, low-complexity decision tree models that make classification decisions using only a few splits, often used as building blocks within ensemble methods.

Shape descriptors

A set of quantitative measurements that characterize the geometric form of an object, such as circularity, elongation, or perimeter.

Sigmoid Function

An activation function common in deep learning that non-linearly maps real inputs to outputs between 0 and 1, being most sensitive to changes in inputs around zero and increasingly compressing extreme positive or negative inputs as they approach 1 or 0 respectively; this characteristic enables it to model probabilities for binary classification and introduce smooth non-linearity.

Single-shot detection

An object detection approach in which localization and classification are performed simultaneously in a single pass through the model, rather than through separate sequential stages.

Skills Atrophy

Skills atrophy is the decrease or loss of human expertise due to underuse. In this context, skills atrophy could be caused by an over-reliance on AI tools. This gradual erosion of the community’s ability would decrease the field’s ability to evaluate AI outputs.

Skip Connection

Skip connections are also sometimes know as residual connections. These connections are made when the output of a layer of the model is added to or concatenated with a later layer in the network while bypassing any layers that may be in between. For example, the layers in a U-Net architecture connect across image resolutions.

Smart microscopy

An imaging approach in which acquisition parameters are automatically adjusted based on features or events recognized in the data, rather than relying solely on manual operator control.

Spatial Domain

The representation of an image as a function of spatial coordinates.

Star-convex Polygon

A geometric shape used in segmentation algorithms like StarDist. Imagine drawing straight lines (rays) from the centre of an object out toward its edges — if you can see the edge from the centre in all directions, the object is considered star-convex. This method works well for blob-like structures such as nuclei, because their general shape can be captured by measuring how far each ray travels from the centre to the boundary.

Stride

When performing a convolution or cross-correlation, the kernel (aka the filter) moves across the entire image. When the stride is 1, this corresponds to moving the filter across every pixel in the image. For larger values of the stride, the kernel is moved more than one element at a time. For example, a stride of 2 would skip every other pixel in the image.

Supervised learning

A deep learning method where models learn from labeled data (input-output pairs), enabling them to learn a mapping function for making predictions or decisions on unseen inputs.

Support vector machine (SVM)

A supervised machine-learning algorithm that learns the decision boundary that best separates data points of different classes by maximizing the margin between them.

Test Data

Data held out from both the training data and validation data, used only after training is complete to give an unbiased estimate of how a model will perform on new, unseen data. Because it plays no role in training or hyperparameter tuning, test data should only be used once a model is finalized.

Texture

A quantitative descriptor of the spatial pattern of intensity variation within a region of interest, capturing properties like smoothness or granularity.

Tokens

Tokens are units of data processed by an AI model. For an LLM, these tokens are chunks that may be whole words or subword pieces. For instance, “microscope” might be a single token, while “fluorescence” might be split into “fluor” + “escence.”

Tool Definitions

Tool definitions are structured descriptions of available functions, their parameters, and expected inputs. When a user makes a request, the model reasons about which tools are needed, generates structured calls with correct parameters, receives results, and decides whether to take further action or report findings.

Training Data

Data used to teach a machine learning model to perform a task, such as classification, segmentation, restoration, or generation. Depending on the method, training data may be paired with target labels or ground truth (as in supervised learning) or used on its own without labels (as in self-supervised or unsupervised learning).

Transfer Learning

A deep learning technique that reuses a model pre-trained on one task as the starting point for a new, related task, leveraging its learned knowledge to improve performance or reduce training requirements. In practice, part of a pretrained neural network (usually the initial layers, responsible for feature extraction) is frozen and reused in a new model. These frozen layers, with the knowledge from a previous dataset, are combined with untrained layers tailored for a specific bioimaging task. During training, only the new layers will be updated, allowing the model to adapt to the new task with limited data.

Transformer Models

A deep learning architecture based on the multi-head attention mechanism; specifically referring to the ‘vision transformer’ architecture. A vision transformer (ViT) is a transformer designed for computer vision. A ViT decomposes an input image into a series of patches (rather than text into tokens), serializes each patch into a vector, and maps it to a smaller dimension with a single matrix multiplication. These vector embeddings are then processed by a transformer encoder as if they were token embeddings.

True Negatives

In a scenario where you have two classes “positive” and “negative”, you try to predict cases as one of those classes. True negatives are the cases that you predicted as negative and were really negative.

True Positives

In a scenario where you have two classes “positive” and “negative”, you try to predict cases as one of those classes. True positives are the cases that you predicted as positive and were really positive.

U-Net

A U-Net is a type of convolutional neural network that was specifically developed for the task of image segmentation. It has since been used for a variety of tasks. The model is so named because downsampling layers are followed by upsampling layers, which are each connected by skip connections, and together this gives diagrams of the network a U shape.

Upsampling

Upsampling is a technique to increase the sampling of input data. For an image, this involves increasing the number of pixels and interpolating values for the newly sampled pixels.

Validation Data

Data set aside from the training data and used to evaluate a model’s performance during training, without being used to update the model’s weights. It is typically used to tune hyperparameters and monitor for overfitting, for example by deciding when to stop training.

Virtual Machine

On a physical computer, you install an operating system (e.g., Windows or Ubuntu) that you interact with. A virtual machine is a program that simulates a complete computer with its own operating system. This lets you run a “computer inside your computer” (e.g., using Linux inside Windows or the other way around). As this simulated computer is separate from your physical one, it adds an extra layer of security, because unless the user specifically allows it, the virtual machine cannot access or connect to your real computer.

Vision-Language Model (VLM)

A Vision-language Model (VLM) is a type of model that can process both images and text with a single architecture. Because they can accept both natural language and visual inputs, they can include context in their analysis.

Visual Question Answering

Visual Question Answering (VQA) is when a model is asked an open-ended question based on an image input. The result is a natural language response to a natural language question associated with an image.

Zernike Modes

A set of orthogonal polynomials used to describe and correct wavefront aberrations in optical systems.