Biology Computational Part 22 Biological Imaging I
E. Glory, G. Steven Vanni Meel Velliste, Robert F. Murphy Copyright 1998, 2000-2007. All rights reserved.
Biological imaging
s
Significant advances in the fields of optics and electronics in the past two decades have greatly increased the utility of imaging for addressing biological questions. s These advances permit
x more diverse types of information to be
extracted from biological specimens x with greater accuracy x and under more demanding conditions.
Chemical and molecular biological probes may be targeted within a specimen
s
s
Imaging relies on generating a detectable signal which can be used as a measure of a property of interest in the specimen. This property of interest is the initial signal, but it must be transduced or changed through several forms before it becomes detectable.
Chemical and molecular biological probes may be targeted within a specimen
s
For example: A protein may be modified so that when it enters a cell and bumps into another protein involved in a specific activity, it fluoresces. The original activity was probably not detectable, but this newly generated fluorescence signal is detectable.
Front-end of imaging system and detector
Specimen Specimen may be difficult to see except where labeled by probe.
Image Formation and Acquisition
s
s
A digital image plane is acquired by recording a digital value proportional to the intensity of light (or other form of energy) impinging on each pixel of a detector This intensity usually corresponds to the amount of light emitted by or reflected from a corresponding point on a specimen
0 0 0 0 0 0 0 3 6 2 0 0 0 7 8 8 4 0 1 8 8 8 8 2 0 3 8 8 8 1 0 0 2 4 3 0 0 0 0 0 0 0
Specimen
Projection of specimen onto dectector grid
Image Pixel
Image Formation
s
Biological images may be acquired via a variety of imaging modes or modalities s Each mode is a combination of an image formation system and a detector
Sample
Image formation system
Detector
While the examples so far have dealt with light microscope images, we will now back up for a few minutes to consider many different types of images before concentrating on light microscopy.
Detector and image types
In general, images may be classified according to what is being detected: s (Visible) light transmission, scattering or emission
3
single wavelength, 3 color, or full spectrum
s s s s s
Electron transmission or scattering X-ray transmission Radioactive particle emission Magnetic field perturbation Physical displacement from "atomic force"
Comparing types of imaging
Method Light Electron Medical X-ray X-ray Diffraction
Autoradiography
Resolution Living (nanometer) specimen? 200 or better Yes 10 No 1000 or Yes better 0.1 No 10 5000 1 1 No Yes No No
Functional MRI/NMR Structural NMR AFM
Light microscopy
s
Three primary types of detectors
x human eye
3
no digital image
x CCD or charge coupled device
3
.
"work horse" of modern biological imaging 3 acquires digital image directly a CCD chip is the actual detector within a CCD camera 1 penny
x PMT or photomultiplier tube
Arc Lamp Excitation Diaphragm Excitation Filter
Fluorescence Microscope
Ocular
Objective Emission Filter
Light sources in the object
s
Consider a fluorescent specimen made of individual molecules of fluorescent dye.
x Each molecule can emit light. x Each dye molecule may be seen as a vanishingly
small emitter. x Such an emitter is called a point source. x The concept of a point source is useful because a point is simple to model, and if we know how a point source is imaged, then we can easily model a complex specimen as a combination of many points and predict how it will be imaged.
Light sources in the object
s
A specific example might be a microscope slide containing cells stained with fluorescent dye. s In an ideal image, a point source would show intensity in only one pixel
Point-spread function
s
s
s
In reality, the light from each point in the specimen is seen to spread out and affect many pixels in the image. The mathematical description of this spreading or blurring process is called a point-spread function (PSF) The point-spread function (PSF) is determined by the optics of the image formation system, including factors such as the refractive index, diameter and magnification of its components
Realistic Image of a Point Source
s
The resulting blurred region in the image can be approximated by a 2D Gaussian distribution This graph shows intensity on the z-axis for a PSF defined in the X-Y plane. Later we will consider a PSF defined in three dimensions.
s
s
Light sources in the object
s
Thus, when a 2D image is acquired, each point in the specimen will be blurred in all directions and will contribute to the recording in many pixels around that pixel to which it directly corresponds
Introduction to 3D Microscopy
s
The spreading of light from a point source actually occurs in three dimensions as will be shown. s First, however, it is necessary to understand the three dimensional (3D) nature of the object and image as acquired via 3D microscopy.
3D Microscopy
s
Detector
When a microscope is focused on a specimen, the detector records an image from a plane. x This is the focal plane. x Parts of the specimen in the focal plane are in the best focus.
Focal plane
s
s
3D data is acquired by combining data from several different focal planes into a stack of images. This is accomplished by changing the distance between the specimen and the microscope's objective lens from one image acquisition to the next.
Image stack Objective
Real 3D image data
s
The next slide shows a real 3D image stack. s The specimen is a HeLa cell labeled with a antibody against the cytoskeletal protein tubulin and a secondary antibody conjugated to a fluorescent dye. s The images were acquired using a confocal fluorescence microscope. s The image stack is presented here as a movie with one acquired image plane per movie frame.
Microtubules in a human cell
QuickTimer and a decompressor are needed to see this picture.
Courtesy of Meel Velliste
Real 3D image of a point source
s
Now, with a better understanding of what makes up a 3D image stack, we can better consider how light from a point source spreads out and is imaged in three dimensions. 3D Reconstruction of Point Spread Function (PSF) from 0.2 Micron Bead
y x
z x Courtesy of Image & Graphics Inc.:
Increasing intensity http://www.imagepro.co.kr/ NOTE: Spreading along the Z-axis is more pronounced.
Image Formation
s
Image formation can be described as:
x
the convolution of an array describing the original specimen or object x with a function describing the image formation system x to yield an acquired image.
The concept of a convolution
s
A convolution may be written in somewhat simplified mathematical form as follows:
i(x,y,z) defines the image in its 3D space according to the form of the equation above. s PSF(x-x',y-y',z-z') defines the amount of light from a point source at x',y',z' that will be observed at x,y,z s o(x',y',z') describes the specimen or object.
s
Image Formation
s
The mathematical view of convolution emphasizes that each point in the sample can contribute to each point in the image
The widefield fluorescence microscopy collects light emitted from all points in the specimen (with varying efficiencies depending on position relative to focal plane) The result for specimens that are thick relative to the depth of focus of the objective is a blurred image
Confocal Microscopy
s
One way to obtain images that better represent the fluorescence distribution just in the focal plane is to use a confocal microscope
Laser Excitation Pinhole
Confocal Microscope Principle
Objective
Excitation Filter
PMT
Emission Filter Emission Pinhole
http://micro.magnet.fsu.edu/primer/confocal/index.html
Image Formats
s
An bit map image normally consists of an 8-bit or 16-bit value for each pixel. s These values are stored as computer files in various formats. s Pixel values are normally stored linearly in a file with the values for the first row of pixels followed immediately by the values for the second row (etc.).
Image Formats
s
At a minimum, an image format contains:
x Image size (# of rows and columns) x Number of bits per pixel x Order in which bytes within words are stored x Number of bytes to skip at the beginning of the
image (the offset)
3
The beginning of image files often has a text header that can be skipped if the above values are known. 3 This header may contain additional descriptive information about the image such as:
subject of image name of person and/or application creating the image
Common Image File Formats
s
TIFF (Tag Image File Format) x Originally for scanners and frame grabbers
x x
Used extensively on many platforms Can be read/written by ImageJ (NIH Image)
x Supports lossless compression
Reference: www.shortcourses.com/chapter07.htm
Common File Image Formats
s
JPEG ("jay-peg" Joint Photographic Experts Group)
x
x
x x
x
Originally referred to a compression method but now refers to the associated file format with or without compression Most common World Wide Web file format 3 Supports progressive display where an image is first displayed at low resolution and then at higher resolution. Uses a lossy compression technique Optimized for storing photographs and not as good for line art Supports 24-bit color
Reference: www.shortcourses.com/chapter07.htm
Common Image File Formats
s
GIF ("jiff" Graphics Interchange Format)
x Also widely used on the Web
3
Supports progressive display
x Mostly used for line art as opposed to
photographs x Only supports 8-bit color
Reference: www.shortcourses.com/chapter07.htm
How to use Images: Outline
s
Image Display s Image Processing s Image Analysis s Image Interpretation Software : ImageJ http://rsb.info.nih.gov/ij/
From Images to Knowledge
Image Image Processing Image
Image
Image Analysis
Numbers
Image
Image Interpretation
Knowledge
Image Display
s
Operations that change display without changing image
x LUT - grayscale or color x Contrast stretching
s
Operations that change image
x reversible x non-reversible (majority)
Image Display
s s
A pixel value is just a number in the data set representing a digital image. Pixel values may be displayed in different ways, determined by a look up table (LUT).
0 0 0 0 0 0 0 3 6 2 0 0 0 7 8 8 4 0 1 8 8 8 8 2 0 3 8 8 8 1 0 0 2 4 3 0 0 0 0 0 0 0
LUT 7-8 4-6 1-3 0 LUT 1-8 0 Hot to cold
LUT 7-8 4-6 1-3 0
Pixel values
Arbitrary
Binary
Image Display - LUT change
Image Display - Enhance contrast
Original (before contrast enhancement)
After enhancement uses full range
Thresholding
Gray-level image
s
Binary image
Thresholding refers to the division of the pixels of an image into two classes: those below a certain value (the threshold) and those at or above it. The two classes are often shown in white and black, respectively. s Thresholding serves as a means to consider only a subset of the pixels of an images.
Thresholding
s
The choice of threshold must be made empirically by considering the nature of the sample, the type and number of objects expected in the image, and/or a histogram of pixel values s The threshold can be specified as a multiple of the background value (normally the most common value) for partial automation
Ridler-Calvard Method
s
Find threshold that is equidistant from the average intensity of pixels below and above it s Ridler, T.W. and Calvard, S. (1978) Picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man, and Cybernetics 8:630-632.
Ridler-Calvard Method
Blue line shows histogram of intensities, green lines show average to left and right of red line, red line shows midpoint between them or the RC threshold
RidlerCalvard Illustration
0.25 0.2 0.15 0.1 Frequency 0.05 0
0
20
40 Pixel Value
60
80
Ridler-Calvard Method
original
thresholded
Thresholding
Thresholding
s
Once a threshold has been applied, the resulting image may be
x displayed in black and white x displayed with above threshold pixels at their
original intensities and below threshold pixels in blue
Thresholding
Process/Binary/Threshold does auto threshold and applies it to make binary image
Binary image operations
Binary image structuring element (clean) Binary image
Binary image operations
s
Erosion
x Remove pixels from edges of objects x Set "on" pixel to "off" if at least one of its eight
neighbors is white given this structure element:
erosion Original (before erosion) Result (after erosion)
Binary image operations
s
Dilation
x Add pixels to edges of objects x Set "off" pixel to "on" if at least one of its
neighbors is black given this structure element:
dilation Original (before dilation) Result (after dilation)
Binary image operations
s
Open
x Smooth objects and fill in small holes x Erosion followed by dilation
erode
dilate
open
Binary image operations
s
Close
x Smooth objects and fill in small holes x Dilation followed by erosion
dilate
erode
close
Binary image operations
erode
dilate
open
close
Binary image operations
s
Outline
x Find "on" pixel, trace around outside until
return to first "on" pixel
Binary image operations
s
Skeletonize
x Remove pixels from the edges of objects until
the objects are one pixel wide
Basic Image Processing Operations
s
Image Math s Kernel/Filter Operations s Image Calculator
Arithmetic Operations
s
Two cases:
x Perform a single operand operation (e.g.,
logarithm, square root) on each pixel of an image x Perform a dual operand operation (e.g., add, multiply) on each pixel of an image using a constant as the second operand
s
In both cases, the result is usually stored in the same pixel location ("storing in place")
Arithmetic Operations
Kernel/Filter Operations
s
Basic idea: Use a matrix (usually square and of odd dimension, e.g., 3x3) in combination with an image to generate a new image s Algorithm:
x For each pixel in the image (the current pixel) x Align the matrix to center it on that pixel x For each position in the matrix, multiply the
corresponding pixel value in the image by the value in the matrix and...