Unformatted text preview: Introduction to Deconvolution Image Processing Andres Kriete Outline Convolution: 2D: Airy Disk (Light Point Spread Function) Convolution with an Image 3D: 3D Point Spread Function Convolution with a Volume Deconvolution: Frequency space decomposition Fourier Transforms Application of the Deconvolution Theorem Some Examples: Image Restoration Example The Microscope Optical Train is Complex Every lens element alters the image in some way We call this the image transfer function: h(x,y,z) The image transfer function of the system is a convolution of all of the image transfer functions of all the lens elements and apertures in the optical train The Definition of Convolution:
Convoluted: adj.1.Having numerous overlapping coils or folds: a convoluted seashell.2.Intricate; complicated: convoluted legal language; convoluted reasoning.
The American Heritage Dictionary of the English Language, Fourth Edition Formation of an Airy Disk Pattern
x 10
6 x 10 6 6 1 4 0.5 2 0 0 2
0.5 4
1 6
1 0.5 0 0.5 1 x 10
6 6 4 2 0 2 4 6 x 10
6 A point in the object space A point convolved with the transfer Microscope function Impulse Response Function [ Point Spread Function (psf)] Impulse Response Function Applied to a line
A great advantage is afforded by the ability to express the response of the optical system to an arbitrary input in terms of the response to certain "elementary" functions into which the input has been decomposed Line Transfer Function How the psf (Impulse Response) Effects Resolution slide courtesy of Edgar Garduno Image Transfer Function h(x,y) Object Image iimage (x, y) = iobject (xi , y j ) h(x  xi , y  y j )
i j Theoretical Model of the Impulse Transfer Function, h(x,y) (PSF) 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0.1 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 x 10 1
6 h r; NA, = ( ) 2 J1 r r ( ) where = 2 ( NA) J1=Bessel function of the first kind The Definition of Convolution:
Convoluted: adj.1.Having numerous overlapping coils or folds: a convoluted seashell.2.Intricate; complicated: convoluted legal language; convoluted reasoning.
The American Heritage Dictionary of the English Language, Fourth Edition Convoluted Mess! Spatial Frequency Decomposition Fourier Transform
0.25m myelin Any image can be decomposed into a series of sines and cosines added together to give the image I(x) = ai (cos k i x) + ibi (sin k i x)
i Amplitudes Phase Fourier Transform
50 0 50 100 150 200 250 300 Pixel Fourier Transform of the Myelin Image Low frequency High frequency Mathematical Formulation of the Fourier Transform
2D Fourier Transform: (g) =   g(x, y)e 2 i( f x x + f y y) dxdy Real Space to..Frequency Space Inverse 2D Fourier Transform: 1 (G) =   G( fx , fy )e 2 i( f x x + f y y) dfx dfy Frequency Space to.. Real Space Inverse Fourier Transform of the Fourier Transform Returns the original Image Fourier transform of myelin F 1 = Very Powerful Tool ! The Wonderful, Great and Amazing Convolution Theorem If
{g(x, y)} = G( fx , fy ) and
{h(x, y)} = H ( fx , fy ) Then: {g(x, y)h(x, y)} = G( fx , fy )H ( fx , fy )
Remembering that the image intensity is a convolution of the impulse function h(x,y) and the object Iobject(x,y)
iimage (x, y) = iobject (xi , y j ) h(x  xi , y  y j )
i j The object intensity can be easily deconvolved from the Smear of the impulse function (PSF) Deconvolution using the Convolution Theorem The image is a convolution of the impulse function h(x,y) (PSF) and the object (the sample):
iimage (x, y) = iobject (xi , y j ) h(x  xi , y  y j )
i j Therefore, the Fourier transform of the image is just the Fourier transform of object times the Fourier transform of the impulse function (PSF) in frequency space {iimage } = {iobject (x, y) h(x, y)} = I obj ( f x , f y )H ( fx , fy )
So, to obtain the object, we simply divide by the Fourier transform of the impulse function (PSF) I obj ( fx , fy ) = I obj ( fx , fy )H ( f x , f y ) H ( fx , fy ) Transform back to real space
I obj ( fx , fy ) = I obj ( fx , fy )H ( fx , fy ) H ( fx , fy ) Finally, we obtain the deconvolved object (sample) by applying an inverse Fourier transform I obj ( f x , fy )H ( fx , fy ) iobject (x, y) = {I obj ( fx , fy )} = H ( fx , fy ) 1 1 A simple division in frequency space yields the Object intensity ! Let's Run though it... Step 1: Acquire the image and it's Fourier transform Convoluted Image Fourier Transform Steps to Deconvolution... Step 2: Obtain the impulse function (PSF) and the Fourier transform of the impulse function (PSF) Point spread function Fourier transform Steps in the Deconvolution... Step 3: Divide and inverse Fourier transform F 1 Convolution and Deconvolution 0.1m fluorescent bead the object deconvolution (+psf otf) convolution (objective lens) Planes of focus of object (bead) infocus light z y observed image out of focus light (airy rings) x Planes of focus of observed image Convolution = the way the microscope optics "distort" the observed image of an object 3D Outoffocus Point Spread Function 3D Impulse Response z x y 2D PSFs vs. zheight The PSF  Point Spread Function 3D  PSF z x z y x z y x y z x y x x40 NA 0.85 Dry / 200 nm fluorescent bead PSF = Point Spread Function = a measure of the convolution caused by the microscope optics A PSF can be determined empirically by imaging a subresolution fluorescent bead Increase in resolution (XY and Z) after deconvolution
Z X Y intensity (in gray scale values) a a xy focal plane b
6000 z axis 6000 b 5000 5000 4000 4000 3000 3000 0.18
2000 2000 0.34 0.65
1000 0 30 25 observed deconvolved fwhm 0.26
1000 0 5 15 20 10 0 c d
5 10 20 25 x or y (pixels) z (pixels) 15 30 0 The PSF and OTF
Real space Frequency space OTF image (API, softWoRx) Cylindrically averaged OTF "Missing Cone" kz 3D  PSF "Missing Cone" 0 z y
kr x 0 0 kz kx kfrequency (Reciprocal space) To simplify deconvolution calculations the OTF is often simplified by assuming radial symmetry for the PSF The "missing cone" represents frequency information not collected by an objective with a limited NA (not all light can be gathered) Deconvolution example From: http://micro.salk.edu/dv/dv.html Deconvolution Methods
Linear methods, computational expense low Nearest neighbor
J(I) = [ o(j) c (( o(j1) + o(j+1) ) * h) ] *g with h for blurr, g for inverse filter of central slice. Simple to execute, but SNR is decreased, c is an estimate
Nonlinear methods, iterative, FFT Leastsquares (Gaussian) Maximum Likelihood (Poisson) Maximum Entropy (Gaussian)
Measured PSF versus blind deconvolution Deconvolution and Noise Microscope = object NOISE Image PSF Good image Noisy image (poor S/N) Much of the complexity of deconvolution is a direct consequence of having to deal with "noise" inherent in image data Estimation of Noise
Too pessimistic estimate of noise (deconvolved image Smooth but less detailed) Too optimistic estimate of doise (deconvolved image generates artifact like structures on left). Source: From SVI webpage Constrained Iterative deconvolution Noisy, Raw image compare outcome with raw data generate "guess" Expectation is Outcome equivalent to raw image constrain and noise filter convolve with PSF Iterations: Modify "guess" and repeat until convolved outcome approaches original image "Initial Guess" at true image Best Guess =Deconvolution Result Deconvolution Results Good raw image Poor S/N image Deconvolution results Problems in Image Restoration
Users applying image restorations are concerned about suitable measures of image quality: Different implementations show different results Some procedures show semiconvergence Stable algorithms can turn into unstable conditions if taken to a high number of iterations (Snyder & Miller, 1987) Methods I Methods II Data requirements for deconvolution
Ensure that the imaging system is correctly set up, aligned and calibrated (pixel sizes). Reduce SA if possible. Sample preparation and correction of optics Collect more light / average images Aim for a high S/N in image data. Make sure that images are collected according to the Nyquist sampling criteria (pixel size in XY and Z step). Collect sufficient image planes in Z. (2D data can be deconvolved but lacks Z information so restoration is limited.) Minimise lamp flicker between Z sections. (corrected for on the DV system) Avoid motion blur from live specimens. (short exposure times) Confocal vs Widefield Deconvolution
Confocal (optical configuration) Discards out of focus light using a pinhole in the light path Less sensitive  throws away light, generally poorer signal to noise Deals well with strong but diffuse signal with a lot of out of focus light (low contrast) More convenient  immediate high contrast images, even with single Z sections Widefield Deconvolution (processing) Reassigns out of focus light to its point of origin More sensitive (and quantitative) Better signal to noise ratio Better for point sources of light and weak signals Less convenient  requires time consuming calculations on expensive computers, best with multiple Z sections. Confocal images can be deconvolved as well Confocal vs Widefield Deconvolution
Widefield WfDecon 3D iter Confocal n n n n Live Drosophila oocyte, Microtubules labelled with TauGFP Confocal and widefield deconvolution have different strengths and weaknesses Deconvolution of Space Telescope Images Deconvolution to Reduce Motion Blur Deconvolution Software Execution
http://www.mediacy.com/Tutorials/AQI_quick_demo/AQi _quick_demo.html More Reading: http://www.mediacy.com/pdfs/BiophotonicsDeconvolutio n.pdf PSF and Nyqist calculator: http://bigwww.epfl.ch/deconvolution/?p=plugins http://support.svi.nl/wiki/NyquistCalculator Image J plugin: http://bigwww.epfl.ch/algorithms/deconvolutionlab/ References
Protein Localization by Fluorescence Microscopy, A practical Approach Ed. V.J. Allan OUP ISBN 0199637407 Chapter by Ilan Davis: How to make and use fluorescent bead slides and how to correct spherical aberration. Wallace, W., Schaefer, L. H., Swedlow, J. R. (2001) A working person's guide to deconvolution in light microscopy. Biotechniques 31, 1076 Swedlow, J. R., Hu, K., Andrews, P. D., Roos, D. S., Murray, J. M. (2002) Measuring tubulin content in Toxoplasma gondii: a comparison of laserscanning confocal and widefield fluorescence microscopy. PNAS 99, 2014 Hiroaka, Y., Sedat, J., Agard, D. (1990). Biophysics Journal 57 325. Determination of threedimentional imaging properties of a light microscope system, Partial confocal behaviour in epifluorescnece microscopy Parton, R. and Davis, I. (2005) Lifting the fog: Image Restoration by Deconvolution. Chapter 19, in Cell Biology, a laboratory handbook. Editor: J.E.Celis. Academic Press. P187200 ...
View
Full Document
 Spring '11
 KOP
 LTI system theory, deconvolution

Click to edit the document details