The small correction volume for conventional wavefront shaping methods limits their application in biological imaging through scattering media. In this paper, we take advantage of conjugate adaptive optics (CAO) and remote focusing (CAORF) to achieve three-dimensional (3D) scanning through a scattering layer with a single correction. Our results show that the proposed system can provide 10 times wider axial field of view compared with a conventional conjugate AO system when 16,384 segments are used on a spatial light modulator. We demonstrate two-photon imaging with CAORF through mouse skull. The fluorescent microspheres embedded under the scattering layers can be clearly observed after applying the correction.
We developed a three-photon adaptive optics add-on to a commercial two-photon laser scanning microscope. We
demonstrated its capability for structural and functional imaging of neurons labeled with genetically encoded red
fluorescent proteins or calcium indicators deep in the living mouse brain with cellular and subcellular resolution.
Marc Reinig, Samuel Novak, Xiaodong Tao, Laurent Bentolila, Dustin Roberts, Allan MacKenzie-Graham, Sirie Godshalk, Mary Raven, David Knowles, Joel Kubby
Our ability to see fine detail at depth in tissues is limited by scattering and other refractive characteristics of the tissue. For fixed tissue, we can limit scattering with a variety of clearing protocols. This allows us to see deeper but not necessarily clearer. Refractive aberrations caused by the bulk index of refraction of the tissue and its variations continue to limit our ability to see fine detail. Refractive aberrations are made up of spherical and other Zernike modes, which can be significant at depth. Spherical aberration that is common across the imaging field can be corrected using an objective correcting collar, although this can require manual intervention. Other aberrations may vary across the imaging field and can only be effectively corrected using adaptive optics. Adaptive optics can also correct other aberrations simultaneously with the spherical aberration, eliminating manual intervention and speeding imaging. We use an adaptive optics two-photon microscope to examine the impact of the spherical and higher order aberrations on imaging and contrast the effect of compensating only for spherical aberration against compensating for the first 22 Zernike aberrations in two tissue types. Increase in image intensity by 1.6× and reduction of root mean square error by 3× are demonstrated.
This paper will review the development of wide-field and confocal microscopes with wavefront sensing and adaptive optics for correcting refractive aberrations and compensating scattering when imaging through thick tissues (Drosophila embryos and mouse brain tissue). To make wavefront measurements in biological specimens we have modified the laser guide-star techniques used in astronomy for measuring wavefront aberrations that occur as star light passes through Earth’s turbulent atmosphere. Here sodium atoms in Earth’s mesosphere, at an altitude of 95 km, are excited to fluoresce at resonance by a high-power sodium laser. The fluorescent light creates a guide-star reference beacon at the top of the atmosphere that can be used for measuring wavefront aberrations that occur as the light passes through the atmosphere. We have developed a related approach for making wavefront measurements in biological specimens using cellular structures labeled with fluorescent proteins as laser guide-stars. An example is a fluorescently labeled centrosome in a fruit fly embryo or neurons and dendrites in mouse brains. Using adaptive optical microscopy we show that the Strehl ratio, the ratio of the peak intensity of an aberrated point source relative to the diffraction limited image, can be improved by an order of magnitude when imaging deeply into live dynamic specimens, enabling near diffraction limited deep tissue imaging.
In this paper, we demonstrate a fast binary intensity modulation based on the measurement of the binary TM. For each correction, the binary TM was calculated based on measurements of the intensity change at the target with a series of input masks. After preloading the measurement masks, the DMD can run at full speed during measurement. The system allows dynamic focusing at 12.5 Hz with 1024 input modes, and more than 60 times intensity enhancement. We demonstrate focusing light through a highly dynamic scattering sample, a live drosophila embryo.
Optical sectioning of biological tissues has become the method of choice for three-dimensional histological analyses.
This is particularly important in the brain were neurons can extend processes over large distances and often whole brain
tracing of neuronal processes is desirable. To allow deeper optical penetration, which in fixed tissue is limited by
scattering and refractive index mismatching, tissue-clearing procedures such as CLARITY have been developed.
CLARITY processed brains have a nearly uniform refractive index and three-dimensional reconstructions at cellular
resolution have been published. However, when imaging in deep layers at submicron resolution some limitations caused
by residual refractive index mismatching become apparent, as the resulting wavefront aberrations distort the microscopic
image. The wavefront can be corrected with adaptive optics. Here, we investigate the wavefront aberrations at different
depths in CLARITY processed mouse brains and demonstrate the potential of adaptive optics to enable higher resolution
and a better signal-to-noise ratio. Our adaptive optics system achieves high-speed measurement and correction of the
wavefront with an open-loop control using a wave front sensor and a deformable mirror. Using adaptive optics enhanced
microscopy, we demonstrate improved image quality wavefront, point spread function, and signal to noise in the cortex
of YFP-H mice.
KEYWORDS: Wavefronts, Wavefront sensors, Adaptive optics, Two photon excitation microscopy, Microscopes, Two photon imaging, Fluorescent proteins, Green fluorescent protein, Objectives, Control systems
A fast direct wavefront sensing method for dynamic in-vivo adaptive optical two photon microscopy has demonstrated.
By using the direct wavefront sensing and open loop control, the system provides high-speed wavefront measurement
and correction. To measure the wavefront in the middle of a Drosophila embryo at early stages, autofluorescence from
endogenous fluorophores in the yolk were used as reference guide-stars. This method does not rely on
fluorescently labeled proteins as guide-stars, which can simplify the sample preparation for wavefront measurement. The
method was tested through live imaging of a Drosophila embryo. The aberration in the middle of the embryo was
measured directly for the first time. After correction, both contrast and signal intensity of the structure in the middle of
the embryo was improved.
Optical microscopy allows noninvasive imaging of biological tissues at a subcellular level. However, the optimal performance of the microscope is hard to achieve because of aberrations induced from tissues. The shallow penetration depth and degraded resolution provide a limited degree of information for biologists. In order to compensate for aberrations, adaptive optics with direct wavefront sensing, where guide-stars are used for wavefront measurement, has been applied in microscopy. The scattering effect limits the intensity of a guide-star and hence reduces the signal to noise ratio of the wavefront measurement. In this paper, we propose to use interferometric focusing of excitation light onto a guide-star embedded deeply in tissue to increase its fluorescence intensity, thus overcoming the signal loss caused by scattering. With interferometric focusing of light, we increase the signal to noise ratio of the laser guide-star through scattering tissue by more than two times as well as potentially extending the thickness of tissue that can be corrected using AO microscopy.
Optical aberrations due to the inhomogeneous refractive index of tissue degrade the resolution and brightness of images
in deep tissue imaging. We introduce a direct wavefront sensing method using cellular structures labeled with fluorescent
proteins in tissues as guide-stars. As a non-invasive and high-speed method, it generalizes the direct wavefront sensing
method for adaptive optics microscopy. An adaptive optics confocal microscope using this method is demonstrated for
imaging of mouse brain tissue. The confocal images with and without correction are collected. The results show
increased image contrast and 3X improvement in the signal intensity for fixed mouse tissues at a depth of 70 μm. The
images of the dendrite and spines are much clearer after correction with improved contrast. The Strehl ratio is improved
from 0.29 to 0.96, a significant 3.3X improvement.
Recently, there has been a growing interest in deep tissue imaging for the study of neurons. Unfortunately, because of the
inhomogeneous refractive index of the tissue, the aberrations degrade the resolution and brightness of the final image.
In this paper, we describe an adaptive optics confocal fluorescence microscope (AOCFM) which can correct aberrations
based on direct wavefront measurements using a point source reference beacon and a Shack-Hartmann Wavefront Sensor
(SHWS). Mouse brain tissues with different thicknesses are tested. After correction, both the signal intensity and contrast
of the image are improved.
We demonstrated the used of an adaptive optic system in biological imaging to improve the imaging characteristics of a
wide field microscope. A crimson red fluorescent bead emitting light at 650 nm was used together with a Shack-Hartmann wavefront sensor and deformable mirror to compensate for the aberrations introduce by a Drosophila embryo.
The measurement and correction at one wavelength improves the resolving power at a different wavelength, enabling the
structure of the sample to be resolved (510 nm). The use of the crimson beads allow for less photobleaching to be done
to the science object of the embryo, in this case our GFP model (green fluorescent beads), and allows for the science
object and wavefront reference to be spectrally separated. The spectral separation allows for single points sources to be
used for wavefront measurements, which is a necessary condition for the Shack-Hartmann Wavefront sensor operation.
In conventional Adaptive optics (AO) system, wavefront aberration is measured by wavefront sensor. However, a point
source is usually not available in most imaging systems. Therefore development of deformable mirror control scheme in
adaptive optics system based on vision information will extend the application area of AO technique. A focus measure
(FM) is a measure of image quality and can be calculated from image data obtained from the image sensor. It can be
used to find the optimal shape of the deformable mirror that compensates the wavefront aberration. In this paper, we
analyze the performance of focus measures under different types of wavefront aberrations. The relation of FM vs. the
aberration strength is analyzed systematically for different aberration modes and different FM functions. The FM
performance under combination of aberrations is also investigated. This research supplies a guide for selection of the
suitable FM function and strategy for control of deformable mirror in the AO system.
Insufficient vision information such as occlusion, low resolvability, and a small field of view (FOV) represent important
issues in microassembly and micromanipulation. In previous research, an active optical system was designed to supply a
compact flexible view. However the complex kinematics makes the system operation and calibration much difficult. In
this paper, a decoupling design for the variable view image system with a telecentric lens group is proposed to decouple
the view angle and scanning mirror angle. The proposed design increases the range of zenith angle. The forward
kinematics is analyzed with the help of vector diffraction theory. The singularity of Jacobin is analysis and the
singularity configurations are identified. In order to verify the proposed system, a prototype system is built up. A series
of experiments on the prototype system shows the validity of the new design.
Vision-based techniques used in automatic microassembly are limited by inherent problems such as small depth of focus
(DOF) and field of view (FOV). Microassembly operations need, however, initially to detect micro parts in a wide FOV
and large DOF yet to maintain high resolution for the final state. This paper proposes an active zooming control method
that enables adjustment of the FOV and DOF dynamically according to the position and focus measure of micro objects.
The proposed method is based on an artificial potential field with the capability to combine different kinds of constraints
such as the FOV, focus measure, and joint limits, into the system. The proposed method can ensure the microscopy to
maintain a wide FOV and large DOF initially, and high resolution at the end. Simulation and microassembly
experimental results are provided to verify feasibility of the proposed approach.
Insufficient vision information due to occlusion and low resolvability is one of the important issues in conventional
optical vision system that limits its application in micromanipulation and microassembly. The variable view imaging
system can help prevent these issues by changing optical system parameters such as spatial position, orientation and
focus plane. Its ability to achieve desired view of the target makes it particularly suitable for observing three dimensional
micro objects in micromanipulation and microassembly. In order to determine the tilt angle, pan angle and view position,
the kinematics of the variable view system was analyzed based on the ray tracing analysis with the help of the vector
refraction theory. This paper also shows its applicability in microassembly by demonstrating micro peg-in-hole insertion
task.
Insufficient vision information such as occlusion and low resolvability is one of the important issues that limit the micromanipulation and microassembly. In this paper, we proposed the active vision system that can interact with the environment by changing optical system parameters such as spatial position, orientation and focus plane. As an optomechatronic system, the proposed system integrates a pair of wedge prism, a scanning mirror, a deformable mirror and off-the-shelf optics. The compact double wedge prisms can change the view direction, however the aberration induced by wedge prisms can be corrected by deformable mirror. Combining with a scanning mirror, active optical system can observe the micro object in different view. Owing to the orthogonality of the Zernike polynomials, the proposed deformable mirror control algorithm can correct the aberration in each Zernike mode instead of controlling each actuator, which simplifies the control issue of deformable mirror. The preliminary experiment setup was built, and initial experiments were demonstrated to investigate the validity of the concept of the proposed system.
In this paper, we propose an active vision system which has variable PSF. The system consists of a deformable mirror, an aperture stop and four lenses. The deformable mirror is placed at the pupil plane and its effective size is determined by the aperture stop at the conjugate position of the mirror. We try to enhance the image resolution using this system. We make four different mirror surface shapes to take four regularly shifted images and use super-resolution algorithm to synthesize higher resolution image from the low resolution observations. It is demonstrated that our method can be used to enhance image resolution which is limited by CCD cell size. We compare the result with the real image and some discussion about algorithmic parameters follows.
Proportional control based visual controller is the main method used in the visual serving, but small proportional gain results in the slowly response and large proportional gain will result in large overshoot or make the system instable. A PD visual controller for microassembly system is presented to acquire better dynamic response. The fuzzy logic is applied to tuning the controller gains which is a model free method. Thus, the difficulty in obtaining precise and detailed system model is avoided and we can get satisfactory performance which is robust to modeling error and external disturbances. Furthermore, image moments are selected as visual features to avoid image singularities and the Jacobian matrix is full rank and upper triangular, thus it has the maximal decoupled structure and simplified the controller. A series of simulations are performed on peg and hole assembly to investigate the feasibility and effectiveness of this method.
IC chip has gradually become smaller and smaller, and thus it requires high packaging density. In chip packaging, accurate alignment of electronic components with respect to PCB is crucial for high quality packaging, especially in flipchip assembly. In this paper, vision system is used to provide relative pose information between flip-chip and substrate. Based on this information, these two parts are aligned accurately using visual servoing. In order to achieve high accuracy alignment, a dual imaging system (DIS) is introduced in this work, which is composed of zoom lenses,
beam-splitter, mirror, CCD, and LED illumination. It can simultaneously observe the solder bumps on flip-chip and the pattern of pads on substrate using one camera. Once the image frame containing flip-chip and substrate is obtained, their features are extracted from the preprocessed image. Extraction of the features enables us to obtain the position and orientation errors between the chip and the substrate. On the base of the measured errors, visual servoing method can determine the instantaneous velocity input of flip-chip at each servoing time and control the relative position and orientation precisely in an on-line manner. We carry out a series of experiments for various magnifications in order to evaluate the performance of the dual imaging system and the visual servoing algorithm as well.
Visual servoing technique used in microassembly limits its application, due to the two inherent problems, small depth of focus and small field of view. To avoid this criticism, in this paper, we present a method of microassembly using active zooming, which can solve these problems. In order to solve the problem of small field of view, the method using active zooming that can prevent the target out of field of view is proposed. In order to solve the problem of the small depth of focus, we proposed a method to control the zooming lens based on the blur measure of the moving target to get the clear
image in the field of view. In this paper, we use affine transformation of feature line to track the object. The proposed
microassembly process consists coarse visual servoing and fine visual servoing. A series of experiments are performed on peg and hole assembly to investigate the feasibility of this method.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.