This PDF file contains the front matter associated with SPIE Proceedings Volume 12767, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

The remarkable ability of metasurfaces to manipulate wavefronts in a versatile manner, coupled with their compact and ultrathin configuration, has garnered significant interest. These characteristics make metasurfaces ideally suited for fulfilling the demands of compact computational imaging systems. In this research endeavor, we present a comprehensive metasurface inverse design model, which operates in an end-to-end method, within a joint optimization framework for computational imaging. To exemplify its efficacy, we demonstrate the application of this model using silicon meta-atoms, showcasing its potential for broadband near-infrared imaging. This work contributes to the advancement of computational imaging techniques, providing a promising avenue for future developments in the field.

The spatial-angular coupling relationship of light field data is fundamental for the scene-disparity estimation. Occlusion, smoothing and noise are major challenges in disparity estimation of light field. Based on the special geometric structure of bi-plane parameterized light field data, we proposed a novel multi-cost loss light field disparity estimation network. The neural network consists of three modules. Firstly, in order to fully utilize the occlusion information contained in the light field, we divide the light field data into four subsets, and a weight shared network is designed to obtain four initial disparity maps, which embody different occlusion situations. Then, the gradient information of the center view sub-aperture image is used to pick the credible disparity from the initial disparity maps. Lastly, a convolutional neural network is designed to further improve the robustness of the merged disparity map in smooth and noisy regions, while maintaining the structural information of the scene. Experimental results on both synthetic and real datasets show that the proposed method can obtain higher-precision disparity.

In this paper, a new method is proposed for light field SOD by using convolutional neural networks. First, the light field dataset is extended by geometric transformations such as stretching, cropping, flipping, rotating, etc. The augmented data are then weighted with natural data to train the light field SOD. We propose a mutual attention approach in this process, extracting and fusing features from RGB images as well as depth maps. Therefore, our network can generate an accurate saliency map from the input light field images after training. The obtained saliency map can provide reliable a priori information for tasks such as semantic segmentation, target recognition, and visual tracking.

Optical lattices are generated by superposition of a set of plane waves, and have been widely used in the quantum information, atomic cooling, condensed-matter physics, optical sectioning imaging and lithography. A periodic two-dimensional optical lattices, weakly confined in its propagation direction, can be used as an excite source to perform multifocal scanning. By controlling the angles of superimposed plane waves, the propagation direction of 2D optical lattices can be adjusted to achieve 2D optical lattices with equal obliquity. In this paper, we have firstly investigated the basic generation principles of the square 2D optical lattices with good symmetry. The influence of numerical aperture, and other parameters on the basic parameters of fundamental optical lattices, had been analyzed with the finite element method. Based on these, the basic parameters of 2D inclined lattices had been further analyzed. The research results will provide a theoretical basis for future applications of optical lattices in optical sectioning imaging.

The use of photoelectric sensors to image the infrared radiation of the tail flame of a jet type high-altitude high-speed unmanned aerial vehicle is one of the technical points for discovering high-altitude targets and anti stealth. The difference in distance between targets in the field of view, as well as the limitations in sensitivity and spectral characteristics of photoelectric sensors, result in multifocal infrared images produced by photoelectric sensors. In order to independently master the multi focus infrared image fusion technology and quality evaluation standards, and facilitate the subsequent integration of artificial intelligence technology, algorithm design, autonomous recognition and tracking, the author conducted a systematic and comprehensive research on it. This article is one of the important contents of these research works. Using the wavelet transform fast decomposition and reconstruction algorithm (DWT) to calculate the fusion results of multifocal infrared images under different rules; the evaluation function for image fusion effectiveness was provided, and verification and analysis were conducted; from the perspective of the final observer of the multi focus infrared image, the definition of image definition quality factor is independent of the test image, observation conditions and observer's knowledge level; the principle of objective evaluation of image fusion quality (PAC) is calculated, and simulation experimental results and analysis are provided.

Detecting Alzheimer’s disease presents considerable challenges in the medical field. Hyperspectral imaging offers a promising approach to identifying the disease by analyzing spectral information from the diseased retina. This study utilized 3,256 ophthalmoscopic images from 137 patients and applied hyperspectral imaging technology to capture spectral data from specific regions within these images. Deep learning models, including ResNet50, Inception v3, GoogLeNet, and EfficientNet, were introduced and evaluated for lesion detection accuracy. In the training phase, these artificial neural networks showed varying accuracies in lesion detection using both original and hyperspectral images. For instance, ResNet50 achieved 80% accuracy with original images and 84% with hyperspectral images, while Inception v3 consistently achieved 80% accuracy with both types of images. GoogLeNet demonstrated 81% accuracy with original images, improving to 83% with hyperspectral images, and EfficientNet recorded accuracies of 80% for original images and 82% for hyperspectral images. The combination of hyperspectral imaging and deep learning models shows promise in enhancing Alzheimer’s disease detection through ophthalmoscopic image analysis, with hyperspectral images exhibiting higher efficacy compared to original images.

In the context of operating in low-light conditions where sufficient visual information is lacking, visual SLAM (Simultaneous Localization and Mapping) becomes a considerably challenging task. To address this issue, we propose a visual SLAM method based on Near-Infrared (NIR) illumination, which operates effectively in complete darkness while being visually friendly to the human eye. This approach employs NIR imagery to estimate the camera's motion and pose, achieving simultaneous localization and mapping. Through experiments and quantitative/qualitative analysis, the effectiveness of this method is demonstrated, particularly in low-light environments. Research findings also indicate that the performance of the NIR-based SLAM system is on par with its visible-light counterpart. Moreover, in indoor settings, the NIR-based SLAM system outperforms the visible-light SLAM system, suggesting that NIR-based SLAM could be a potential solution for robust camera pose estimation.

Hyperspectral imagery (HSI) classification, with the goal of assigning an appropriate land cover label to each hyperspectral pixel, is a challenging part of hyperspectral remote sensing. Recently, convolutional neural network-based HSI classification methods have shown superior performance due to their excellent locally contextual modeling ability. However, the ability of these methods to obtain deep semantic features is limited, and the computational cost increases markedly as the number of layers increases. In this work, we propose a novel spectral–spatial adaptive transformer (SSAT) model to adapt a pre-trained model for effective HSI classification. The main architecture of SSAT is based on vision transformer, which could aggregate features at different levels. Furthermore, we have designed an adaptive encoder block including spectral adaption, spatial adaption, and joint adaptation to extract HSI features in the spectral–spatial domains. Finally, the classification map is obtained from the fully connected layer. Extensive experiments have been conducted to validate the effectiveness of the proposed SSAT compared with seven typical HSI classification methods. Results demonstrate that the key classification evaluation index overall accuracy (OA) outperforms other comparative methods by at least 2.03%. Classification maps reveal the superior visualization effect, demonstrating that SSAT is an efficient tool for HSI classification.

Spectral images provide rich spatial and spectral information, enabling quantitative analysis of the same material and qualitative analysis of different materials. Due to the outstanding material identification capabilities of spectral technology, it is widely used in high-precision target detection tasks in complex scenes. Currently, adversarial sample attack techniques are advancing rapidly, however, most research on adversarial sample attacks in the field of object detection has been focused on RGB three-channel images. The exploration of adversarial techniques for object detection in multi-channel spectral images is still in its early stages. In this work, we propose a method for adversarial sample generation based on spectral images, which belongs to black-box attack and targeted attack. The reported technique, named Spectral Detection Adversary (SDA), is utilized to cause spectral image object detection networks to misclassify camouflage targets as real targets. We introduce a spectral analysis and comparison method to distinguish between real targets and camouflage targets, additionally, we propose a spectral dimension encoding method for various categories of real and camouflaged targets, thereby causing disruption in the adversary’s spectral image object detection network. In the most effective group, experimental verification revealed a reduction of more than 60% in the recall of camouflaged targets and a decrease of over 30% in the precision of real targets.

The reconstruction accuracy of polarization images is limited by the performance of the modulators such as polarizers/retarders etc. and mostly requires calibration. Mathematical models are usually used to find the optimal modulation conditions, and involve trade-offs between measurement time and accuracy. In this paper, we propose a full-Stokes image reconstruction (FIR) network to reconstruct polarization images from random modulated images. The network is constructed based on a cycled pixel-to-pixel conditional adversarial network, which has clear advantages in learning the mapping from input images to output images. The network with losses that utilize the physical features between the Stokes vectors is trained, and the reconstruction of complete polarization information is achieved without polarimetric calibration by our method. Simulations and experiments demonstrate the network’s wavelength independence and modulation independence, proving the effectiveness and robustness of the FIRNet in this paper.

A large open aperture in an optical system can capture high-resolution images but yields a shallow depth of field. To overcome this issue, we propose a method for improving microscopy imaging systems by using a variable-focus liquid lens to achieve 3D focus scanning. Specifically, the focal length of the imaging system was changed by the liquid lens, and a sequence of 12 images was captured in different focal planes. The image scale was adjusted according to the change in focal length, and the phase of the image was corrected by the phase only correction method. Then the in-focus pixels were abstracted by employing the Laplacian operator. Finally, an all-in-focus sharp image was generated, and a depth map was obtained. Additionally, to accelerate the processing speed, the Fast Fourier Transform image processing during phase correction was optimized. Meanwhile, we propose a parallel optimization solution for the original processing flow.

Dynamic projection mapping for moving objects has attracted much attention in recent years. However, conventional approaches have faced some issues, such as the target objects being limited to planar objects, the limited moving speed of the targets, and the limitation of their narrow depth of field. Based on the high-speed liquid lens optics, an adaptive three-dimensional projection display method could project an always in-focus image on the target. Meanwhile, the location of the non-planar object could be detected, and calculated the mapped projection contents, as a result, a stable "printed" projection mapping should be viewed on a moving non-planar object.

In line structured light vision measurement, due to the occlusion of the object, the complete information might not be collected. We propose a method to acquire the object’s all point data by dual-camera cooperative measurement. First, the transformation models between cameras and motion coordinates are constructed. Subsequently, the light plane is calibrated by special points from the images of the calibration plate which is raised 3 times with an appointed distance. Finally, the data points are fused and optimized with particular matrix. The results show that the average fusion error is reduced from 0.5668 mm to 0.1253 mm. This method improves the reconstruction accuracy greatly.

Infrared thermography converts infrared radiation to infrared images and allows the object surface temperature to be measured through radiometric calibration. However, infrared images lack depth information and cannot accurately represent the surface temperature distribution of three-dimensional (3D) objects. At the same time, the temperature measurement results of an infrared thermal imager can be affected by changes in the directional emissivity of the object surface, making it difficult to guarantee the accuracy of temperature measurement. 3D imaging is commonly used to obtain the surface morphology of an object to provide reliable 3D data for subsequent research. The paper uses structured light 3D imaging and infrared thermography to reconstruct the 3D temperature field and investigates the influence of directional emissivity on temperature measurement results. Specifically, the point cloud is obtained using the structured light 3D imaging algorithm based on Gray-code encoding. The reconstructed 3D point cloud is then fused with an infrared image for 3D temperature field reconstruction. Based on the in-depth analysis of the influence of the object surface directional emissivity on the temperature measurement results, the temperature measurement error caused by the change in directional emissivity is corrected by using the geometric shape information of the object surface. The experimental results indicate that the proposed 3D temperature field reconstruction method, which is based on the fusion of structured light imaging and infrared thermography, improves the accuracy of temperature measurement results based on an effective reconstruction of the 3D temperature field. This method has important application value in various fields such as disease assessment, component processing, and instrument testing.

Depth image reconstruction has been of interest in single photon LiDAR. The difficulty of high-accuracy depth image reconstruction, for example, the low-reflection objects are ignored sometimes, results from the signal intensity in the reconstructing, which is heavily affected by the target characteristics. The confidence interval width is utilized to guide the recovery of depth images for achieving high-accuracy depth imaging under the non-negligible differences in target characteristics. This work proposes a confidence interval (CI)-guided depth imaging method, which evaluates the uncertainty of depth estimation with the 95% confidence interval of normal distribution. Three necessary steps exist in process with this CI-guided depth imaging method. Firstly, the noise responses are eliminated using the local gating method. Then, the depth image is reconstructed by a pixel-wise maximum likelihood estimator, and the CI guidance image is calculated from the 95% confidence interval of the mean normal distribution. Finally, the adaptive thresholding segmentation algorithm based on the CI guidance image is adopted to achieve the reconstruction of depth images. The CI-guided depth imaging method presents a unique perspective between signal and noise. This paves the way to improve the depth image recovery accuracy with the state-of-the-art photon-efficient imaging algorithm.

Microlens arrays are widely used in various industrial cameras and detectors, and their wavefront consistency directly affects the performance of optical systems. The current non-interferometry computational imaging methods for measuring the wavefront of microlens arrays typically produce boundary artifacts and partially dope noise of high-frequency information. In order to accurately study and measure the consistency, relative position and wavefront aberration of each element in the microlens array. This paper presents a computational imaging method based on the intensity transfer equation (TIE) of discrete cosine transform (DCT) analysis and a novel hybrid Gerchberg-Saxton iteration (HGS-TIE) combining error reduction algorithm and input/output algorithm. This algorithm improves the accuracy of high-frequency phase measurement at the transitions between microlens and substrate. At the same time, boundary artifacts are eliminated, improving the overall accuracy of the measurement.

We developed SLODAR (SLOpe Detection And Ranging) in order to measure atmospheric characteristics in Geochang, Korea for the purpose of utilizing adaptive optics technology. In order to operate adaptive optics, it is necessary to measure the Fried parameter (r0) and Refractive index structure constant (Cn2) of the corresponding region, which can be measured through atmospheric characteristics measuring module. SLODAR mounts two wavefront sensors on a telescope and directs a binary star towards each sensor using a prism mirror. Then it measures the slope by detecting the focus shift caused by the atmospheric disturbance using the binary star. We attached z-axis motors to SLODAR and Prism to ensure that the focal position would always be the same even if the separation angle of the binary star changed. SLODAR can measure atmospheric characteristics at various altitudes, depending on the separation angle of the binary star. It is divided into wide mode (measured altitude about 400 ~ 700 m) and narrow mode (measured altitude about 5 ~ 9 km). We obtained r0 and Cn2 through Auto-Covariance and Cross-Covariance, respectively, from the measured slope. Currently, we are measuring the annual average r0 Cn2 in Geochang, Korea.

We present an implementation of tomography using MATLAB, specifically focusing on the Learn & Apply technique. The primary objective of our study is to assess the level of reconstruction errors encountered in tomography. We compare the results obtained from our implementation with the Object-Oriented, Matlab and Adaptive Optics (OOMAO) code, aiming to quantify the discrepancies between the two approaches. Furthermore, we investigate the impact of laser guide star (LGS) altitude variations on the reconstruction errors. By simulating tomographic reconstructions under different LGS altitudes, we analyze the variations in the error values and their relation to the atmospheric conditions. Additionally, we examine the influence of LGS divergence angle on the reconstruction errors, considering various divergence angles in our simulations. Afterwards, we explore the effect of increasing the number of layers used in the learning process on the simulation time.

Point spread function (PSF) is the response of an optical system to a point source. The ideal PSF of traditional optics is Airy pattern corresponding to images without aberration. However, the PSF shape is considered to be flexible in computational imaging territory, which can be artificially manipulated, and the target image is restored by reconstruction algorithm. It is called PSF engineering and can improve the imaging quality of optical system. The method can break through traditional optical limitations and even endow optical systems with new imaging capabilities. In this paper, we will discuss PSF engineering method based on wavefront phase modulation, which is more commonly used for its advantage of high throughput. Its application, the depth of field extension of an in-situ plankton imaging system is discussed based on end-to-end joint optimization, demonstrating a defocus invariant imaging quality.

In this paper, we discuss different approaches which may be used to solve the caustic illumination problem including the approximation of the light wavefront method, the backward and forward photon mappings, and propose the method for the effective calculation of the caustic illumination maps to be integrated into a common DirectX rendering pipeline. The proposed method is based on the photon mapping method and adds additional rendering pipeline steps for calculating the visibility maps for forward and backward caustics and then merging them to form the caustic illumination map of the whole scene. The proposed method was implemented, tested, and compared with the ground truth result acquired with realistic rendering software. The test results along with rendered images are presented.

In some cases, there is a problem associated with the synthesis of images of scenes whose data cannot be transferred to the rendering system, while the rendering software cannot be installed on the customer's hardware. This may be related to the fear of possible data leakage and the unlicensed use of the software. To solve this problem, we propose a remote rendering method that uses the remote interaction between the isolated rendering system which controls the rendering process and runs on the server side, and scene data which provides basic rendering functions and runs on the client side.

One of the most actively developing areas of digital image processing algorithms and complex technical vision systems is security systems. The focus of this research is video surveillance systems with biometrical face recognition algorithms. Intelligent analysis of an object (visitor's face) in such a system can be divided into sequential stages - detection (band of the ROI on the image from video), recognition (comparison with a reference photo in a database), and additional functions (for example, tracking, emotion recognition, etc.). At the same time, the rapid development of image and video falsification technologies emphasized the need to protect the system from attempts to fake biometric data used for verification. The techniques of face spoofing include statistically and dynamic two- and three-dimensional attacks, attacks using doubles and avatars, and more. The methods of detecting spoofing are studied in the area of face liveness problems. Neural network architectures and libraries were used as the main methods during research. This research concentrated on the software development for biometrical video monitoring system, which is more secure against face spoofing attacks. For testing algorithms for detecting face liveness, a small real-time video surveillance system prototype consisting of several cameras and providing monitoring of one room was designed. One of the requirements for our system was the use of conventional 2D cameras. For this reason, face liveness detection methods that require the use of 3D cameras or additional sensors were excluded from the field of view at the stage of choosing research methods. This paper presents an analysis of existing approaches to identify face spoofing attacks and compares them with the neural networks-based approach.

Digital holography (DH) is a dependable method for observing micro-nano structures and 3D distribution by combining amplitude and phase information. In recent years, pixel super-resolved (PSR) technique improves the SBP of holography. By introducing measurement diversity, PSR phase retrieval approaches are able to solve low-resolution issues due to limited sensor pixels. However, in the existing wavefront modulation PSR technique, dozens or even hundreds of randomly generated phase masks are usually required, resulting in time-consuming measurement and reconstruction. Reducing the amount of data can save time, but lead to poor accuracy and noise robustness. In this paper, we propose a novel PSR holography method with complementary patterns. Specifically, we use a pair of patterns that are exactly complementary in value, while others are randomly generated 0-1 phase patterns. Using this pair, the integrity of the target information contained is guaranteed in the diffraction intensity data set. In addition, the method can effectively improve resolution with limited data, speeding up the measurement and reconstruction process. A series of simulations demonstrate the effectiveness of complementary patterns, achieving more than a 3 dB enhancement in PSNR index compared with the random phase patterns.

In order to provide the management staff with information about tourists' picking in agriculture tourism, accurate recognition of tourists' picking behaviours is needed. Firstly, this paper proposes a method to accurately distinguish picking actions from other interfering actions mainly based on the change of joint angles. Secondly, a novel method of picking target determination based on the distance relationship between the wrist and the fruit is proposed for the position relationship between the human hand and the fruit in the picking behaviours. Finally, a method of setting state flags is proposed to avoid misidentification of the picking target due to the failure of fruit target detection. Experiments Show that it is proved that the above methods can accurately identify the picking behaviours.

Simultaneous localization and mapping (SLAM) technology is widely used in smart factories, logistics, unmanned vehicles, high precision mapping and other fields, where inaccurate positioning or even failure is the key problem to be improved and solved by LiDAR SLAM algorithm. In this paper, we propose a loosely coupled SLAM algorithm based on lidar and inertial measurement unit (IMU). Firstly, linear interpolation of the laser point cloud is performed by introducing IMU data to correct the point cloud distortion. Second, a height threshold and a dynamic clustering threshold are introduced in the point cloud segmentation part to determine the classification. Again, the cumulative error is effectively avoided by using Scan-Context descriptors, and the next step is executed to find matching loopback frames based on the Euclidean distance, and the loopback constraints are added after the successful verification of both. Finally, a local map consisting of key frames is aligned with the current frame to obtain the inter-frame matching error, and then a joint optimization function is constructed jointly with the IMU pre-integration error and the loopback detection constraint to solve the laser odometer pose. The results on the KITTI dataset show that the improved laser odometer accuracy is higher than that of the LeGO-LOAM scheme in compliance with the odometer real-time requirements.

In recent years, deep learning has exhibited remarkable performance in image classification. Nevertheless, traditional deep-learning-based techniques heavily depend on the availability of high-quality images for conveying information. This reliance results in the inefficient utilization of hardware and software resources across various stages, including image acquisition, storage, and processing. Additionally, these techniques often necessitate substantial amounts of data to effectively learn the underlying mapping, posing challenges in practical scenarios where acquiring a sufficient volume of paired data proves difficult. In this paper, we introduce a novel approach for image-free few-shot recognition by using a single-pixel detector. Our method comprises two fundamental stages. First, we design a neural network that integrates encoding and decoding modules, which can learn optimized encoding masks based on the statistical priors. Second, we employ these optimized masks to generate compressed 1D measurements. Subsequently, these measurements are fed into the classification network, preceded by the decoding module trained during the initial stage. The parameters of this decoding module serve as the initialization parameters for the subsequent stage of training. Furthermore, we incorporate a meta-training strategy, commonly used in few-shot classification, to mitigate dataset requirements during the second stage of training. Simulation results illustrate the effectiveness of our approach in image-free classification directly from 1D measurements, bypassing the time-consuming image reconstruction process. Our technique achieves a substantial reduction in data volume by two orders of magnitude while relying on only a limited number of paired data samples.

Image stitching is a technology that combines multiple images taken by different cameras to create a larger field of view. It has wide applications in scenarios such as surveillance and virtual reality, making it an important topic in computer vision. This paper addresses the challenges associated with stitching ground images with inconspicuous features using traditional methods. These challenges include poor feature extraction capabilities and issues like misalignment, artifacts, and structural deformations introduced during stitching. The paper proposes the utilization of unsupervised learning techniques to enhance the quality of image stitching. The network primarily consists of two parts: image alignment and image reconstruction. In the image reconstruction part, deformation rules for image stitching are learned through both a low-resolution branch and a high-resolution branch. Finally, it evaluates the stitched images before and after improvement using image stitching evaluation metrics. Experimental results demonstrate that this approach successfully mitigates artifacts and distortions introduced during stitching, resulting in an improved image stitching quality.

While mobile phones offer convenience in our daily lives, they also introduce associated security risks. For instance, in high-security settings like confidential facilities, casual mobile phone usage and calls can inadvertently lead to the leakage of sensitive information. In response to such security concerns, this paper proposes an algorithm for recognizing mobile phone behaviors in high-resolution images with a wide field of view.To improve inference speed, we introduce the C3_Faster module. To address the challenge of detecting small-sized targets in images, we propose a boundary loss function. This reduces the scale sensitivity of IoU loss and mitigates model underperformance in detecting small objects. Experimental results demonstrate that, our improved algorithm achieved a 7.6% increase in mAP and a 38% improvement in inference speed. These findings highlight the effectiveness of our enhanced algorithm, making it well-suited for the task of mobile behavior recognition in secure environments.

Camera calibration is one of the key tasks in the field of computer vision, finding extensive applications in various domains, including photogrammetry, 3D reconstruction, augmented reality, and autonomous driving. The Direct Linear Transform (DLT) algorithm, a classical approach for camera calibration, estimates camera parameters by solving a system of linear equations. However, traditional DLT methods may face accuracy and stability issues when dealing with noise, distortion, and nonlinear effects. To address these limitations, this paper introduces a camera calibration method based on the Improved DLT algorithm. This method incorporates distortion models into the traditional DLT algorithm and utilizes Levenberg-Marquardt (LM) optimization techniques to enhance calibration accuracy and stability. The key steps involve data preparation, DLT estimation, and nonlinear optimization. Experimental results demonstrate that the Improved DLT algorithm outperforms traditional DLT methods, particularly in cameras with significant distortion and wide field of view. It exhibits smaller reprojection errors and achieves a more uniform error distribution, especially at image edges. This research contributes by providing a more accurate and robust camera calibration method, offering valuable tools for computer vision applications, and advancing the development and application of computer vision technology.

Coded aperture snapshot spectral imaging (CASSI) is a technique that can capture 3D hyperspectral images (HSIs) of scenes in a single shot. However, the quality of the reconstructed HSIs is affected by various optical aberrations and system noise. Existing deep learning methods for HSI reconstruction do not consider these degradation patterns and thus lack generalization ability to real CASSI data. In this paper, we propose a practical method to recover high-quality HSIs from low-quality CASSI data. We use a spectral imaging simulation to generate authentic training data that reflects the optical aberrations of the CASSI system. We then train a generative network on this data to remove blur and chromatic aberrations from the CASSI measurements. Our experiments show that our method can effectively improve the quality of the reconstructed HSIs and can be easily applied to real CASSI systems.

In this paper, we propose a novel portable confocal-controlled LIBS microscopy method with highly integrated structure. A 2D MEMS mirror scanning mirror was utilized to achieve beam transverse scanning and compact imaging structure design. Confocal-controlled LIBS was utilized to increase the focusing abilities and reduced the size of the laser ablation, thereby achieving a better spatial resolution imaging. Thereafter, 3D elemental mapping was constructed from the morphological and LIBS spectral data of the sample. The system had a lateral resolution of geometrical and elemental maps of up 800 nm and 15 μm, which is basically consistent with the theoretical analysis.

This article proposes a two-dimensional material layer determination and mapping method based on high-precision confocal Raman spectroscopy. It achieves nanometer-level axial focusing accuracy, and the lateral resolution of Raman spectroscopy is better than 600 nm. Compared with non-confocal Raman systems, laser confocal Raman systems can accurately determine the number of layers of molybdenum disulfide materials within 10 layers, significantly improving the accuracy of layer determination and spectral imaging stability of existing Raman imaging techniques for two-dimensional materials. This method provides a scientific basis for the design and performance optimization of semiconductor devices.

Since feature matching of image pairs brings a heavy computational burden, Structure-from-Motion faces great challenges in efficiency, especially for unordered large-scale image collections. To solve it, we propose a hierarchical image matching method in this paper. Our approach starts with an iterative image retrieval scheme, which can efficiently find potentially overlapping image pairs as candidates and avoid unnecessary computation. Then, feature extraction, feature matching and geometric verification are implemented in candidates to find the verified image pairs and inlier feature correspondences. Experiments on benchmark datasets and large-scale unordered datasets demonstrate that our method performs competitiveness in efficiency, without degrading the accuracy, compared with the state-of-the-art system.

In order to obtain the fine structure of biological tissues and living cells, microscopic imaging techniques are required to have tomographic capabilities. We propose a dynamic speckle fluorescence illumination microsystem based on active light manipulation, which uses the dynamically changing speckle pattern to illuminate the sample to be measured in full field, rotates the sample to be measured through linear laser tweezers, and obtains a three-dimensional fluorescence tomography image by extracting the fluorescence signal that changes sharply in the focal plane. The system changes the previous passive observation to active observation, and realizes the acquisition of three-dimensional structural chromatographic images with high temporal and spatial resolution without affecting the nature of the cells themselves and their environment, and the long-term dynamic quantitative monitoring of their life activity processes.

We propose a light-sheet fluorescence microscopy imaging method based on active optical manipulation with linear laser optical tweezers. In this imaging method, a column lens is utilized to generate an excitation light sheet, and a linear laser light tweezer is adopted to capture and precisely control the rotation of the experimental object. The linear laser optical tweezers are used to capture and precisely control the rotation of the experimental object. The plane of the light sheet is perpendicular to the rotational plane of the experimental object, and the rotation of the experimental object at different angles can be controlled to realize the light sheet on the experimental object layer. The light sheet is perpendicular to the rotational plane of the subject. Most importantly, compared to traditional microscopy methods, this solution directly manipulates and observes the experimental object, which reduces the risk of the object changing position in the sample container. It reduces the uncertainty and error of image acquisition and reconstruction caused by the change of the position of the experimental object in the sample container.

Spectral imaging can obtain spectral information of target scenes and has wide applications many fields such as biomedical, military, agricultural, food safety. Coded aperture snapshot spectral imager (CASSI) proposed a snapshot spectral imaging system by employing compressed sensing theory. However, the use of the dispersion element leads to a deviation in the direction of light propagation, resulting in a complex system. This paper proposes a spectral imager that utilizes metasurface array instead of dispersion elements to encode the spectrum of the incident light and uses macro pixel segmentation method for spatial code. In terms of design, we designed nine nanofin structures with different longer and shorter axes. And these nine nanofin structures are formed as a macro pixel. Due to the fact that the design of the metasurface is not a C4 structure, it has a polarization conversion effect. When linearly polarized light is incident, the output will become two orthogonal polarized light with different spectral transmission. By utilizing the polarization image sensor on the detection end, more information can be obtained in one snapshot to increase the accuracy of recovery.

Infrared imaging is widely used in low-light environments, remote sensing, and camouflage recognition, due to its high sensitivity, response, and all-weather adaptability. However, conventional infrared cameras are limited by their high cost per pixel, low resolution, and low contrast, resulting in the loss of valuable scene information. To address this challenge, we report a method for the fusion of high-resolution visible light images and single-pixel infrared images using a single-pixel detector. Our method consists of two modules: the single-pixel imaging (SPI) module and the fusion module. The SPI module uses a spatial light modulation system to obtain a one-dimensional measurement of the infrared image and then uses the reconstruction algorithm to obtain the infrared single-pixel image, thus avoiding the need for high-cost infrared cameras and obtaining the high-quality infrared single-pixel image with low sampling rate. The fusion module uses an infrared single-pixel image and a high-resolution visible image to obtain the hybrid image which combines the interested information from the visible light image and the infrared single-pixel image, thereby achieving low-cost imaging that preserves the detailed information from visible images and maintains the salient features from infrared images. A series of simulations demonstrate that our method benefits from the high resolution and high contrast of visible light imaging and the high sensitivity of infrared imaging. Our technique has the potential to improve complex environment detection, nighttime detection, and covert detection.

In recent years, the optical design of refractive beam shaping system has been widely studied. In laser beam shaping, because the laser light source is usually not uniformly distributed, to achieve uniform distribution or other predetermined wavefront and intensity distribution, it is necessary to use aspheric or freeform surface to build laser shaping system. However, the angular Gaussian distribution laser with different waist generally cannot be used in the same beam shaping system. We design a beam shaping system. When the incident angular Gaussian distribution beam has different waist, the output beam can always achieve the predetermined wavefront and intensity distribution. This work is challenging and rewarding. We propose a design framework, including the calculation and optimization process of the initial system. In the beam shaping system, when the waist of the incident angular Gaussian distribution beam varies widely, the output beam still maintains the predetermined wavefront and intensity distribution. A design example is given to verify the feasibility and effectiveness of the proposed beam shaping design method. This method can be used in the fields of laser imaging, laser display and laser processing. The high level of optical performance proves the effectiveness and feasibility of the proposed design method.

Compared with array detectors (such as CCD or CMOS), single pixel detectors have potential in invisible band and weak light applications to broaden the spectrum of spectral imaging. Existing leading methods for spectral reconstruction (SR) focus on designing deeper or wider convolutional neural networks (CNNs) to learn the end-to-end mapping from the RGB image to its hyperspectral image (HSI). These CNN-based methods achieve impressive restoration performance while showing limitations in capturing the long-range dependencies and self-similarity prior. To cope with this problem, we propose a novel spectral-attention transformer(SAT-net) method for single-pixel multispectral reconstruction. In addition, we introduce total variation (TV) to maintain the smooth structure of HSI. The experimental results of simulation and real data show that the proposed SAT-net is superior to other traditional algorithms based on compressive sensing(CS) methods.

By capturing a series of low-resolution images which have known or unknown sub-pixel displacement between each other, high resolution image could be reconstructed through algorithms such as IBP, POCS and so on. This technique mainly aims to solve the problem of aliasing effect caused by under-sampling but one problem exists. While applying sub-pixel shift based super-resolution reconstruction, point spread function is used to simulate the imaging process but usually the point spread function corresponding to the low-resolution imaging system is used, which does not match reconstruction in high-resolution grid. According to our previous researches, the wave-front coding technique could be used to realize single image amplification based super-resolution reconstruction because the point spread function corresponding to the high-resolution grid could be digitally generated in a more accurate way. In this manuscript, the rotationally symmetric wave-front coding technique and the sub-pixel shift based super-resolution imaging are combined together and there are two advantages. First, because of decrease of the magnitude of optical transfer function caused by wave-front coding, the aliasing effect in the intermediate images is reduced keeping pitch size unchanged. Second, while doing the reconstruction in high-resolution grid, the computed point spread function corresponding to the high-resolution grid is used, which better matches the high-resolution grid. The numerical results demonstrate that better image could be obtained by incorporating rotationally symmetric wave-front coding into sub-pixel shift based super-resolution imaging.

In precision medicine, the diagnosis and prognosis of colorectal cancer (CRC) has always been a clinical focus, and subjective evaluation of histological slides by highly trained pathologists remains the gold standard in this field. With the innovation of deep learning in image analysis, convolutional neural networks (CNNs) can extract quantitative information from HE pathological images, and We found a certain correlation between this information and the progression of colorectal cancer. In this study, we first used a CNN model to classify HE pathological images. The CNN model was trained and validated using patches of 86 (from the NCT biobank and the UMM pathology archive) and 25 (from the DACHS study in the NCT biobank) colorectal HE pathological images, respectively. With this tool, we performed automated tissue decomposition of representative multitissue HE images from the The Cancer Genome Atlas (TCGA) cohort. Based on the output neuron activations in the CNN, we calculated the tumor-stroma-ratio (TSR). This score was an independent prognostic factor for overall survival (OS) in a multivariable Cox proportional hazard model. Finally, we validated these findings in an independent CRC dataset. Again, the score was an independent prognostic factor for OS. This study, deep learning methods can decompose complex tissue pathological images and extract prognostic factors from HE pathological images.

Current work proposes a novel approach for joint layout, object pose, and mesh reconstruction from scanned point clouds that leverages the latest neural network architectures and addresses the challenge of missing point data. The goal of this research is to develop an advanced approach that can generate accurate and complete 3D models from point cloud data obtained from real-world environments. The proposed method leverages state-of-the-art neural network architectures, including PointNet++ and Transformer, to accomplish the task. PointNet++ is used for feature extraction, while Transformer is used for joint feature representation learning. By combining these two networks, the proposed method can effectively encode the complex geometries of the 3D scenes and generate high-quality reconstructions. The core concept of the method is to initially segment the point cloud into small fragments using a neural network. Each fragment is then reconstructed as a polygonal mesh. Addressing the restoration of missing points in point cloud data is a significant problem that this paper tackles. It adapts a robust method, which utilizes the L1-Median algorithm and local point cloud features, to effectively fill in these missing points. This approach is capable of adjusting to various geometric structures and rectifying topological connection errors, enabling the reconstruction of complete and accurate models even when some original data is missing. The proposed method is compared against several state-of-the-art approaches and has the potential to be a valuable tool in a range of applications, including architecture, engineering, digitization of cultural heritages, as well as augmented and mixed reality systems.

The current work introduces a neural processor architecture that allows hardware acceleration in the processing of convolutional neural networks (CNNs). The purpose of this study is to design the architecture and microarchitecture of the neural processor, which can be used to solve the problems of environmental analysis and recognition of objects that are located in the scene in augmented reality systems, presented as energy-efficient and compact wearable devices. The proposed architecture provides the opportunity to adjust the variable parameters of blocks for data processing and storage to optimize the performance, energy consumption and resources used to implement the neural processor. The article offers variants of scaling of computing blocks and memory blocks in the architecture of a neural processor, which can be used for increasing the performance of the end product. The paper describes a tool that generates a neural processor based on given limitations on power consumption and performance and on the structure of convolutional neural networks that you need to use for data processing. The proposed tool can potentially become a valuable product in the field of designing hardware accelerators of convolutional neural networks, as it allows to increase the degree of automation of the process of synthesis of neural processors for further implementation in mixed reality systems, made as portable devices. In general, this thesis presents tools that allow the developer of software based on convolutional neural networks for mixed reality systems to synthesize energy-efficient processors to accelerate the processing of convolutional neural networks.

Monocular vision-based 3D-scanning systems have revolutionized object and environment capture. The authors have developed a system that uses a rotating camera and an angle sensor to capture images from different angles. Through deep learning techniques, depth maps are extracted from the images. The resulting depth map is used to generate a 3D point cloud model, which is preprocessed to enhance efficiency. This research considered several methods to improve depth accuracy of panorama image, as well as to recalibrate depth maps and smooth 360-degree 3D models. Our method produces better results than the pre-trained model. The authors use a LIDAR to get distance and increase accuracy of model. To visualize the model, the web framework was developed. The rendered model can be accessed through a web browser, providing functionalities such as coordinate selection and distance calculation.

For the correct reconstruction of the real-world geometry, we propose an iterative approach that utilizes the method of differential rendering. This method is based on optimizing the parameters of scene geometry objects to approximate the synthesized scene images to their real images obtained, for example, through digital photography. For the optimization methods to work efficiently during rendering, a series of differential scene images corresponding to the increments of individual scene geometry parameters are formed. To construct the differential scene images, it is necessary to create a parametric model of its geometry. Since the scene geometry is usually represented as a triangular mesh, the triangle vertices become the parameters of the model. However, the number of triangles requires enormous computational resources, which may be impractical. We propose the parametric model of scene geometry, based on a combination of approximate voxel and refined triangular mesh representations. In this model, two levels of geometry modification are suggested. At the first level voxels are the directly modifiable parameters. At the second level individual or connected triangles can be modified precisely. The rendering process is divided into two parts. The first part involves calculating the luminance of direct illumination in the limited representation of the scene, solely related to the modified geometry or its shadow area. The second part involves calculating the luminance of indirect illumination. The calculation of differentials is based on the assumption that the luminance of indirect illumination remains almost unchanged under slight geometry modifications. These solutions significantly reduce the computation time of differentials while preserving the physical correctness of the calculation results and enabling the calculation of noise-free increments of differential scene images for a noisy base image.

The method of photon maps allows to efficiently calculate caustic illumination but requires the use of accelerating spatial partitioning structures to store photonic maps, provide quick access to them, and accelerate the collection of illumination. In this paper, we propose to use a combined accelerating structure based on an octree and hash tables. The structure itself represents an octree consisting of multiple levels of hash tables. This structure makes it possible to obtain a high space partitioning adaptability achieved by using tree structure, while providing quick access to the cells of the structure by utilizing the regular hash table structure. The use of hash tables allows access to the cells of the structure arbitrarily speeding up the search and traversal of cells and photons inside the structure. Unlike other regular structures, hash tables allow to exclude empty cells from storing in memory. Using a tree structure allows to further reduce the memory requirement and ensure the adaptability of space partitioning. The paper proposes an algorithm for generating a combined accelerating structure based on a dynamic analysis of the generating structure. As well as algorithms for finding cells and traversing them with a ray, which allow using this structure to accelerate the photon map method when calculating surface and volume light scattering. SIMD operations are used to speed up the hash tables collision resolution. Efficiency comparison of the photon map method using the proposed accelerated structure and similar tree like structures shows the acceleration of the proposed solution for multi-thread computation.

As a rule, specially trained neural networks are engaged in the recognition and classification of objects in RGB and RGB-D images. The quality of object recognition depends on the quality of neural network training. Since the network cannot go far beyond the limits of the training set, the problems of forming datasets and their correct annotating are of particular relevance. These tasks are time-consuming and can be difficult to perform in real-world conditions since it is not always possible to create the required real-world illumination and observation conditions. So, synthesized images with a high degree of realism can be used as input data for deep learning. To synthesize realistic images, it is necessary to create appropriate realistic models of scene objects, illumination and observation conditions, including ones achieved with special optical devices. However, this is not enough to create a dataset, since it is necessary to generate thousands of images, which is hardly possible to do manually. Therefore, an automated solution is proposed, which allows us to automatically process the scene to observe it from different angles, modify the scene by adding, deleting, moving, or rotating individual objects, and then perform automatic annotation of a desired scene image. As a result, not only directly visible scene object images but also their reflections may be annotated. In addition to the segmented image, a segmented point cloud and a depth map image (RGB-D) are built, which helps in training neural networks working with such data. For this, a Python scripting interpreter was built-in into a realistic rendering system. It allows us to perform any actions with the scene that are allowed in the user interface to control the automatic synthesis and segmentation of images. The paper provides examples of automatic dataset generation and corresponding trained neural network results.

Currently, for most VR/AR applications it is necessary to reconstruct physically-correctly the real world, and the optical properties of objects. In this paper, to solve this problem, we propose to use a parametric model of optical properties, which is a mix of several known models, which has a small number of parameters and correctly simulates the propagation of light in the scene. In addition, based on the parameters of the model, differential rendering of the image is performed, as a result of which the obtained differentials can be further used in the optimizer. The optimizer, in turn, can modify the model parameters to improve the rendering quality and thus bring the proposed model of optical properties closer to the real optical properties of objects.

Color features play a unique role in vehicle recognition. The color recognition algorithms based on deep learning neural networks are studied in this paper. A color recognition experiment is carried on to some typical deep learning neural networks, the result of the experiment proves that Yolov5 has faster training speed and higher accuracy for vehicle color recognition, so Yolov5 is chosen for the color recognition. The structure of yolov5 is optimized by adding C2f module replacing C3 module and adjusting the parameters of HSV color space when it is applied to identify 8 typical vehicle colors using BIT Vehicles data set. The modified Yolov5 make the accuracy of vehicle color recognition improved effectively to the complex color vehicles and part covered vehicles comparing with original yolov5 network.

This paper mainly focuses on the rain streak removal task from a single image. Based on the observation that the distribution of rain streak in an image is sparse. We propose a two- phase single image deraining method. Firstly, it detects the rain locations with a proposed anisotropic global gradient prior (AGGP) and generates a rain mask for rain streak removal. Then it recovers the information in rain distorted region with AGGP based multi-layer image inpainting model. Furthermore, to solve the multi-variable optimization problems, we develop an alternating half-quadratic algorithm by introducing alternating algorithm and the variable split method. Both experiments on synthesized and realworld images show the proposed method outperforms state-of-the-art methods in terms of rain streak removal and image multi-layer information preservation.

The existence of noise will seriously affect quality of image. Image denoising method is important for obtaining high-quality image. Most of traditional denoising methods need to estimate the noise level, which are unstable in the actual denoising scene. As a data-driven method, the development of deep learning shows great potential in the field of image denoising. In this paper, a method combining image denoising model and deep learning framework is proposed. Welldesigned multi-scale restoration network Noise-Net embedded this method optimizing neural network training to obtain ideal image recovery results. By down-sampling the original noisy image input at different scales, the noisy image features are extracted. These multi-scale features are summed and combined. The addition of the residual module improves the network training ability and effectively prevents the network from overfitting. The network is optimized by Convolutional Block Attention Module (CBAM). It can enable effective extraction of image features in the spatial and frequency domains. Network input is noisy image, clear image as label. The training phase is divided into two stages: noisy data generation and simulated images for pre-training. 2000 images of DOTA 1.0 dataset constitute as training set and 1000 images as test set. By adding different noises such as Gaussian noise and Poisson noise to the image, the data set is constructed with the label image. The loss function of the absolute minimum error is calculated and sent to the Adam optimizer for parameter optimization. Numerical simulation and experimental results show that Noise-Net has an effect on image denoising ability.

In the flight mission of airborne hyperspectral camera, due to the characteristics of hyperspectral data with high spatial resolution, high spectral resolution and many spectral channels, the amount of data acquired is also exploding. However, the traditional data storage system has high latency and poor read and write performance, which cannot meet the real-time storage performance requirements of hyperspectral image data. In order to achieve high-speed storage of airborne hyperspectral camera image data, a data storage system based on Soc FPGA is proposed in this paper. The system takes ZYNQ 7035 chip as the core, builds the system in Vivado environment to realize data transmission based on PCIe channel, and designs NVMe high-speed interface according to NVMe protocol specification under Linux system. The storage system receives the image data through the Cameralink interface, and uses SSD as the storage device to ensure the high-speed transmission and storage of data to meet the storage requirements of the hyperspectral imaging field.

The article proposes an algorithm for detecting the boundaries of objects of vertical and horizontal structures, fixed in various electromagnetic ranges. Different electromagnetic ranges can be used as analyzed data. To analyze the data and identify stable signs of boundaries, a combined algorithm for analyzing streaming video in the frequency and time domains is proposed. A multicriteria algorithm for primary data processing in the frequency domain is proposed. A modification of the multicriteria data filtering algorithm in the time domain is presented. An algorithm for complex data analysis has been developed for reconstructing the boundaries of objects. A combined algorithm for detecting the boundaries of objects has been developed. A condition has been developed for stable detection of object boundaries with the possibility of their detection. The description of the possibility of forming simplified images by analyzing one data channel and transferring the feature mask to other data channels is given. The effectiveness of the proposed algorithms was evaluated on test images. On the example of the analysis of combined data from the ultraviolet matrix of a video camera (stationary marks on objects) and a visible camera (changeable data from monitors). The applicability of the proposed approach for detecting combined control and positioning problems by industrial devices is shown. The fields of application and the possibility of integrating other data ranges are described.

The most technologically sophisticated equipment for the production of integrated circuits is equipment for the production of semiconductor wafers, lithography, etching and implantation. In addition to operations on photolithography and the production of semiconductor wafers, the technological chain for the production of a modern microcircuit includes cutting semiconductor wafers, mounting them on a substrate, testing, bonding and packaging. All of them require high-precision manipulation work. The creation of machines that solve such problems involves, in addition to the use of sufficiently precise mechanics, also the development of methods for recognizing objects and positioning a working tool based on video data, which will increase the accuracy of mounting crystals, the repeatability of operations and, consequently, reduce the percentage of scrap and the cost of final products. The main problem of introducing vision systems is their integration into numerical control systems of installations in order to increase the speed of technological operations. Thus, the development of real-time algorithms that determine the position of semiconductor crystals relative to the substrate, their implementation in the form of programs for specialized controllers integrated directly into CNC systems is an urgent task. The paper presents the implementation of a video stream processing system from high-resolution cameras, which determines the contours of a semiconductor crystal, its position relative to the substrate, including the rotation angle, and outputs them directly to the programmable logic controller of the CNC system. The basis for the system was a programmable logic integrated circuit Cyclone IV which implements a 64-bit RISC-V core.

A key functional role is served by the helical surfaces of carbide end mills that can be manufactured during diamond grinding wheel. Localized changes in the form of the helical surface can be caused by abrasion, high pressure, and grinding wheel wear. Therefore, it is extremely important to measure the physical samples of products with a helical surface according to the criterion of profile accuracy, rake angle and core diameter. A specialized inspection machine in reflected light can be used to obtain images across the helical groove. Manually extracting a number of defects from photos takes time. Using defect recognition algorithms, effective and quick quality control of a ground helical surface can be established. As a result, effective surface quality control can be achieved in the machine tool industry. In this study, an innovative approach to determine a defect's shape and location as well as an algorithm for removing it are presented. Both of these approaches are integrated into the technological process used to manufacture products with helical surfaces. With the goal to recognized create suggestions for image analysis using different image levels, the suggested approach provides logically smoothing histograms and limiting contrast as an image pre-processing, based on an analysis of images with useful and faulty parts. Achieved successful extraction of areas of adhesive, diffusion, abrasion and chips from the image through post-processing. The article presents a new approach to recognizing adhesive and diffusion defects on the helical surface of a mill after grinding. When developing this approach, it was revealed that areas with alternating profile changes are most susceptible to the formation of defects under conditions of increased heating of the working area, and specialized inductors for searching for defects in localized areas according to the criterion of pixel brightness intensity were proposed.

In recent decades, with the development of digital technologies and the spread of the Internet, photography has become an integral part of our daily lives. Often, images with objects superimposed in text, symbols, or drawings. These superimposed objects can serve a variety of purposes. Adding text or graphics can highlight specific details in an image or convey important information, such as the event's date, location, or specifics. Adding graphic elements may be used commercially to promote products, services, or a brand. Another area of application of such objects is the field of intellectual property protection or verification of the authenticity of a digital image. The problem with such images is that objects can completely overlap the image, leading to a distortion of the perceived information. In this article, we propose an effective convolutional autoencoder model for removing watermarking and restoring an image's usable part. This network can perfectly remove the embedded, visible watermark, restoring the image. Experimental results demonstrate the effectiveness and superiority of our scheme.

Image inpainting is a challenging task where large missing regions must be filled based on the available data. It is an interesting new research topic in multimedia computing and image processing. This paper proposes a novel deep learning reconstruction method using an adaptive generative adversarial neural network (aGAN). The key feature of the proposed network is the dynamic estimation of the trend of the loss function. This estimate finds the optimal balance between adversarial and pixel-by-pixel losses. We introduced the coefficient of the approximating curve, which allows us to estimate the trend dynamics of the loss function. Depending on the value of this curve, we adjust the weight of the pixel-by-pixel loss to make the restored area sharper. The proposed method allows a higher sharpness of the reconstructed area and a more stable learning process. Experiments on the dataset show that our method successfully predicts multimedia information in missing regions and significantly outperforms the state-of-the-art methods.