Recent mobile imaging seeks to expedite the autofocus process by embedding a phase detector in the image sensor to provide information for controlling both the magnitude and direction of lens movement. Compared to conventional contrast-detection autofocus, phase-detection autofocus (PDAF) is able to quickly bring the lens toward the in-focus position. However, the presence of sensor noise, the lack of image contrast, and the spatial offset between the left and right phase detectors can easily affect the performance of phase detection. We present a statistical approach to address this issue by characterizing the distribution of phase shift for a given distance of the lens to the in-focus position. We model the phase shift as a skew-normal distribution and verify it empirically. The results show that the skew-normal distribution is indeed a proper model for the phase shift data. We also propose a method based on Bayes’ theorem to determine the lens movement. Experimental results show that the proposed method is able to improve the reliability of PDAF.

As a local invariant feature of videos, the spatiotemporal interest point (STIP) has been widely used in computer vision and pattern recognition. However, existing STIP detectors are generally extended from detection algorithms constructed for local invariant features of two-dimensional images, which does not explicitly exploit the motion information inherent in the temporal domain of videos, thus weakening the performance of existing STIP detectors in a video context. To remedy this, we aim to develop an STIP detector that uniformly captures appearance and motion information for video, thus yielding substantial performance improvement. Specifically, under the framework of geometric algebra, we first develop a spatiotemporal unified model of appearance and motion-variation information (UMAMV), and then a UMAMV-based scale space of the spatiotemporal domain is proposed to synthetically analyze appearance information and motion information in a video. Based on this model, we propose an STIP feature of UMAMV-SIFT that embraces both appearance and motion variation information of the videos. Three datasets with different sizes are utilized to evaluate the proposed model and the STIP detector. We present experimental results to show that the UMAMV-SIFT achieves state-of-the-art performance and is particularly effective when dataset is small.

In the case of poor lighting conditions, it is easy to capture the underexposed images with low contrast and low quality. Traditional single-image enhancement methods often fail in revealing image details because of the limited information in a single-source image. A single underexposed image enhancement method based on adaptive decomposition and convolutional neural network (CNN) is proposed. The CNN training models only need images with different brightness rather than a strict ground-truth image. First, a simple effective synchronous decomposition method is proposed to solve the synergy problem in multisource image decomposition. Then, two CNN models are designed for the high-frequency part and low-frequency part, respectively. They process the high-frequency and low-frequency sub-bands, instead of the entire source images. The weight map obtained from the CNN model represents the contrast distribution. The exposure map generated by gradient-based visibility assessment indicates the exposure distribution. Finally, the weight map and the exposure map are multiplied to generate the final decision map. Experimental results demonstrate that the proposed method outperforms competing methods.

We present a holistic technique for recognition of text in cursive scripts using printed Urdu ligatures as a case study. Convolutional neural networks (CNNs) are trained on high-frequency ligature clusters for feature extraction and classification. A query ligature presented to the system is first divided into primary and secondary ligatures that are separately recognized and later associated in a postprocessing step to recognize the complete ligature. Experiments are carried out using transfer learning on pretrained networks as well as by training a network from scratch. The technique is evaluated on ligatures extracted from two standard databases of printed Urdu text, Urdu printed text image (UPTI) and Center of Language Engineering (CLE), as well as by combining the ligatures of the two datasets. The system realizes high recognition rates of 97.81% and 89.20% on the UPTI and CLE databases, respectively.

We propose a steganographic scheme for stylized images. Given a natural image, an image style transfer algorithm is executed twice to produce two similar stylized images with different parameters. One of them is used for embedding and another one is employed as a reference. For the cover image, embedding costs are assigned to measure the detection risk of the modifications. Each embedding cost is then asymmetrically adjusted into two polarity costs to further measure the detection risk of different modification polarities with the guidance of the reference image. After embedding with syndrome-trellis coding, the difference of the stegoimage and the reference image is minimized, which results in a high undetectability against steganalysis. Experimental results also prove that the proposed scheme performs much better than the existing steganographic methods when examined by modern steganalytic tools.

The detection of smoke in the initial stage is vital for preventing fire events. Therefore, we present a method of smoke heatmap detection using computer vision. First, the smoke region is segmented by encoder–decoder with atrous separable convolution (Deeplabv3+), and the edge of smoke is optimized with conditional random field to achieve pixel-level detection of early fire smoke. Subsequently, the heatmap of smoke thickness based on HSV or gray feature is established, and the space–time distribution of the smoke region is analyzed. In addition, generative adversarial network is used to predict the future frames and smoke trend heatmap, which will contribute to the development of fire protection and provide suggestions for rescue or evacuation. The experimental results show that the proposed method can accurately detect the fire smoke in different scenes and provide an effective heatmap analysis scheme, as well as provides basic data for further study on the trend of fire.

Multimodality image fusion provides more comprehensive information and has an increasingly wide range of uses. For the remote sensing image fusion, traditional multiresolution analysis (MRA)-based methods always have insufficiencies in contrast with spatial details. At the same time, traditional sum of modified Laplace may do blocking artifacts. In order to overcome these deficiencies, we propose a remote sensing image fusion method based on the mutual-structure for joint filtering and saliency detection. Our method uses joint filtering to facilitate the correct extraction of the high and low frequency from source images. The saliency detection method also improves the effect of low-frequency fusion, and the high-frequency sub-bands calculate the extended sum of modified Laplace for better fusion. The method is compared with other five classical fusion methods. The experimental results show that the algorithm effectively preserves the structural information and textural information of the image and improves the sharpness of the fused image. It turns out to have many advantages in subjective and objective evaluation.

Iris recognition (IR) is widely adopted for human identification in the biometric fields. However, results using handcrafted features designed with the domain knowledge are obtained by conventional IR algorithms which are complex. Convolutional neural networks (convnets) have been proved to be robust with superior learning capability in many computer vision fields. The deep convnets perform better than shallower convnets. However, training the deep convnets for IR is difficult. IrisConvDeeper, an adaptive architecture based on convnets combining Softmax classifier that is used to alleviate the problems of training deep convnets for IR, is proposed. In addition, a better performance is obtained. Furthermore, a shallower architecture termed as IrisConvShallower is designed for comparison. IrisConvDeeper consists of structures called dense block and has 12 or 14 convolutional layers. Meanwhile, IrisConvShallower consists of six or seven standard convolutional layers. The performance of the proposed architectures is tested on two public iris databases, which are collected under different conditions: CASIA-Iris-V3 and IITD iris image databases. The experimental results demonstrate that the proposed IrisConvDeeper outperforms most of state-of-the-art approaches in terms of correct recognition rate, except some approaches that adopted specific methods and identified fewer subjects compared with our methods.

The accurate detection and localization of an eye in a facial image is important for many computer vision applications, such as face recognition, fatigued driving detection, and gaze estimation. Although research on eye detection has matured, most existing eye detection methods produce poor performance in various practical scenarios where there exists variation in facial expressions or illumination, people wearing clear eyeglasses, and so on. We have proposed a method that can locate eyes under the above-mentioned varied environmental conditions. The proposed approach follows two steps: eye candidate detection and eye candidate verification. In the first step, two features, namely semicircular edge shape and semiellipse edge shape features, are proposed to detect the eye candidates. In the second step, all of the selected eye candidates are verified using a support vector machine trained with the fusion of local binary pattern, cell mean intensity, and histogram of oriented gradients features. The proposed method has been tested under different conditions of AR, the Chinese Academy of Sciences’ Pose, Expression, Accessories, and Lighting, and the facial recognition technology databases. The experimental results suggest that the proposed method provides better performance as compared to the existing methods in terms of precision and recall.

Due to the increasing need for testing solutions to complex hardware designs, several efforts were made in order to improve high-level synthesis (HLS) techniques. These solutions are conceived in such a way that they had to provide reasonable agreements in terms of design time, resources involved, and performance. Generally speaking, two main constraints should be satisfied for HLS applications. The first constraint consists in the ability to process complex systems at a reasonable cost, whereas the second revolves around considering some test constraints in the first tasks of the HLS flow. To fulfill these two constraints, we treated a case study using HLS for intraprediction, dequantization, and inverse transform decoding blocks of an high efficiency video coding (HEVC) decoder. For this experiment, version 10 was used of the HEVC test model (HM) reference software containing >200 functions and over 8000 lines of code. In addition, the suggested algorithm was implemented in an software/hardware (SW/HW) environment using Xilinx ZC 702-based platform. Finally, taking advantage of HLS optimization methods, the hardware design can process 6, 13, 71, and 285 video frames per second for 1600p, 1080p, 480p, and 240p video resolutions, respectively. Yet, the SW/HW designs can only decode 0.5, 1.5, 4, and 15.2 frames per second for the same video resolutions, i.e., with a gain of 3% in frame rate and 60% in power consumption compared to SW implementation.

Double JPEG compression is often used to conceal some illegal operations on the source image. Since there exist some traces when an image undergoes double JPEG compression, the detection of double JPEG compression can be used as a tool of digital forensics. In the past decade, researchers have focused on detecting double compression with the same quality factor (QF) but cannot provide reliable results when the QF is approximately below 70. To remedy this, we propose to analyze the low-frequency components and the error image and extract two types of features to improve the detection accuracy. Since the high-frequency components are easily lost in the JPEG compression, the low-frequency components are utilized to extract the reliable features. With this in mind, our method first analyzes the low-frequency discrete cosine transform coefficients and the lost information. Then, the features are extracted to characterize the difference between the two sequential compressions. Finally, the feature set is fed to a support vector machine for classification. Experimental results on two standard image databases verify that our method could improve the detection accuracy of the low QF double compressed JPEG images with the same QF. The average classification accuracy of the different QFs (the range of QF is usually set from 70 to 20) was obtained as 87.75%.

We propose a weakly supervised learning algorithm with size constraints based on modified deep convolutional neural networks (CNN) to segment the optic disc in fundus images. Comparing with the existing fully supervised method, we only use image-level labels and bounding box labels to guide segmentation. To obtain a more accurate coarse foreground segmentation map with image-level labels and treat them as “GroundTruth” for the next training stage, we combine the improved constraint CNN method and GrabCut method to generate the coarse foreground segmentation map. Then we design a weak loss function to constrain the output size and training network base on a modified U-net model with the generated foreground segmentation map. The proposed algorithm demonstrates state-of-the-art results on RIM-ONE database and DRISHTI-GS database.

Images captured on a hazy day are often degraded by the particle matter suspended in air, which introduces haze that severely distorts the quality of the acquired data. Eliminating haze, therefore, is a critical task in real-world applications. We propose a deep light-weight residual model, called image dehazing deep fully convolutional network (ID-DFCN), based on residual learning that directly projects the given hazy image to the residual onto the hazy image and the corresponding haze-free image. As a result, the haze-free image is obtained by adding the estimated residual. The proposed ID-DFCN is an end-to-end and light-weight model, which enables efficient hardware implementation. Qualitative and quantitative evaluations on synthetic and real-world hazy images show that the proposed model achieves comparable and even superior results in comparison to several state-of-the-art methods.

Image prior and sparse coding learning methods have important uses in image denoising. Many denoising methods learn priors either from the noisy image itself or an external clean image dataset. But using only these as priors does not always reconstruct the image effectively. In addition, when the image is corrupted by noise, the local sparse coding coefficient obtained from a noisy image patch is inaccurate, restricting denoising performance. We present a noise removal framework based on external prior learning and an internal mean sparse coding method, making use of the innate sparsity and nonlocal self-similarity (NSS) of natural images. Specifically, we first obtain external priors from a clean natural image dataset by Gaussian mixture model. The external priors are applied to guide the subspace clustering of internal noisy image patches, and a compact dictionary is generated for each internal noisy patch cluster. Then an internal mean sparse coding strategy based on NSS is introduced into the sparse representation model, whose regularization parameters then are deduced through a Bayesian framework. An iterative shrinkage method is employed to solve the l₁-optimization problem in the sparse representation model. Application of the noise removal model to 16 test images demonstrates denoising performance exceeding other competing methods.

Recognizing microexpression serves as a vital clue for affective estimation. Fast and discriminative feature extraction has always been a critical issue for spontaneous microexpression recognition applications. A microexpression analysis framework is proposed by adaptively key frame extraction and representation. First, to remove redundant information in the microexpression video sequences, the key frame is adaptively selected on the criteria of structural similarity index between different face images, Second, robust principal component analysis is applied to obtain the sparse information in the key frame, which not only retains the expression attributes of the microexpression sequence, but also eliminates useless interference. Furthermore, we construct dual-cross patterns to get the final microexpressions representation for classification. Repeated comparison experiments were performed on the SMIC and CASME2 databases to evaluate the performance of the proposed method. Experimental results demonstrate that our proposed method gets higher recognition rates and achieves promising performance, compared with the traditional microexpression recognition.

In cross-view action recognition, there remains a challenge that the action representation will lack the ability of transfer learning when the feature space changes. To solve this problem, a cross-view action recognition approach using a bilayer discriminative model is proposed. We first extract the key poses to capture the essence of each action sequence and represent each key pose by a bag of visual words (BoVW) in a single view. We then construct a bipartite graph between the heterogeneous poses and apply multipartitioning to cocluster the view-dependent visual words for developing the cross view bags of visual words feature, which is more discriminative in the presence of view changes. The novelty is to design a bilayer classifier consisting of SVM and HMM at the frame level and sequence level, respectively, to make up for the loss of temporal information when using a BoVW to represent the whole action sequence. Finally, DTW is used as a pruning algorithm to lessen the number of nodes for searching the Viterbi path. Extensive experiments are performed on two well-known multiple view action datasets IXMAS and N-UCLA, and a detailed performance comparison with the existing view-invariant action recognition techniques indicates that the proposed method works equally well in accuracy and efficiency.

The problem of learning a similarity measure for cross-view person identification in a disjointed camera network is addressed. Our learning framework is based on the projected gradient approach and is suitable for large-scale applications. The hinge loss with the triplet constraints is used as the objective function. Contrary to other approaches where a dual formulation is used, we propose to optimize the objective function in the primal form, with the help of a minibatch gradient descent algorithm. The choice of learning rate and its schedule are nontrivial tasks in such optimization schemes. We have studied empirically the effect of using different strategies for learning rate schedules. Our experimental study includes conventional strategies such as fixed or diminishing learning rates and recently developed adaptive gradient methods such as root-mean-square propagation (RMSProp) for learning rate schedule. The experimental results are presented on three benchmark datasets of person reidentification, namely VIPeR, CUHK Campus, and CUHK03. Our experimental results demonstrate that the RMSProp is well suited for the proposed learning framework. A comparison with recent methods in the domain presents the justification of the proposed approach.

Rainy images severely degrade visibility. Thus, deraining is an important task for applications ranging from image processing to computer vision. We propose a deep learning-based method to remove rain streaks from a single image. Specifically, we first design a deraining unit that employs dilation convolution and squeeze-and-excitation operations, respectively, to obtain more spatial contextual information and semantic correlation. In the deraining unit, multifeatures at different levels can be obtained by using convolutions with different dilation factors, and they are fused to maintain the primary features of rain streaks. Then, we interconnect the deraining units by dense connections that can maximize the information flow along features from different levels and make them be associated. Both deraining units and dense connections make our network have stronger representative ability of the rain streaks layer. Experimental results show that our proposed deraining method outperforms state-of-the-art methods by a good margin in Rain100H, Rain100L, and Rain1200 datasets, while using fewer parameters.

We propose an approach for jointly filling holes and upsampling depth information for RGB-D images captured with common acquisition systems, where RGB color information is available at all pixel locations whereas depth information is only available at lower resolution and entirely missing in small regions referred to as “holes.” Depth information completion is formulated as a minimization of an objective function composed of two additive terms. The first data fidelity term penalizes disagreement with the observed low-resolution data. The second regularization term penalizes weighted depth deviations from a local linear model in spatial coordinates, where the weights are experimentally determined to ensure consistency between the RGB color image and the estimated depth image. Analogous to techniques used for optimization formulations of image matting, the completed depth image is then obtained by solving a large sparse linear system of equations. We also propose a memory-efficient implementation of the proposed method based on the conjugate gradient method. Visual evaluation of results obtained with the proposed algorithm demonstrates that the method provides high-resolution depth maps that are consistent with the color images. Furthermore, the memory-efficient implementation significantly reduces memory requirements, allowing for computation of the upsampled, hole-filled depth maps for typical RGB-D images on normal workstation hardware. Quantitative comparisons demonstrate that the method offers an improvement in accuracy over the current state-of-the-art techniques for depth information completion. Importantly, statistical analysis, which we present in this paper, also reveals that prior evaluations of depth upsampling accuracy are potentially biased because the evaluations inappropriately used preprocessed hole-filled data as “ground truth.” An implementation of the proposed algorithm can be accessed and executed through Code Ocean: https://codeocean.com/capsule/5103691/tree/v1.

Recently, correlation filter-based trackers have been widely investigated due to their high efficiency and robustness. However, most of them use a fixed cosine window to deal with boundary effects and ignore the reliability of tracking results, which results in poor tracking performance when the target endures severe occlusion and large appearance variation. In order to deal with these issues, we propose a tracking framework with an adaptive cosine window, which is composed of a reliability estimation module and a redetection module. First, we incorporate the object likelihood map into the traditional fixed cosine window to form an adaptive cosine window, which can enlarge the searching region and effectively cope with boundary effects. Second, the peak-to-sidelobe ratio of HOG-based correlation response map and the color score of each frame are adopted to estimate the reliability of tracking results. Third, we introduce the Siamese tracker to redetect targets in case of tracking failures. Finally, a target pyramid scheme is built to deal with scale variation. Extensive experiments on the OTB-2013, OTB-2015, TemplateColor, UAV123@10 fps, and VOT-2015 demonstrate that our proposed method outperforms favorably against the state-of-the-art methods with real-time tracking speed.

We present a channel-wise (CW) framework for data embedding in display images using barcodes. Data are embedded in one or more channels of the color image encoded as a monochrome barcode pattern and extracted from the corresponding channels by the decoder. We highlight two advantageous schemes, realized in the proposed framework: red, green, blue CW barcodes (RGB-CWBs), and blue channel image-barcodes (BC-IBs). Compared with a conventional monochrome black and white barcode BW-B with the same spatial footprint, RGB-CWBs utilize the three channels of color image more effectively and enable a three-fold increase in data rate, whereas BC-IBs significantly reduce the embedding distortion by exploiting the reduced sensitivity of the visual system to spatial detail in blue colors. For RGB-CWBs, we develop an effective framework for the cancellation of cross-channel interference between the display and capture channels using a model for the display and capture processes and for the ambient illumination. Model parameters required for interference cancellation (IC) are learned on-the-fly using either expectation-maximization or through suitable design of the synchronization patterns as “pilot-blocks.” Extensive experiments over multiple display and capture devices validate the advantages of the framework: RGB-CWBs (with IC) and the BC-IBs offer bit error rates comparable to their monochrome counterparts while providing either a three-fold increase in data rate or a significant reduction in distortion, respectively.

Due to the less constrained setup in standoff iris recognition systems, it is likely to capture nonideal iris images with gaze angle, pupil dilation, reflections, and occlusions. The combined effect of pupil dilation and gaze angle on iris recognition is examined. We first highlight the effects on synthetic images generated with a biometric eye model using a ray-tracing algorithm. Then, we quantify the effects of pupil dilation and gaze angle on the real frontal and off-angle images at different dilation levels. Our experiments reveal that the larger differences in dilation levels and gaze angles between the compared iris images increase the Hamming distance. Even if the linear rubber-sheet normalization helps to minimize the dilation effect in frontal images, it cannot fully eliminate it in off-angle iris images because of not only the pupil dilation and three-dimensional iris texture but also the corneal refraction distortion and limbus occlusion. We also observe that the gaze angle is the main reason for the performance degradation in steeper off-angle images, where the effect of the dilation is limited. In addition, since the iris region in off-angle and dilated iris images is smaller than that in frontal and constricted iris images, their interclass Hamming distance distribution is shifted toward the intraclass distribution, which may increase the false match rate.

As streak tube imaging is a concentric spherical electrostatic focusing system, geometric distortion is caused by the relative change in radial magnification. This reduces the spatial resolution of imaging. The physical principle of changing the radial magnification of an electronic optical imaging system is analyzed to correct the geometric distortion of the imaging system and to improve spatial resolution. Distortion is corrected using the calibration model of a spherical optical system. The contrast of a reticle intensity image is compared before and after correction, and the ultimate resolution is calculated. Contrast is increased by ∼4  %  , and the ultimate resolution is increased by ∼2  lp  /  mm after correction.

Correlation filter-based tracking methods realize dense sampling via the circular shift of an image patch, thereby improving the generalization ability of the trackers. However, the discriminative power of these methods is still limited due to the boundary effect and ridge regression framework. Instead of a traditional correlation filter, a classifier is learned in the form of a support correlation filter. Aiming at the enhancement of the discriminative power, the support correlation filter learning on the training samples is accomplished under the large margin framework. Furthermore, a cropping operation is implemented on each feature map to extract the central part of the feature as a reliable training sample. In particular, this operation is implicitly carried out through the integration into the learning framework. An alternating optimization algorithm is further developed to obtain the solution for the large margin support correlation filter. The experimental results on the benchmark dataset demonstrate the superior performance of the method.

Object detection involves solving two main problems: identifying the object and its location. This transforms the problem into a classification and localization problem. Currently, research shows that a convolutional neural network (CNN) can be used to solve the classification problem, and the object detection module incorporating a region proposal method can be used to locate the object. Although a CNN based on region proposals can achieve high recall, which improves detection accuracy, its performance cannot meet the actual requirements for small-size object detection and precise localization. This is mainly due to the feature maps extracted from the CNN and the quality of the region proposals. We present a cross-layer fusion feature network (CLFF-Net) for both high-quality region proposal generation and accurate object detection. The CLFF-Net is based on the cross-layer fusion feature that extracts hierarchical feature maps and then aggregates them into a unified space. The fused feature map appropriately combines deep layer semantic information, middle layer supplemental information, and shallow layer location information for an image to build the CLFF-Net, which is shared for both generating region proposals and detecting objects via end-to-end training. Extensive experiments using a casting dataset demonstrate its promising performance compared to state-of-the-art approaches.

Remote sensing images recognition technology has great significance in many aspects, such as military navigation and environmental monitoring. We propose an improved single shot multibox detector approach by combining some strategies, including upsampling, focal loss, and proper calibration of key parameters. Comprehensive experiments on three remote sensing images datasets have demonstrated the effectiveness of the proposed approach in benchmarking with several state-of-the-art object detection methods.

Proper classification of fingerprints still poses difficult issues in large-scale databases due to ambiguity in intraclass and interclass structures, discontinuity in low-quality images, and ridges. To address these challenges, we propose a feature named local diagonal and directional extrema pattern (LDDEP) as a descriptor for classification of fingerprints. The proposed method utilizes first-order derivatives to find values and indices of local diagonal and directional extremas. The local extrema values are then compared with the central pixel intensity value to find the correlation with the neighbors. Eventually, the descriptor is generated with the help of the indices and local extrema values. Furthermore, the proposed descriptor is fed into K-nearest neighbor and support vector machine (SVM) for classifying the fingerprint images into four and five groups, respectively. The LDDEP descriptor is compared with the existing methods on two databases, namely National Institute of Standards Technology Special Database 4 (NIST SD 4) and Fingerprint Verification Competition (FVC). Our experiments have shown that, on the 4000 image NIST SD 4 test dataset, the proposed descriptor achieved a classification accuracy of 95.15% for five classes and 96.85% for four classes for half of the dataset, and an accuracy of 95.5% for five classes and 96.63% for four classes for the entire test dataset using SVM classifier. Similarly, FVC databases for the LDDEP descriptor gave classification accuracy of 98.2% using SVM classifier. The proposed method gave higher accuracies compared to the existing methods.

We present a method of constructing a full thermal panorama from a collection of images taken by a long wavelength infrared and visible camera system. Due to recent development in joint stereo calibration, our method is able to project a thermal image onto a corresponding visible image. After both images are in the same domain, panoramic projection from visible images can also be applied to thermal images. Our method also extends the scope of the projection function by allowing nonzero distance between center of rotation and optical center of rotating camera as well as arbitrary camera’s primary axes that are no longer parallel to the world axes. We show that our method works on a dataset with various camera movements and compare our results with standard software made specifically for a visible panorama and a thermal panorama. Furthermore, we are able to combine thermal panoramas of the same scene with a simple calculation. With an addition of guided upsampling, we can simultaneously improve thermal image resolution and field of view with a single run of the method.

Chinese flower-bird characters are gems of traditional Chinese art. It is an artistic font and the strokes of the characters are designed as beautiful patterns of flowers, birds, etc. The generation of such characters requires painters’ great efforts. Imagine that if we only need to sketch the outline of the ideal flower-bird characters using a pencil, and then we can quickly obtain these artistic characters, which will be of great significance in promoting their development, allowing more people to appreciate and even create this art by computer. Recently, with the development of deep learning and the invention of generative adversarial networks (GANs), some studies on font generation have made new progress. However, there is no research on the generation of flower-bird characters. We provide a solution by designing a GAN-based architecture to generate flower-bird characters. More specifically, a generator inspired by U-Net translates pencil drawings or brush drawings to flower-bird characters, and a patch-level discriminator distinguishes whether the received image is real. In addition to adversarial loss, a valid loss term called structural similarity loss is designed to further drive the network to generate satisfactory images. The quantitative analysis and user perceptual validation show the effectiveness of our method.

For extended target and group target tracking, most existing random matrix approaches assume that the measurements are linear in the kinematic state and in the noise with its covariance being a random matrix to represent the extension of an extended target or target group. However, in practice, the measurements (e.g., range and bearing) are nonlinear in the kinematic state and extension noise. Only the matched linearization (ML) approach has been proposed to linearize the nonlinear measurement function based on minimum mean square error criterion. However, if the state prediction error is not small, it will incur large linearization error and then the variational Bayesian measurement update approach produces inaccurate estimation results. To this point, using the ML technology, we present a recursive update filtering-based variational Bayesian update (RUF-VBU) approach for better estimation performance. In addition, the square-root implementation of RUF-VBU is proposed for the recursive interruption problem caused by the nonpositive definite covariance matrix. The simulation results demonstrate the effectiveness of the proposed approach.

Image transmission holds a major share in data communication, and thus secure image transmission is currently a challenging domain of research. A secure image transmission scheme is proposed that physically transmits the encrypted image employing visual cryptography scheme (VCS). During physical transmission, the meaningless shares may attract curious hackers and if captured and stacked, the secret may be revealed. Moreover, the increase in transmission overhead due to multiple share images resulted from a single secret image after encryption is another concern regarding the physical implementation of VCS. Focusing on both observations, vector quantization (VQ) is used to encode as well as to compress each of the shares before transmission. To utilize VQ, its two parameters, cell width and dimension of grid, are needed to be optimized for various kind of images without compromising the randomness property of the shares. Hence, a particle swarm optimization-guided VQ is proposed, and furthermore, a multilayer perceptron in conjunction with an autoencoder are also trained in synchronism with that to automatically obtain the optimal VQ for each image type during the transmission. The proposed scheme is successfully implemented with different types of images for secure physical transmission with a 62.8% data volume reduction and 98.07% image quality retrieval.

Graph-based salient object detection has been widely applied in many applications, because of its excellent performance and strong theoretical basis. Basically, the performance of this type of methods depends on the correctness in foreground seed selection. In research aiming to exactly identify the seeds on foreground objects, an external prior has been defined in recent work as having an image boundary that is mostly background (called boundary prior), so the foreground seeds must locate around the image center. However, this is not the case when salient objects are spatially close to the image boundary. This problem will cause a severe error in salient object detection, because background noises are likely mixed in foreground seeds. To solve this problem, we propose a robust foreground seed selection method for salient object detection. In our method, the external prior and multiple internal image features are combined for foreground seed selection. Our method can relax the limitation of the external prior and make the foreground seed selection more adaptive and robust to diverse samples. As a result, the proposed method can generate satisfying results, no matter where the salient object is located. This advantage is demonstrated by experimental comparisons with several state-of-art methods.

We present extensive evaluations of deep features pretrained by state-of-the-art deep convolutional neural networks (DCNNs) for predictions of human fixations. The evaluations are conducted using a bottom-up saliency model, which utilizes deep features of DCNNs pretrained for object classification. Using various selections of deep feature maps, 35 implementations of the bottom-up saliency model are computed, evaluated, and compared over three publicly available datasets using four evaluation metrics. The experimental results demonstrate that the pretrained deep features are strong predictors of human fixations. The incorporation of multiscale deep feature maps benefits the saliency prediction. The depth of DCNNs has a negative effect on saliency prediction. Moreover, we also compare the performance of the proposed deep features-based bottom-up saliency model with the other eight bottom-up saliency models. The comparison results show that our saliency model can outperform other conventional bottom-up saliency models.

We present an algorithm for risk prediction of road surface grip where skidding and sliding occur as main road surface problems. Prediction is done by defining a fine texture classification of the properties of road aggregate. In an experimental setup, data acquisition is performed with a supervised mobile vehicle scanning system, using a vehicle equipped with a camera and temperature sensor during movement along an arterial road. Image processing is performed by testing four texture feature extraction methods: Gabor filters, wavelet transform, gray level co-occurence matrix, and edge histogram descriptor, among which the Gabor transform shows the best results. The extraction of texture feature vectors follows by statistical algorithms for measuring feature vector similarity and reference vector selection, leading to image texture classification. The algorithm itself is upgraded by incorporating simultaneous surface temperature measurements in order to create and validate the final fine surface texture classification. The roads are classified and segmented into high-, medium-, and low-risk roads according to skid danger, enabling the creation of a map of high-risk zones. We validate our risk prediction algorithm by referring to crash rate data from the Road Traffic Safety Agency of Serbia database. This algorithm enables the location and mapping of high-risk zones and can be used as a support for autonomous driving and navigation.

We propose an unsupervised segmentation method based on simple non-iterative clustering (SNIC) and adaptive density-based spatial clustering of applications with noise (DBSCAN). The method is not sensitive to parameter settings. And cluster parameter suitable for each image can be automatically calculated. SNIC superpixel segmentation is applied in achieving over-segmented images to solve the problem of the image resolution being too high. Then, adaptive DBSCAN clustering is proposed to cluster the over-segmented superpixel blocks to solve the problem of over-segmentation and manual adjustment of DBSCAN parameters. Finally, k-means and connected regions are used for postprocessing to remove the shadow superpixel blocks from the clustered image and to ensure the integrity of a single microstructure. The effectiveness of this method is proved by many experiments. Based on this method, we provide a fast labeling method to help experts quickly label metallographic images.

Quadratic image filter involves the second-order multiplications of an input image mask in addition to linear terms, and determining the weights of the quadratic filter using optimization methods requires intense computational power due to the cost of the resulting fitness function. A graphics processing unit (GPU)-based algorithm is proposed to determine quadratic image filter weights using genetic algorithms and particle swarm optimization methods. Since the most time consuming part of determining the mask weights using heuristic algorithms is the fitness computation process, the fitness computation process is designed to work on the GPU, and a significant acceleration is achieved. For this purpose, different designs such as direct method and population-based method have been developed within the GPU to minimize training time. According to experimental results, the proposed methods provide significant accelerations over sequential implementation. The first method computes the fitness function for each population member separately, and it produces about 43 times acceleration over the sequential implementation. The second method developed is based on the complete evaluation of the population, and it produces about 151 times acceleration over the sequential implementation.

In computer vision, the object tracking technique is a complex task to predict the object target under challenging task occlusion, scaling, and illumination. A robust object tracking algorithm is proposed based on adaptive patch feature matching and scale estimation. Initially, tracking technique extracts geometric locations using defined directional distances and object location of the previous frame. The feature matching is carried out between the previous frame featured vector and overlapping patches feature vectors of the present frame. The matched patch is bounded by the identified location in the current frame. For object scaling, speeded up robust features point matching technique is applied to an enlarged patch of the present frame and the tracked patch of the previous frame. Finally, the proposed algorithm updates motion, feature, and geometric location vectors to keep tracking between successive frames. The feature vector update between successive frames solves the aforementioned challenging issues in object tracking. For experimentation, the proposed algorithm is tested with available tracking benchmarks. The performance evaluation results show that the proposed technique is better in terms of computational time and accuracy compared to conventional tracking techniques.

Recently, discriminative correlation filters (DCF) have achieved enormous popularity in the tracking community due to high efficiency and fair robustness. With a circular structure, DCFs transform computationally consuming spatial correlation into efficient element-wise operation in the Fourier domain. We argue that this element-wise solution can be derived only in the case of single-channel features. In terms of tracking with multichannel features, this element-wise solution trains each feature dimension independently and fails to learn a joint correlation filter. To tackle this problem, we propose a rigorous solution to closed-form correlation filter tracking. This rigorous solution can be computed pixel by pixel from a small linear equation system. Experimental results demonstrate that our rigorous pixel-wise solution achieves better tracking performance than the baseline element-wise solution.

Recently, experiments show that deep super-resolution (SR) networks with error feedback mechanisms can achieve better accuracies than purely feed-forward networks. However, such error feed-back networks use one-step mapping, which increases the difficulty for training with large-scale factors. We first propose a stage-progressive back-projection network to progressively reconstruct images by simply dividing the entire back-projection stage into several levels. We convert one-step large-scale sampling into multiple samplings with a moderate scale factor through such division. At each level, we process low-resolution-to-high-resolution/high-resolution-to-low-resolution mapping with a scale factor of 2. In order to enhance the effect of dense connections and to further improve the performance, we propose the unit-progressive back-projection network, in which we construct progressive projection units to avoid one-step mappings with large-scale factors. Additionally, we recommend a subpixel convolutional layer and its inverse transform as the mapping layer since it computes each pixel of the outputs covering more pixels from the inputs. Extensive quantitative and qualitative evaluations on benchmark datasets show that our algorithms substantially improve performance on large-scale SR tasks.