This PDF file contains the editorial “Special Section Guest Editorial:Special Section on Quality Control by Artificial Vision: Nonconventional Imaging Systems” for JEI Vol. 24 Issue 06

Activated sludge process is a widely used method to treat domestic and industrial effluents. The conditions of activated sludge wastewater treatment plant (AS-WWTP) are related to the morphological properties of flocs (microbial aggregates) and filaments, and are required to be monitored for normal operation of the plant. Image processing and analysis is a potential time-efficient monitoring tool for AS-WWTPs. Local adaptive segmentation algorithms are proposed for bright-field microscopic images of activated sludge flocs. Two basic modules are suggested for Otsu thresholding-based local adaptive algorithms with irregular illumination compensation. The performance of the algorithms has been compared with state-of-the-art local adaptive algorithms of Sauvola, Bradley, Feng, and c-mean. The comparisons are done using a number of region- and nonregion-based metrics at different microscopic magnifications and quantification of flocs. The performance metrics show that the proposed algorithms performed better and, in some cases, were comparable to the state-of the-art algorithms. The performance metrics were also assessed subjectively for their suitability for segmentations of activated sludge images. The region-based metrics such as false negative ratio, sensitivity, and negative predictive value gave inconsistent results as compared to other segmentation assessment metrics.

An image detail enhancement method to effectively visualize low contrast targets in high-dynamic range (HDR) infrared (IR) images is presented regardless of the dynamic range width. In general, high temperature dynamics from real-world scenes used to be encoded in a 12 or 14 bits IR image. However, the limitations of the human visual perception, from which no more than 128 shades of gray are distinguishable, and the 8-bit working range of common display devices make necessary an effective 12/14  bits HDR mapping into the 8-bit data representation. To do so, we propose to independently treat the base and detail image components that result from splitting the IR image using two dedicated guided filters. We also introduce a plausibility mask from which those regions that are prominent to present noise are accurately defined to be explicitly tackled to avoid noise amplification. The final 8-bit data representation results from the combination of the processed detail and base image components and its mapping to the 8-bit domain using an adaptive histogram-based projection approach. The limits of the histogram are accommodated through time in order to avoid global brightness fluctuations between frames. The experimental evaluation shows that the proposed noise-aware approach preserves low contrast details with an overall contrast enhancement of the image. A comparison with widely used HDR mapping approaches and runtime analysis is also provided. Furthermore, the proposed mathematical formulation enables a real-time adjustment of the global contrast and brightness, letting the operator adapt to the visualization display device without nondesirable artifacts.

Different texture descriptors are proposed for the automatic classification of skin lesions from dermoscopic images. They are based on color texture analysis obtained from (1) color mathematical morphology (MM) and Kohonen self-organizing maps (SOMs) or (2) local binary patterns (LBPs), computed with the use of local adaptive neighborhoods of the image. Neither of these two approaches needs a previous segmentation process. In the first proposed descriptor, the adaptive neighborhoods are used as structuring elements to carry out adaptive MM operations which are further combined by using Kohonen SOM; this has been compared with a nonadaptive version. In the second one, the adaptive neighborhoods enable geometrical feature maps to be defined, from which LBP histograms are computed. This has also been compared with a classical LBP approach. A receiver operating characteristics analysis of the experimental results shows that the adaptive neighborhood-based LBP approach yields the best results. It outperforms the nonadaptive versions of the proposed descriptors and the dermatologists’ visual predictions.

The leadframe fabrication process normally involves additional thin-metal layer plating on the bulk copper substrate surface for wire bonding purposes. Silver, tin, and copper flakes are commonly adopted as plating materials. It is critical to assess the density of the plated metal layer, and in particular to look for porosity or voids underneath the layer, which may reduce the reliability during high-temperature stress. A fast, reliable inspection technique is needed to assess the porosity or void weakness. To this end, the characteristics of x-rays generated from bulk samples were examined using an energy-dispersive x-ray (EDX) detector to examine the porosity percentage. Monte Carlo modeling was integrated with Castaing’s formula to verify the integrity of the experimental data. Samples with different porosity percentages were considered to test the correlation between the intensity of the collected x-ray signal and the material density. To further verify the integrity of the model, conventional cross-sectional samples were also taken to observe the porosity percentage using Image J software measurement. A breakthrough in bulk substrate assessment was achieved by applying EDX for the first time to nonelemental analysis. The experimental data showed that the EDX features were not only useful for elemental analysis, but also applicable to thin-film metal layer thickness measurement and bulk material density determination. A detailed experiment was conducted using EDX to assess the plating metal layer and bulk material porosity.

Electroencephalography (EEG) is the signal generated by electrical activity in the human brain. EEG topographic maps (topo-maps) give an idea of brain activation. Functional connectivity helps to find functionally integrated relationship between spatially separated brain regions. Brain connectivity can be measured by several methods. The classical methods calculate the coherence and correlation of the signal. We have developed an algorithm to map functional neural connectivity in the brain by using a full search block matching motion estimation algorithm. We have used oddball paradigm to examine the flow of activation across brain lobes for a specific activity. In the first step, the EEG signal is converted into topo-maps. The flow of activation between consecutive frames is tracked using full search block motion estimation, which appears in the form of motion vectors. In the second step, vector median filtering is used to obtain a smooth motion field by removing the unwanted noise. For each topo-map, several activation paths are tracked across various brain lobes. We have also developed correlation activity maps by following the correlation coefficient paths between electrodes. These paths are selected when the correlation coefficient between electrodes is >70%. We have compared the motion estimation path with the correlation coefficient activation maps. The tracked paths obtained by using motion estimation and correlation give very similar results. The inter-subject comparison shows that four out of five subjects tracked path involves all four (occipital, temporal, parietal, frontal) brain lobes for the same stimuli. The intra-subject analysis shows that three out of five subjects show different tracked lobes for different stimuli.

A new three-dimensional metric reconstruction method for straight homogeneous generalized cylinders with elliptical cross-section (SHGC-E) from a single image is addressed. By using an elliptical truncated cone shape, the elliptical cross-section is computed by back-projection. The conditions under which the imaged limb can be considered an imaged meridian under perspective projection are discussed and proved. Furthermore, the imaged meridian of SHGC-E is identified and recovered by some of the proposed special properties of SHGC-E.

We propose a model of depth camera based on a four-lens device. This device is used for validating alternate approaches for calibrating multiview cameras and also for computing disparity or depth images. The calibration method arises from previous works, where principles of variable homography were extended for three-dimensional (3-D) measurement. Here, calibration is performed between two contiguous views obtained on the same image sensor. This approach leads us to propose a new approach for simplifying calibration by using the properties of the variable homography. Here, the second part addresses new principles for obtaining disparity images without any matching. A fast algorithm using a contour propagation algorithm is proposed without requiring structured or random pattern projection. These principles are proposed in a framework of quality control by vision, for inspection in natural illumination. By preserving scene photometry, some other standard controls, as for example calipers, shape recognition, or barcode reading, can be done conjointly with 3-D measurements. Approaches presented here are evaluated. First, we show that rapid calibration is relevant for devices mounted with multiple lenses. Second, synthetic and real experimentations validate our method for computing depth images.

Segmentation and classification are prolific research topics in the image processing community. These topics have been increasingly used in the context of analysis of cementitious materials on images acquired with a scanning electron microscope. Indeed, there is a need to be able to detect and to quantify the materials present in a cement paste in order to follow the chemical reactions occurring in the material even days after the solidification. We propose a new approach for segmentation and classification of cementitious materials based on the denoising of the data with a block-matching three-dimensional (3-D) algorithm, binary partition tree (BPT) segmentation, support vector machines (SVM) classification, and interactivity with the user. The BPT provides a hierarchical representation of the spatial regions of the data, allowing a segmentation to be selected among the admissible partitions of the image. SVMs are used to obtain a classification map of the image. This approach combines state-of-the-art image processing tools with user interactivity to allow a better segmentation to be performed, or to help the classifier discriminate the classes better. We show that the proposed approach outperforms a previous method when applied to synthetic data and several real datasets coming from cement samples, both qualitatively with visual examination and quantitatively with the comparison of experimental results with theoretical ones.

This paper deals with an automated preflight aircraft inspection using a pan-tilt-zoom camera mounted on a mobile robot moving autonomously around the aircraft. The general topic is image processing framework for detection and exterior inspection of different types of items, such as closed or unlatched door, mechanical defect on the engine, the integrity of the empennage, or damage caused by impacts or cracks. The detection step allows to focus on the regions of interest and point the camera toward the item to be checked. It is based on the detection of regular shapes, such as rounded corner rectangles, circles, and ellipses. The inspection task relies on clues, such as uniformity of isolated image regions, convexity of segmented shapes, and periodicity of the image intensity signal. The approach is applied to the inspection of four items of Airbus A320: oxygen bay handle, air-inlet vent, static ports, and fan blades. The results are promising and demonstrate the feasibility of an automated exterior inspection.

This work aims to present an extended framework for automatically recognizing suspicious activities in outdoor perimeter surveilling systems based on infrared video processing. By combining size-, speed-, and appearance-based features, like the local phase quantization and the histograms of oriented gradients, actions of small duration are recognized and used as input, along with spatial information, for modeling target activities using the theory of hidden conditional random fields (HCRFs). HCRFs are used to classify an observation sequence into the most appropriate activity label class, thus discriminating high-risk activities like trespassing from zero risk activities, such as loitering outside the perimeter. The effectiveness of this approach is demonstrated with experimental results in various scenarios that represent suspicious activities in perimeter surveillance systems.

Today, industries ensure the quality of their manufactured products through computer vision techniques and nonconventional imaging. Three-dimensional (3-D) scanners and nondestructive testing (NDT) systems are commonly used independently for such applications. Furthermore, these approaches combined constitute hybrid systems, providing a 3-D reconstruction and NDT analysis. These systems, however, suffer from drawbacks such as errors during the data fusion and higher cost for manufacturers. In an attempt to solve these problems, a single active thermography system based on scanning-from-heating is proposed in this paper. In addition to 3-D digitization of the object, our contributions are twofold: (1) the nonthrough defect detection for a homogeneous metallic object and (2) fiber orientation assessment for a long fiber composite material. The experiments on steel and aluminum plates show that our method achieves the detection of nonthrough defects. Additionally, the estimation of the fiber orientation is evaluated on carbon-fiber composite material.

This paper proposes a two-scale approach for the description of fibrous materials from tomographic data. It operates at two scales: coarse scale to describe weaving patterns and fine scale to depict fiber layout within yarns. At both scales, the proposed approach starts with the segmentation of yarns and fibers. Then, the fibrous structure (fiber diameters, fiber and yarn orientations, fiber density within yarns) is described. The segmentation algorithms are applied to a chunk of a woven ceramic-matrix composite observed at yarn and fiber scales using tomographic data from the European synchrotron radiation facility. The fiber and yarn segmentation results allow investigation of intrayarn fiber layout. The analysis of intrayarn fiber density and orientations shows the effects of the weaving process on fiber organization, in particular fiber compaction or yarn shearing. These results pave the way toward a deeper analysis of such materials. Indeed, the data collected with the proposed methods are a key starting point for realistic image synthesis. Such images may in turn be used to validate the fiber and yarn segmentation algorithms. Besides, and above all, they will allow material behavior simulation, aiming at the evaluation of the material’s strengths and weaknesses inferred from its fibrous architecture.

This paper addresses the use of orthogonal polynomial basis transform in video classification due to its multiple advantages, especially for multiscale and multiresolution analysis similar to the wavelet transform. In our approach, we benefit from these advantages to reduce the resolution of the video by using a multiscale/multiresolution decomposition to define a new algorithm that decomposes a color image into geometry and texture component by projecting the image on a bivariate polynomial basis and considering the geometry component as the partial reconstruction and the texture component as the remaining part, and finally to model the features (like motion and texture) extracted from reduced image sequences by projecting them into a bivariate polynomial basis in order to construct a hybrid polynomial motion texture video descriptor. To evaluate our approach, we consider two visual recognition tasks, namely the classification of dynamic textures and recognition of human actions. The experimental section shows that the proposed approach achieves a perfect recognition rate in the Weizmann database and highest accuracy in the Dyntex++ database compared to existing methods.

The conception of a prototype combining two hyperspectral cameras, one ranging from visible to near-infrared and the other covering short-wave infrared, is presented. The prototype aims at the characterization of millimeter-sized metallic alloys particles, originating from end-of-life vehicles and waste electrical and electronic equipment recycling. This paper is meant to serve as a support for a similar project by presenting difficulties encountered and available solutions. The calibration steps necessary to obtain quality reflectance data are also described. Classification results obtained on 100 metallic fragments dataset are finally presented.

This paper presents two automatic methods for visual grading, deterministic and probabilistic, designed to solve the industrial problem of evaluation of seed lots from the characterization of a representative sample. The sample is thrown in bulk onto a tray placed in a chamber for acquiring color image in a controlled and reproducible manner. Two image-processing methods have been developed to separate and then characterize each seed present in the image. A shape learning is performed on isolated seeds. Collected information is used for the segmentation. The first approach adopted for the segmentation step is based on simple criteria such as regions, edges, and normals to the boundary. Marked point processes are used in the second approach, leading to tackling of the problem by a technique of energy minimization. In both approaches, an active contour with prior shape is performed to improve the results. A classification is done on shape or color descriptors to evaluate the quality of the sample.

A stable and unsupervised version of the fuzzy C-means algorithm, named FCM-optimized (FCMO), is presented. The originality of the proposed algorithm stems from (1) the introduction of an adaptive incremental procedure to initialize class centers, which makes the algorithm stable and deterministic; therefore, the classification results do not vary from one run to another and (2) the use of an unsupervised evaluation criterion to estimate the optimal number of classes. The validation of FCMO with regard to stability, reliability in class number estimation, and classification efficiency is shown through experimental results on synthetic monocomponent and real multicomponent images.

We focus on quality control of mechanical parts in aeronautical context using a single pan-tilt-zoom (PTZ) camera and a computer-aided design (CAD) model of the mechanical part. We use the CAD model to create a theoretical image of the element to be checked, which is further matched with the sensed image of the element to be inspected, using a graph theory–based approach. The matching is carried out in two stages. First, the two images are used to create two attributed graphs representing the primitives (ellipses and line segments) in the images. In the second stage, the graphs are matched using a similarity function built from the primitive parameters. The similarity scores of the matching are injected in the edges of a bipartite graph. A best-match-search procedure in the bipartite graph guarantees the uniqueness of the match solution. The method achieves promising performance in tests with synthetic data including missing elements, displaced elements, size changes, and combinations of these cases. The results open good prospects for using the method with realistic data.

We are interested in the performance of currently available algorithms for the detection of cracks in the specific context of aerial inspection, which is characterized by image quality degradation. We focus on two widely used families of algorithms based on minimal cost path analysis and on image percolation, and we highlight their limitations in this context. Furthermore, we propose an improved strategy based on a-contrario modeling which is able to withstand significant motion blur due to the absence of various thresholds which are usually required in order to cope with varying crack appearances and with varying levels of degradation. The experiments are performed on real image datasets to which we applied complex blur, and the results show that the proposed strategy is effective, while other methods which perform well on good quality data experience significant difficulties with degraded images.

The three-dimensional (3-D) workflow (acquisition-processing-compression) is, in most cases, sequenced into several independent steps. Such approaches result in an acquisition of an important number of 3-D points. After acquisition, the first processing step is a simplification of the data by suppressing many of the computed points. We propose a coarse-to-fine acquisition system that makes it possible to obtain simplified data directly from the acquisition. By calculating some complementary information from two-dimensional (2-D) images, such as 3-D normals, multiple-homogeneous regions will be segmented and affected for a given primitive class. In contrast to other studies, the whole process is not based on a mesh. The obtained model is simplified directly from the 2-D data acquired by a 3-D scanner.

Capsule quality inspection is important and necessary in the pharmaceutical industry. The popular methods often mis-detect capsule head defects. To solve this problem, we propose a high-quality visual defect inspection method for capsule heads. In detail, first, capsule head images are captured by high-speed cameras with ring illuminators. Then, radial symmetry transform (RST) is employed to locate region of interest (ROI). Next, the ROI image is enhanced by homomorphic filter and binarized by basic global thresholding. After that, six discriminative features of ROI are extracted, which are skeleton feature, binary density, number of connected boundaries, RST power, mean, and variance. Finally, these features are classified by support vector machine to inspect the quality of the capsule head. The experiment is carried out on a self-established capsule image database, Northeastern University Capsule Image Database Version 1.0. According to our experiment, the proposed method can detect ROI correctly for all of the capsule head images and inspection accuracy achieves a true positive rate of 100.00% and true negative rate of 100.00%.

This PDF file contains the editorial “Special Section Guest Editorial:Special Section on Ultrawide Context- and Content-Aware Imaging” for JEI Vol. 24 Issue 06

This paper proposes a solution to contrast agent (CA) detection in angiograms by considering x-ray images as intensity images and applying mathematical morphology operators. We present two detection approaches, one based on the intensity infimum and the other based on the dual reconstruction. The evaluation using several data sets shows that both techniques are able to detect the presence of the contrast medium volume. Moreover, the dual reconstruction–based method is proven to be faster in processing time and more effective than the intensity infimum–based method in distinguishing the intensity change at the same location from the displacement of the same region. In addition, we show how to track the CA passage through a region of interest by observing the intensity evolution in successive submasks.

Combined databases of road markings and traffic signs provide a complete and full description of the present traffic legislation and instructions. Such databases contribute to efficient signage maintenance, improve navigation, and benefit autonomous driving vehicles. A system is presented for the automated creation of such combined databases, which additionally investigates the benefit of this combination for automated contextual placement analysis. This analysis involves verification of the co-occurrence of traffic signs and road markings to retrieve a list of potentially incorrectly signaled (and thus potentially unsafe) road situations. This co-occurrence verification is specifically explored for both pedestrian crossings and yield situations. Evaluations on 420 km of road have shown that individual detection of traffic signs and road markings denoting these road situations can be performed with accuracies of 98% and 85%, respectively. Combining both approaches shows that over 95% of the pedestrian crossings and give-way situations can be identified. An exploration toward additional co-occurrence analysis of signs and markings shows that inconsistently signaled situations can successfully be extracted, such that specific safety actions can be directed toward cases lacking signs or markings, while most consistently signaled situations can be omitted from this analysis.

Electroencephalography (EEG) and magnetic resonance imaging (MRI) are noninvasive neuroimaging modalities. They are widely used and could be complementary. The fusion of these modalities may enhance some emerging research fields targeting the exploration better brain activities. Such research attracted various scientific investigators especially to provide a convivial and helpful advanced clinical-aid tool enabling better neurological explorations. Our present research was, in fact, in the context of EEG inverse problem resolution and investigated an advanced estimation methodology for the localization of the cerebral activity. Our focus was, therefore, on the integration of temporal priors to low-resolution brain electromagnetic tomography (LORETA) formalism and to solve the inverse problem in the EEG. The main idea behind our proposed method was in the integration of a temporal projection matrix within the LORETA weighting matrix. A hyperparameter is the principal fact for such a temporal integration, and its importance would be obvious when obtaining a regularized smoothness solution. Our experimental results clearly confirmed the impact of such an optimization procedure adopted for the temporal regularization parameter comparatively to the LORETA method.

Salient objects’ detection and segmentation are gaining increasing research interest in recent years. A saliency map can be obtained from different models presented in previous studies. Based on this saliency map, the most salient region (MSR) in an image can be extracted. This MSR, generally a rectangle, can be used as the initial parameters for object segmentation algorithms. However, to our knowledge, all of those saliency maps are represented in a unitary resolution although some models have even introduced multiscale principles in the calculation process. Furthermore, some segmentation methods, such as the well-known GrabCut algorithm, need more iteration time or additional interactions to get more precise results without predefined pixel types. A concept of a multiresolution saliency map is introduced. This saliency map is provided in a multiresolution format, which naturally follows the principle of the human visual mechanism. Moreover, the points in this map can be utilized to initialize parameters for GrabCut segmentation by labeling the feature pixels automatically. Both the computing speed and segmentation precision are evaluated. The results imply that this multiresolution saliency map-based object segmentation method is simple and efficient.

Sunflicker is a common phenomenon observed in shallow water districts, which severely degrades underwater image quality. However, very few studies have addressed this problem. In this paper, we propose a method to remove sunflicker, which takes the sunflicker distribution amount (SDA) into consideration. The images without sunflicker are approximately recovered by applying principal component analysis theory to a small number of registered images of the same scene. Based on the fact that the difference between the originally registered image and the approximately recovered image includes lost details with high frequencies and sunflicker with low frequencies, a low-pass filter is applied to the difference to obtain pure sunflicker and subsequently reap the image details. The visual inspection, histogram analysis, and comparison approach with motion-compensated filtering are conducted under conditions of different SDAs. Relevant results demonstrate the effectiveness and advantages of the proposed method.

We propose a solution for automatic color correction between two images/videos based on region correspondences. It starts with image segmentation by marker-controlled watershed transformation, which is faster and produces more uniform regions with better adherence to object boundaries than the segmentation in previous color correction approaches. Then regions between two images are matched using point feature correspondences, which are invariant to geometric transformation and illumination change. Finally, the color-distorted image is corrected using the color statistics of corresponding regions and the color transfer functions weighted by influence masks. We demonstrate the experimental results using several data sets and evaluate the color correction by different measures of image similarity.

Video quality measurement is an important component in the end-to-end video delivery chain. Video quality is, however, subjective, and thus, there will always be interobserver differences in the subjective opinion about the visual quality of the same video. Despite this, most existing works on objective quality measurement typically focus only on predicting a single score and evaluate their prediction accuracies based on how close it is to the mean opinion scores (or similar average based ratings). Clearly, such an approach ignores the underlying diversities in the subjective scoring process and, as a result, does not allow further analysis on how reliable the objective prediction is in terms of subjective variability. Consequently, the aim of this paper is to analyze this issue and present a machine-learning based solution to address it. We demonstrate the utility of our ideas by considering the practical scenario of video broadcast transmissions with focus on digital terrestrial television (DTT) and proposing a no-reference objective video quality estimator for such application. We conducted meaningful verification studies on different video content (including video clips recorded from real DTT broadcast transmissions) in order to verify the performance of the proposed solution.

We propose a new adaptive block-wise lossless image compression algorithm, which is based on the so-called alphabet reduction scheme combined with an adaptive arithmetic coding (AC). This new encoding algorithm is particularly efficient for lossless compression of images with sparse and locally sparse histograms. AC is a very efficient technique for lossless data compression and produces a rate that is close to the entropy; however, a compression performance loss occurs when encoding images or blocks with a limited number of active symbols by comparison with the number of symbols in the nominal alphabet, which consists in the amplification of the zero frequency problem. Generally, most methods add one to the frequency count of each symbol from the nominal alphabet, which leads to a statistical model distortion, and therefore reduces the efficiency of the AC. The aim of this work is to overcome this drawback by assigning to each image block the smallest possible set including all the existing symbols called active symbols. This is an alternative of using the nominal alphabet when applying the conventional arithmetic encoders. We show experimentally that the proposed method outperforms several lossless image compression encoders and standards including the conventional arithmetic encoders, JPEG2000, and JPEG-LS.

With the rapid development and popularity of the network, the openness, anonymity, and interactivity of networks have led to the spread and proliferation of pornographic images on the Internet, which have done great harm to adolescents’ physical and mental health. With the establishment of image compression standards, pornographic images are mainly stored with compressed formats. Therefore, how to efficiently filter pornographic images is one of the challenging issues for information security. A pornographic image recognition and filtering method in the compressed domain is proposed by using incremental learning, which includes the following steps: (1) low-resolution (LR) images are first reconstructed from the compressed stream of pornographic images, (2) visual words are created from the LR image to represent the pornographic image, and (3) incremental learning is adopted to continuously adjust the classification rules to recognize the new pornographic image samples after the covering algorithm is utilized to train and recognize the visual words in order to build the initial classification model of pornographic images. The experimental results show that the proposed pornographic image recognition method using incremental learning has a higher recognition rate as well as costing less recognition time in the compressed domain.

A local feature descriptor, the local gradient binary count pattern (LGBCP), is proposed for face recognition. Unlike some current methods that extract features directly from a face image in the spatial domain, LGBCP encodes the local gradient information of the face’s texture in an effective way and provides a more discriminative code than other methods. We compute the gradient information of a face image through convolutions with compass masks. The gradient information is encoded using the local binary count operator. We divide a face into several subregions and extract the distribution of the LGBCP codes from each subregion. Then all the histograms are concatenated into a vector, which is used for face description. For recognition, the chi-square statistic is used to measure the similarity of different feature vectors. Besides directly calculating the similarity of two feature vectors, we provide a weighted matching scheme in which different weights are assigned to different subregions. The nearest-neighborhood classifier is exploited for classification. Experiments are conducted on the FERET, CAS-PEAL, and AR face databases. LGBCP achieves 96.15% on the Fb set of FERET. For CAS-PEAL, LGBCP gets 96.97%, 98.91%, and 90.89% on the aging, distance, and expression sets, respectively.

Research into the removal of mixed noise is a hot topic in the field of image denoising. Currently, weighted encoding with sparse nonlocal regularization represents an excellent mixed noise removal method. To make the fitting function closer to the requirements of a robust estimation technique, an extreme value technique is used that allows the fitting function to satisfy three conditions of robust estimation on a larger interval. Moreover, a random disturbance sequence is integrated into the denoising model to prevent the iterative solving process from falling into local optima. A radon transform-based noise detection algorithm and an adaptive median filter are used to obtain a high-quality initial solution for the iterative procedure of the image denoising model. Experimental results indicate that this improved method efficiently enhances the weighted encoding with a sparse nonlocal regularization model. The proposed method can effectively remove mixed noise from corrupted images, while better preserving the edges and details of the processed image.

To improve the detection performance for perturbed quantization (PQ) class [PQ, energy-adaptive PQ (PQe), and texture-adaptive PQ (PQt)] stego images, a detection method based on possible change modes is proposed. First, by using the relationship between the changeable coefficients used for carrying secret messages and the second quantization steps, the modes having even second quantization steps are identified as possible change modes. Second, by referencing the existing features, the modified features that can accurately capture the embedding changes based on possible change modes are extracted. Next, feature sensitivity analyses based on the modifications performed before and after the embedding are carried out. These analyses show that the modified features are more sensitive to the original features. Experimental results indicate that detection performance of the modified features is better than that of the corresponding original features for three typical feature models [Cartesian calibrated PEVny (ccPEV), Cartesian calibrated co-occurrence matrix features (CF), and JPEG rich model (JRM)], and the integrated feature consisting of enhanced histogram features (EHF) and the modified JRM outperforms two current state-of-the-art feature models, namely, phase aware projection model (PHARM) and Gabor rich model (GRM).

Methods of stitching static panoramas are unsuitable for video-rate stitching from camera arrays, because these methods are too computationally intensive for real-time operation and do not take advantage of prior knowledge of camera positions or the coherence between successive frames of a video sequence. We propose feedback stitching, which embeds the stitching process in a feedback loop, so that as new frames are captured, any new stitching errors occurring in the video sequence are analyzed and corrected as the sequence progresses. These algorithms are suitable for multiscale cameras, a camera array technology proven to be capable of gigapixel snapshot and video imaging, to allow for real-time compensation of any registration or parallax errors.

Reliable object detection is very important in computer vision and robotics applications. The histogram of oriented gradients (HOG) is established as one of the most popular hand-crafted features, which along with support vector machine (SVM) classification provides excellent performance for object recognition. We investigate dimensionality deduction on HOG features in combination with SVM classifiers to obtain efficient feature representation and improved classification performance. In addition to lean HOG features, we explore descriptors resulting from dimensionality reduction on histograms of binary descriptors. We consider three-dimensionality reduction techniques: standard principal component analysis, random projections, a computationally efficient linear mapping that is data independent, and locality preserving projections (LPP), which learns the manifold structure of the data. Our methods focus on the application of eye detection and were tested on an eye database created using the BioID and FERET face databases. Our results indicate that manifold learning is beneficial to classification utilizing HOG features. To demonstrate the broader usefulness of lean HOG features for object class recognition, we evaluated our system’s classification performance on the CalTech-101 dataset with favorable outcomes.

We present a general purpose blind image quality assessment (IQA) method using the statistical independence hidden in the joint distributions of divisive normalization transform (DNT) representations for natural images. The DNT simulates the redundancy reduction process of the human visual system and has good statistical independence for natural undistorted images; meanwhile, this statistical independence changes as the images suffer from distortion. Inspired by this, we investigate the changes in statistical independence between neighboring DNT outputs across the space and scale for distorted images and propose an independence uncertainty index as a blind IQA (BIQA) feature to measure the image changes. The extracted features are then fed into a regression model to predict the image quality. The proposed BIQA metric is called statistical independence (STAIND). We evaluated STAIND on five public databases: LIVE, CSIQ, TID2013, IRCCyN/IVC Art IQA, and intentionally blurred background images. The performances are relatively high for both single- and cross-database experiments. When compared with the state-of-the-art BIQA algorithms, as well as representative full-reference IQA metrics, such as SSIM, STAIND shows fairly good performance in terms of quality prediction accuracy, stability, robustness, and computational costs.

We present a sparse coding based dense feature representation model (a preliminary version of the paper was presented at the SPIE Remote Sensing Conference, Dresden, Germany, 2013) for hyperspectral image (HSI) classification. The proposed method learns a new representation for each pixel in HSI through the following four steps: sub-band construction, dictionary learning, encoding, and feature selection. The new representation usually has a very high dimensionality requiring a large amount of computational resources. We applied the l₁/l_q regularized multiclass logistic regression technique to reduce the size of the new representation. We integrated the method with a linear support vector machine (SVM) and a composite kernels SVM (CKSVM) to discriminate different types of land cover. We evaluated the proposed algorithm on three well-known HSI datasets and compared our method to four recently developed classification methods: SVM, CKSVM, simultaneous orthogonal matching pursuit, and image fusion and recursive filtering. Experimental results show that the proposed method can achieve better overall and average classification accuracies with a much more compact representation leading to more efficient sparse models for HSI classification.

We present a corner-detection method named arc length-based angle estimator (AAE). Different from most of the existing approaches, AAE focuses on employing angle detection for finding corners, because angle is an important measure for discrete curvature. AAE provides a new robust solution to the estimation of the K-cosine. In AAE, the K-cosine estimation issue in the x, y space is considered as the problem of the slope estimations in the s, x and s, y spaces, where s is the arc length. Then, weighted least square fitting is employed to address such a slope estimation issue. Experimental results demonstrate that AAE can achieve promising performance in comparison with some recent state-of-the-art approaches under two commonly used evaluation metrics, namely average repeatability and localization error criteria.

This paper proposes a learning-based superresolution algorithm using text characteristics for text images. The proposed algorithm consists of a learning stage and an inference stage. In the learning stage, a sufficient number of low-resolution (LR) to high-resolution (HR) block pairs are first extracted from various LR–HR image pairs that are composed of texts. Then, we classify those block pairs into 512 clusters and, for each cluster, calculate the optimal two-dimensional (2-D) finite impulse response (FIR) filter to synthesize a high-quality HR block from an LR block and store the block-adaptive 2-D FIR filters in a dictionary with their associated index. In the inference stage, we find the best-matched candidate to each input LR block from the dictionary and synthesize the HR block using the optimal 2-D FIR filter. Finally, an HR image is produced via proper postprocessing. Experimental results show that the proposed algorithm provides superior visual quality to images from previous works and outperforms previous processes in terms of computational complexity.

We propose a new approach for Kronecker compressive sensing of hyperspectral (HS) images, including the imaging mechanism and the corresponding reconstruction method. The proposed mechanism is able to compress the data of all dimensions when sampling, which can be achieved by three fully independent sampling devices. As a result, the mechanism greatly reduces the control points and memory requirement. In addition, we can also select the suitable sparsifying bases and generate the corresponding optimized sensing matrices or change the distribution of sampling ratio for each dimension independently according to different HS images. As the cooperation of the mechanism, we combine the sparsity model and low multilinear-rank model to develop a reconstruction method. Analysis shows that our reconstruction method has a lower computational complexity than the traditional methods based on sparsity model. Simulations verify that the HS images can be reconstructed successfully with very few measurements. In summary, the proposed approach can reduce the complexity and improve the practicability for HS image compressive sensing.

We develop a low-rank approach for image restoration by exploiting the image’s nonlocal self-similarity. We assume that the matrix stacked by the vectors of nonlocal similar patches is of low rank and has sparse singular values. Based on this assumption, we propose a new image deconvolution algorithm that decouples the deblurring and denoising steps. Specifically, in the deblurring step, we involve a regularized inversion of the blur in the Fourier domain, which amplifies and colors the noise and corrupts the image information. Hence, in the denoising step, a singular-value decomposition of similar packed patches is used to efficiently remove the colored noise. Furthermore, we derive an approach to update the estimation of noise variance for setting the threshold parameter at each iteration. Experimental results clearly show that the proposed algorithm outperforms many state-of-the-art deblurring algorithms such as iterative decoupled deblurring BM3D in terms of both improvement in signal-to-noise-ratio and visual perception quality.

Surface fingerprint scanners are limited to a two-dimensional representation of the fingerprint topography, and thus, are vulnerable to fingerprint damage, distortion, and counterfeiting. Optical coherence tomography (OCT) scanners are able to image (in three dimensions) the internal structure of the fingertip skin. Techniques for obtaining the internal fingerprint from OCT scans have since been developed. This research presents an internal fingerprint extraction algorithm designed to extract high-quality internal fingerprints from touchless OCT fingertip scans. Furthermore, it serves as a correlation study between surface and internal fingerprints. Provided the scanned region contains sufficient fingerprint information, correlation to the surface topography is shown to be good (74% have true matches). The cross-correlation of internal fingerprints (96% have true matches) is substantial that internal fingerprints can constitute a fingerprint database. The internal fingerprints’ performance was also compared to the performance of cropped surface counterparts, to eliminate bias owing to information level present, showing that the internal fingerprints’ performance is superior 63.6% of the time.

The H.264/AVC video coding standard is used in a wide range of applications from video conferencing to high-definition television according to its high compression efficiency. This efficiency is mainly acquired from the newly allowed prediction schemes including variable block modes. However, these schemes require a high complexity to select the optimal mode. Consequently, complexity reduction in the H.264/AVC encoder has recently become a very challenging task in the video compression domain, especially when implementing the encoder in real-time applications. Fast mode decision algorithms play an important role in reducing the overall complexity of the encoder. In this paper, we propose an adaptive fast intermode algorithm based on motion activity, temporal stationarity, and spatial homogeneity. This algorithm predicts the motion activity of the current macroblock from its neighboring blocks and identifies temporal stationary regions and spatially homogeneous regions using adaptive threshold values based on content video features. Extensive experimental work has been done in high profile, and results show that the proposed source-coding algorithm effectively reduces the computational complexity by 53.18% on average compared with the reference software encoder, while maintaining the high-coding efficiency of H.264/AVC by incurring only 0.097 dB in total peak signal-to-noise ratio and 0.228% increment on the total bit rate.

Region duplication is a simple and effective operation to create digital image forgeries, where a continuous portion of pixels in an image is copied and pasted to a different location in the same image. Many prior copy–move forgery detection methods suffer from their inability to detect the duplicated region, which is subjected to various geometric transformations. A keypoint-based approach is proposed to detect the copy–move forgery in an image. Our method starts by extracting the keypoints through a fast Hessian detector. Then the adaptive minimal–maximal suppression (AMMS) strategy is developed for distributing the keypoints evenly throughout an image. By using AMMS and a sped-up robust feature descriptor, the proposed method is able to deal with the problem of insufficient keypoints in the almost uniform area. Finally, the geometric transformation performed in cloning is recovered by using the maximum likelihood estimation of the homography. Experimental results show the efficacy of this technique in detecting copy–move forgeries and estimating the geometric transformation parameters. Compared with the state of the art, our approach obtains a higher true positive rate and a lower false positive rate.

Video foreground segmentation lays the foundation for many high-level visual applications. However, how to dig up the effective features for foreground propagation and how to intelligently fuse the different information are still challenging problems. We aim to deal with the above-mentioned problems, and the goal is to accurately propagate the object across the rest of the frames given an initially labeled frame. Our contributions are summarized as follows: (1) we describe the object features with superpixel-based appearance and motion clues from both global and local viewpoints. Furthermore, the objective confidences for both the appearance and motion features are also introduced to balance the different clues. (2) All the features and their confidences are intelligently fused by the improved Dempster–Shafer evidence theory instead of the empirical parameters tuning used in many algorithms. Experimental results on the two well-known SegTrack and SegTrack v2 datasets demonstrate that our algorithm can yield high-quality segmentations.

The objective of large-scale object retrieval systems is to search for images that contain the target object in an image database. Where state-of-the-art approaches rely on global image representations to conduct searches, we consider many boxes per image as candidates to search locally in a picture. In this paper, a feature quantization algorithm called binary quantization is proposed. In binary quantization, a scale-invariant feature transform (SIFT) feature is quantized into a descriptive and discriminative bit-vector, which allows itself to adapt to the classic inverted file structure for box indexing. The inverted file, which stores the bit-vector and box ID where the SIFT feature is located inside, is compact and can be loaded into the main memory for efficient box indexing. We evaluate our approach on available object retrieval datasets. Experimental results demonstrate that the proposed approach is fast and achieves excellent search quality. Therefore, the proposed approach is an improvement over state-of-the-art approaches for object retrieval.

Orientation features of the palmprint have been widely investigated in coding-based palmprint-recognition methods. Conventional orientation-based coding methods usually used discrete filters to extract the orientation feature of palmprint. However, in real operations, the orientations of the filter usually are not consistent with the lines of the palmprint. We thus propose a competitive region orientation-based coding method. Furthermore, an effective weighted balance scheme is proposed to improve the accuracy of the extracted region orientation. Compared with conventional methods, the region orientation of the palmprint extracted using the proposed method can precisely and robustly describe the orientation feature of the palmprint. Extensive experiments on the baseline PolyU and multispectral palmprint databases are performed and the results show that the proposed method achieves a promising performance in comparison to conventional state-of-the-art orientation-based coding methods in both palmprint verification and identification.

Unlike score level fusion, feature level fusion demands all the features extracted from unimodal traits with high distinguishability, as well as homogeneity and compatibility, which is difficult to achieve. Therefore, most multimodal biometric research focuses on score level fusion, whereas few investigate feature level fusion. We propose a face–iris recognition method based on feature level fusion. We build a special two-dimensional-Gabor filter bank to extract local texture features from face and iris images, and then transform them by histogram statistics into an energy-orientation variance histogram feature with lower dimensions and higher distinguishability. Finally, through a fusion-recognition strategy based on principal components analysis and support vector machine (FRSPS), feature level fusion and one-to-n identification are accomplished. The experimental results demonstrate that this method can not only effectively extract face and iris features but also provide higher recognition accuracy. Compared with some state-of-the-art fusion methods, the proposed method has a significant performance advantage.

The moiré effect has been used in high-accuracy positioning and alignment systems for decades. Various methods have been proposed to identify and locate moiré fringes in order to relate the pattern information to dimensional and displacement measurement. These methods can be broadly categorized into manual interpretation based on human knowledge and image processing based on computational algorithms. An artificial neural network (ANN) is proposed to locate moiré fringe centers within circular grating moiré patterns. This ANN approach aims to mimic human decision making by eliminating complex mathematical computations or time-consuming image processing algorithms in moiré fringe recognition. A feed-forward backpropagation ANN architecture was adopted in this work. Parametric studies were performed to optimize the ANN architecture. The finalized ANN approach was able to determine the location of the fringe centers with average deviations of 3.167 pixels out of 200 pixels (≈1.6%) and 6.166 pixels out of 200 pixels (≈3.1%) for real moiré patterns that lie within and outside the training intervals, respectively. In addition, a reduction of 43.4% in the computational time was reported using the ANN approach. Finally, the applicability of the ANN approach for moiré fringe center determination was confirmed.

Gait is a relatively biometric modality which has a precious advantage over other modalities, such as iris and voice, in that it can be easily captured from a distance. Although it has recently become a topic of great interest in biometric research, there has been little investigation into gait spoofing attacks where a person tries to imitate the clothing or walking style of someone else. We recently analyzed for the first time the effects of spoofing attacks on silhouette-based gait biometric systems and showed that it was indeed possible to spoof gait biometric systems by clothing impersonation and the deliberate selection of a target that has a similar build to the attacker. To gain deeper insight into the performance of current gait biometric systems under spoofing attacks, we provide a thorough investigation on how clothing can be used to spoof a target and evaluate the performance of two state-of-the-art recognition methods on a gait spoofing database recorded at the University of Southampton. Furthermore, we describe and evaluate an initial solution coping with gait spoofing attacks. The obtained results are very promising and point out interesting findings which can be used for future investigations.

Concatenating multicamera videos with differing centers of projection into a single panoramic video is a critical technology of many important applications. We propose a real-time video fusion approach to create wide field-of-view video. To provide a fast and accurate video registration method, we propose multistage hashing to find matched feature-point pairs from coarse to fine. In the first stage of multistage hashing, a short compact binary code is learned from all feature points, and then we calculate the Hamming distance between each two points to find the candidate-matched points. In the second stage, a long binary code is obtained by remapping the candidate points for fine matching. To tackle the distortion and scene depth variation of multiview frames in videos, we build hybrid transformation with depth adjustment. The depth compensation between two adjacent frames extends into multiple frames in an iterative model for successive video frames. We conduct several experiments with different dynamic scenes and camera numbers to verify the performance of the proposed real-time video fusion approach.

The newest video compression standard, high-efficiency video coding (HEVC), obtains a higher coding efficiency than previous video compression standards, at the expense of a significant increase of computational complexity. To reduce computational complexity, implementation-friendly encoding algorithms that efficiently trade off coding efficiency and complexity should be designed. We address this issue by proposing a fast prediction unit (PU) partition mode selection scheme from the perspective of spatial correlation of visual signal. Motion similarity of spatially neighboring blocks is first introduced into the PU partition mode selection for HEVC, and the concept of motion homogeneous coding unit (CU) is first proposed. For the motion homogeneous CU, a new method is proposed to reduce the number of its candidate PU partition modes to speed up the PU partition mode selection. Experimental results show that the proposed scheme can achieve lower computational complexity than the five state-of-the-art fast methods while maintaining a similar rate distortion performance.