Spatial enhancement of low-resolution hyperspectral imagery using high-resolution multispectral imagery is often done via image fusion algorithms. Regardless of the algorithm used, pixels containing edges, corners, shadows, dark/low-contrast materials, etc., present the most challenge to any algorithm. Due to this, confidence on sharpening results are often low at these ’trouble pixels’. This paper presents our initial experiments and results in leveraging spatial information to drive and improve the fusion process. We present an adaptive algorithm workflow that adjusts to the spatial conditions identified for those pixels. We also designed a novel edge detection scheme based on spectral angle calculations on either high- or low-resolution imagery. Target signatures were synthetically implanted on pixels identified as strong edges, and an ACE detector is run on all fused and reference imagery. Our results show that, based on calculated ACE target detection ROC curves, modifying the NNDiffuse algorithm to include factors that leverage spatial features (i.e., spectral differences between neighboring pixels, differences in ‘edgeness’ of neighboring pixels) produced significant improvements in detection rates compared to the classical (non-modified) NNDiffuse algorithm.
The spatial resolution of a hyperspectral image (HSI) can be enhanced by utilizing a co-registered high-resolution multispectral image (MSI) via an image fusion algorithm. This is an active area of research with applications ranging from precision agriculture, to security issues, to mineral identification and mapping, etc. The nearest-neighbor diffusion (NNDiffuse) algorithm has seen tremendous success in being utilized as a pansharpening algorithm to fuse an MSI with a broadband high-resolution panchromatic image and is implemented in commercial software applications, such as ENVI (L3Harris). We extended NNDiffuse to the problem of hyprespectral–multispectral image fusion (HS + MS). Hyperspectral pansharpening is a special case of this: a single high-resolution broadband (panchromatic) image is fused with a low-resolution HSI. Deep-learning (DL)-based methods can achieve excellent results in the area of HS + MS data fusion, but DL algorithms are data hungry: they need extensive training data, require special computing architectures, are slower to implement, etc. NNDiffuse introduces much less spectral distortions compared to state-of-the-art methods, does not need any training data or expensive computing resources, and is significantly faster compared to DL-based approaches. We: (1) demonstrate the utility of NNDiffuse in low-resolution HSI–MSI sharpening; (2) test three deep learning-based image fusion algorithms and compare performance with NNDiffuse and other non-DL-based fusion algorithms using global/image-wide quality metrics; and (3) assess fusion performance using an application based approach: adaptive coherence estimator target detection. The proposed method is tested against several datasets of varying scene content and complexity, and we demonstrate that the NNDiffuse fusion method outperforms the other baseline methods when the application of target detection is considered.
The nearest-neighbor diffusion-based algorithm (NNDiffuse) has seen great success in multispectral pansharpening. Here, we extend the capabilities of NNDiffuse to perform image fusion of high-res multispectral and low-res hyperspectral images (HRMSI+LRHSI fusion). Unlike learning-based frameworks which are computationally expensive and require extensive optimization and/or training time, NNDiffuse is fast, radiometrically accurate (introduces less spectral distortions compared to other state-of-the-art methods), and requires no training data. We introduce the utility of NNDiffuse in hyperspectral-panchromatic and hyperspectral-multispectral sharpening and look at workflows dealing with low-res HSI bands that fall outside the HRMSI spectral response functions. Sharpened image quality is assessed using image-wide metrics. Sharpening performance is also measured on the fused images in terms of their utility in pixel classification and ACE target-background separability. NNDiffuse and DL- and non DL-based methods are presented using hyperspectral imagery from SHARE2012, AVIRIS-NextGen, Hyperspec-VNIR-C, and ROSIS sensors.
Target detection is one of the most important applications utilizing the rich spectral information from hyperspectral imaging systems. Data fusion algorithms applied on hyperspectral datasets address the inherent spatial-spectral resolution tradeoff in these imaging systems by combining spectral information from hyperspectral data with spatial information from hi-res panchromatic or multispectral images (e.g., hi-res RGB). This paper presents the first attempt at using an iterative target implantation technique as a modification to Wald's protocol to assess the performance of data fusion algorithms in target detection tasks. More specifically, this paper looks at how the sharpening process localizes and discriminates the subpixel target from its background, and characterizes an image-wide detectability of any single subpixel target independent of location in the image. We used NNDi use as our pansharpener to perform HRPAN+LRHSI data fusion and the adaptive coherence estimator (ACE) as our target detector. Results show that our methodology is effective at assessing (1) how the sharpening process enhances target-background separability within any 5x5 window anywhere on the image and (2) how the sharpening process enhances the detectability of a single subpixel target over the entire hyperspectral image.
Pan-sharpening - fusing the spatial and spectral information between panchromatic (PAN) and multispectral (MSI) or hyperspectral (HSI) imagery of a common scene is a hot topic in remote sensing due to a wide range of applications such as target detection, vegetation monitoring, and subsurface detection (e.g. landmines), among others. However, the focus of panchromatic sharpening is generally placed on visual quality of the resulting image and image-wide summary spectral accuracy metrics. Here we are interested in radiometrically accurate panchromatic sharpening of hyperspectral imagery with particular emphasis on spectral algorithm performance. Four pansharpening algorithms are applied to hyperspectral imagery and evaluated for spectral/radiometric fidelity. Two datasets from SHARE2012 were used: one which features rural scene elements and one which features an urban scene. Target detection was also performed to evaluate algorithm sharpening performance. We find that although visually the performance of the four algorithms were roughly similar, they differ in spectral/radiometric fidelity as well as performance in ACE target detection.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.