|
1.IntroductionMany imaging sensors, including synthetic aperture radar (SAR), light detection and ranging (LiDAR) sensors, hyperspectral cameras, and wide-area motion imagery (WAMI) sensors, have been developed to obtain target scene images for various applications such as object detection, entity classification, multiple-target tracking, and activity-based intelligence.1 The amount of the raw data obtained with these sensors is large. For example, a hyperspectral camera captures multiple (usually 100+) images of the same target scene with different wavelengths, and the amount of data obtained is usually in the order of several megapixels (an example is in Ref. 2). Another example is WAMI, which generates images over city-sized areas to enable monitoring of vehicle and pedestrian movements.3,4 A typical WAMI image data size is over 144 megapixels (), and the next-generation WAMI image data size will be in the level of 1.6 gigapixels ().5 To transmit the raw data to the users or processing units, either a wideband channel or a long-time interval is needed.6,7 To reduce the required communication bandwidth or the transmission time, the raw data should be compressed. Lossless compression is able to preserve all the information, but has limited reduction power. On the other hand, lossy compression, which may result in very high compression ratio, suffers from interpretability loss as quantified by the National Imagery Interpretability Rating Scale (NIIRS).8,9 NIIRS is a subjective quantification of image interpretability according to the types of tasks a certified image analyst (IA) is able to perform with the imagery for a given rating level. NIIRS has been defined for the following four types of imaging modalities: visible (EO), infrared (IR), radar (SAR), and multispectral.10–13 NIIRS is a 10-level scale with each level defined by a set of information extraction tasks called criteria. Example criteria for EO, IR,14,15 and SAR16 are given in Table 1. The criteria consist of a verb indicating the level of recognition (e.g., distinguish, detect, or identify), the target or object of interest (e.g., building), and some qualifier (e.g., type, size, or feature). Table 1Example EO, IR, RADAR, and multispectral NIIRS level 3 criteria for ground order of battle.
Imagery collection and the selection of the compression impact the performance of image fusion methods.17 When a user interprets an image as per high level information fusion,18 there is a need to understand the context in which the compression level is desired.19 Adaptive context management is desired to determine the correct level of performance desired.20 One way to determine the balance between user needs and the compression level desired is through the performance analysis of image quality.21–23 Through the use of image quality, various image processing methods have been developed for cloud architectures.24,25 An open research question is the alignment of machine-level image interpretability with that of human observers,26,27 although initial comparisons suggest the human perception and machine-level processing are sensitive to different image characteristics.28,29 Many examples to compute the NIIRS have been reported11 and updates are included in the Motion Imagery Standards Board. Recent efforts include the Video-National Imagery Interpretability Rating Scale (VNIIRS)30–33 which can be used for video analysis,34,35 but they still require extensive validation of how to be applied in dynamic imagery collections. The general image quality equations (GIQEs) include GIQE3 (i.e., version 3), GIQE4, and GIQE5. GIEQ3 and GIQE4 were developed for hardcopy images.36,37 GIQE5 focuses on softcopy images. Griffith presented a preliminary version of GIQE5.38 Both hardcopy and softcopy methods are a function of ground sampling distance (GSD), relative edge response (RER), and the signal-to-noise ratio (SNR). In this paper, we present the Compression Degradation Image Function Index (CoDIFI) framework that predicts the NIIRS degradation of an image due to compression. The foundation of this framework is the GIQE, which relates the image quality measure in NIIRS to sensor parameters and acquisition conditions with the goal of objectively predicting the NIIRS rating of images obtained from an imagery collection setting with known sensor parameters and acquisition setting. Specifically, parameters such as GSD, RER, edge overshoot (), noise gain (), and SNR are involved in GIQEs. Based on GIQE, we derive the general image quality degradation equations (GIQDEs) to predict the interpretability loss due to compression. A major feature of GIQDE is that it eliminates the inclusion of GSD, which cannot be inferred easily from the imagery, in its final form when the image quality degradation is a result of data compression. In this paper, a two-stage CoDIFI framework is presented. In the first stage, automated image analytics estimate the RER as well as the edge overshoot of a given image. In the second stage, image analysis is performed on a synthetic binary edge image to build the CoDIFI model that relates NIIRS degradation and ratio of edge gradients before and after compression. Using the CoDIFI model, the compression-induced NIIRS degradation can be inferred by edge gradients obtained before and after compression. The proposed CoDIFI-NIIRS framework can be utilized to predict the NIIRS degradation for a given compression setting, thus, enabling a user to broker the maximum compression setting while maintaining a specified NIIRS rating. This paper is organized as follows. In Sec. 2, two versions of GIQEs are reviewed. Section 3 derives the GIQDE for GIQE version 3. The automated image analytics developed to estimate edge profiles is presented in Sect. 4. In Sec. 5, the CoDIFI model construction is explained. Finally, experiments are presented in Sec. 6, as well as performance validation in Sec. 7 followed by conclusions in Sec. 8. 2.General Image Quality EquationThe NIIRS rating of a given image is obtained from certified IAs, who are usually not available. Many efforts have been made to relate the measure of image quality in terms of NIIRS to sensor parameters, and the results are the GIQEs.37 GIQEs predict NIIRS as a function of the imaging sensor and the acquisition setting of relevant parameters: GSD, RER, SNR, noise gain (), and edge overshoot height (). The parameters RER, , and are defined after image enhancements are performed. The GIQE for an EO sensor is given as37 and where is the geometric mean of ground sample distance in inches, is the geometric mean of the normalized RER, is the geometric mean of edge overshoot due to modulation transfer function compensation (MTFC)/enhancement, is the noise gain due to MTFC/enhancement, SNR is the signal-to-noise ratio, and if ; 3.16 if , if ; 2.817 if .GIQE3 was released in December 1994 to the unmanned aerial vehicle/sensors community whereas GIQE4 was published in November 1997 for the development of the commercial space imaging industry.37 Both GIQEs were empirically determined using linear regression technique assuming hardcopy viewing. One major difference between GIQE3 and GIQE4 lies in the definition of GSD. In GIQE3, GSD is defined in the plane orthogonal to the line of sight while, in GIQE4, GSD is defined in the ground plane. The derivation and validation of GIQE4 was based on a set of 359 images whose characteristics are listed in Table 2.38 Note that GIQE4 may not be accurate for an image whose characteristic is outside of the listed range. GSD is the actual ground distance in inches between two adjacent pixels. The GSD value is usually included in or has to be calculated from image metadata and cannot be obtained from simple image analysis. Table 2Characteristic range used to derive and validate GIQE4.
RER mainly affects the contrast of an image. It is estimated using the Stennis Space Center specified edge target as shown in Fig. 1(a) and its tilted version.39 An RER value is obtained by estimating the slopes of edge profiles within the image as shown in Fig. 1(b). In principle, RER estimates effective slope of the imaging system’s edge response. Fig. 1(a) Stennis Space Center specified edge target for RER estimation. (b) Normalized edge response for RER estimation. ![]() The edge overshoot height () of normalized edge response is the result of the application of MTFC, whose aim is to increase the image contrast but inevitably results in edge overshoot/edge ringing artifacts. Figure 2 shows the overshoot phenomenon, which is reproduced from Ref. 40. Figure 2(b) is obtained by applying small low-fidelity image-sharpening kernels on the image shown in Fig. 2(a). Edge-ringing artifacts are clearly observed in Fig. 2(b). The edge overshoot due to image sharpening is given in Fig. 2(c). Fig. 2An illustration of edge overshoot height due to image-sharpening. (a) Image before sharpening, (b) image after sharpening, and (c) edge overshoot height. ![]() The application of MTFC inevitably amplifies noise. Noise gain , due to the application of MTF, can be calculated from the coefficients, , of MTFC kernels for a pixel For example, with the following MTFC, which is a symmetric sharpening kernel, the value can be computed to be 3.51. Without the knowledge about the actual MTFC kernel, cannot be obtained. In Ref. 41, a fixed value of 4.16 was used for both Quickbird and IKONOS images. Fortunately, in the GIQDEs to be introduced next, the parameter is no longer involved when the change of SNR due to compression is small. Finally, SNR is defined as the ratio between the power of image signal with the DC component excluded and the power of noise signal. For a given noise free image , noise image , and noise-corrupted image , the SNR is computed as where and are the height and width of the image and is the pixel location.3.Image Quality Degradation EquationIn an analysis of the general image quality equation,42 it is concluded that GIQE3 image quality predictions are more accurate than those from GIQE4 in a certain scenario. In addition, GIQE4 introduces discontinuity when RER is equal to 0.9. For this reason, we adopt GIQE3 instead of GIQE4 for the discussion in this section. However, similar discussion can be made using GIQE4. Denote the GIQE estimated NIIRS for an image data before and after compression as andNote that, in both cases, the parameters GSD and are not changed as they are sensor setting related parameters and are not affected by compression. With Eqs. (5) and (6), the change of NIIRS due to compression can be derived as By expressing the SNR after compression as its Taylor series at , then where is assumed to be much less than .We call Eq. (8) the GIDQE, which predicts the interpretability loss due to compression. From Eq. (8), it is observed that the parameters GSD, , and SNR involved in GIQE are no longer required to predict the interpretability loss due to compression. 4.Image Analytics for Edge Profile EstimationRER and edge overshoot height () are defined through edge profiles as can be seen in Figs. 1(b) and 2(c). This section presents an image analytic approach that performs edge profile extraction from which RER and can be estimated. Figure 3 shows the edge profile extraction workflow, which involves a number of modules such as Canny edge detector, Hough transform, and edge stripes determination. Fig. 3Edge profile extraction workflow along with a description of the output of each module and the corresponding sample output in Fig. 4. ![]() The first two modules, Canny edge detector and Hough transform, are employed to extract line edges from the input image. For each extracted line edge, the edge stripes determination module extracts the corresponding edge stripe. A sample extracted edge stripe is provided in Fig. 4(c). Next, the edge intensity determination module is used to determine the intensity value that defines the edge point in each column of the edge stripes. Before an edge point can be decided, it is necessary to define the maximum and minimum intensity values, which will be normalized to one and zero, respectively. To this end, considering the possibility of having edge overshoot, the maximum intensity for an edge is taken as the median of the intensities of the first two rows of the bright side of each edge stripe. Rather than the minimum value, the minimum intensity is taken as the fifth percentile of the edge stripe to eliminate the possible outliers in each edge stripe. Fig. 4Illustration of the proposed edge profile extraction workflow. (a)–(g) are the outputs at various stages of the workflow depicted in Fig. 3. ![]() Once the maximum and minimum intensity values are defined for each edge stripe, their mean value is taken as the intensity value that defines an edge point. After the edge intensity of each edge stripe is obtained, the edge center of each edge profile can be assessed. The edge profile is determined by searching for the location of each edge profile whose intensity is within a -neighborhood of the edge intensity determined in the previous step. Here, is a predefined small value that determines the precision of the edge center, which is set it to be 0.001 in this work. Once the edge center is found for each edge profile, it is resampled at an array of positions centered at each edge center. Figures 4(d) and 4(e) show the edge profiles of an edge stripe before and after the edge profiles alignment module. To compute RER value, edge profiles have to be normalized to from 0 to 1 as shown in Fig. 1(b). In the module Edge intensity determination, the maximum and minimum intensity values within an edge stripe have been decided. Denote them as and . Edge profiles normalization is performed as , where is the normalized intensity and is the original intensity of a pixel of the raw edge profile, and and are the maximum and minimum intensity values determined for each edge stripe. Figure 4(f) shows the edge profiles normalized from those shown in Fig. 4(e). The last module, edge profiles finalization, produces a single edge profile for each edge stripe along with a quality measure. The final single edge profile is obtained by averaging the raw edge profiles of each edge stripe. However, not all averaged edge profiles of each edge stripe are equally reliable. The edge profiles module produces, in addition to the mean edge profile, a quality measure based on the variance of the raw edge profiles defined as where is the number of raw edge profiles in the edge stripe in consideration, is the ’th raw edge profile, and is the mean edge profile. Figure 4(g) shows a sample mean edge profile. To select the one that best represents the given image for RER computation, the process begins by first selecting a set of mean edge profiles whose variances are within the least 10% of the mean edge profiles available in the given image. Then the mean RER value is used as the RER value of the image.After obtaining the edge profile, it is straightforward to determine the RER by taking the difference of the edge profile values at location and as shown in Fig. 1(b). To estimate edge overshoot height, we follow the approach described in Ref. 43. First, obtain normalized edge profile from to 3 pixels from edge center. Then, the maximum value between and from the center is taken as the edge overshoot if it is greater than 1. Otherwise, the value at from the edge center is used as value. This approach is graphically shown in Fig. 5. There are two cases in Fig. 5. Case 1 indicates undershoot in the edge profile. In this case, the value at position 1.25 pixel is adopted as value, which is about 0.8. In case 2, the maximum value between 1 and 3 occurs at position 1.75 with value 1.2, which is greater than 1. Therefore, the value in case 2 is determined to be 1.2. 5.Compression Degradation Image Function Index ModelAs will be seen in the experimental results presented in Sec. 6, the estimated NIIRS degradation is not a smooth function of compression ratio by directly plugging the parameters estimated in Sec. 4 in Eq. (8). This is not surprising as the image analytics presented in Sec. 4 is able to reliably extract edge profiles only in the simplest case, for example, a binary edge image as displayed in Fig. 6. Therefore, for images that lack clear, strong, straight line edges, the proposed edge profile extraction approach is doomed to failure. For this reason, an alternative approach is desired for practical application. From the derived GIQDE given by Eq. (8) and the definitions of RER and as shown in Figs. 1 and 2, it seems reasonable to conclude that NIIRS degradation is directly related to the gradients calculated at edges before and after compression. For this reason, we assume that NIIRS degradation can be modeled as a function of the ratio of edge gradients obtained before and after compression. That is, where is the edges in the image before compression and is the edges in the compressed image, is the gradient operator, and is the model to be estimated. We name the model as the CoDIFI in this work.44Once this model is available, reduction of NIIRS rating can be predicted simply by the ratio of gradients obtained before and after compression at the same edge points. In this work, a simple neural network (NN) (shown in Fig. 7) is employed to obtain the model. Fig. 7Structure of the NN employed to model the relationship between the gradient ratio and NIIRS degradation. ![]() The training data are obtained by applying the image analytics presented in Sec. 4 to a series of sequentially degraded images of the image given in Fig. 6. The degraded images are generated by sequentially blurring the simulated edge image shown in Fig. 6 with fixed sized () Gaussian low-pass filters with different standard deviation values ranging from 0.2 to 3. Figure 8 shows some of the resulting degraded images. After the generation of training data, a simple NN with one hidden unit was trained to model the relationship between the gradient ratio and NIIRS degradation. Figure 9 shows the resulting CoDIFI where the blue asterisk symbol indicates the training data. 6.Experimental Comparisons of Estimated NIIRS LossIn the experiments, two approaches are used to estimate NIIRS degradation due to compression. The first approach directly applies Eq. (8) with parameters estimated from the automated image analytics presented in Sec. 4. The second approach applies the CoDIFI model constructed in Sec. 5 to estimate NIIRS degradation due to compression. A more detailed description follows. Approach 1: Estimation of RER and .
Approach 2: gradient ratio at edge points
Ten urban and 10 rural images as shown in Figs. 10 and 11 are used in the experiment. Urban images are characterized by more distinctive edges than rural images; thus, the edge profile-based approach is expected to work more reasonably in urban images. For compression schemes, JPEG and JPEG2000 are adopted and experimental results are shown in Figs. 12Fig. 13Fig. 14–15. Fig. 10Ten urban images used in the experiment. (a)Urban image 1, (b) urban image 2, (c) urban image 3, (d) urban image 4, (e) urban image 5, (f) urban image 6, (g) urban image 7, (h) urban image 8, (i) urban image 9, (j) urban image 10. ![]() Fig. 11Ten rural images used in the experiment. (a) Rural image 1, (b) rural image 2, (c) rural image 3, (d) rural image 4, (e) rural image 5, (f) rural image 6, (g) rural image 7, (h) rural image 8, (i) rural image 9, and (j) rural image 10 ![]() Figure 12 shows the results for the JPEG compression and Fig. 13 for JPEG2000 using the urban images. In Figs. 12–15, the NIIRS degradation is plotted as a function of compression ratio and is estimated by the edge profile-based (EPB) approach and the CoDIFI model. From these two approaches over two scenarios, the following observations are presented to provide a high-level assessment of comparisons.
From these two approaches, the following observations resulted.
The first two observations reveal the limit of the EPB approach. That is, EPB performs well only when the line edges can be reasonably extracted. Since many line edges can be observed in urban images, the results from urban images are more reasonable than those from rural images, where line edges are rarely present. They also imply the validness of the CoDIFI-based approach because it produces similar but much smoother curves. The third observation indicates that JPEG compression may outperform JPEG2000 in terms of interpretability loss at very low compression rate. The roughness of the curves resulting from applying JPEG compression on rural images may due to the blocky artifact of JPEG compression as well as the lack of line edges in rural images. 7.Validation of the CoDIFI MethodExperimentation using an independent NIIRS-assessment method provides an empirical validation of CoDIFI. Previous research has demonstrated the value of NIIRS-based methods for assessing compression of imagery9 and video data.45,8,46 Additional investigations have shown relationships between loss in image interpretability and objective image metrics.35 The approach is to compare the NIIRS loss as rated by expert human observers to predict NIIRS loss reported by CoDIFI. The imagery data used for validation are new images that were not used in the development of CoDIFI. A set of still frames were extracted from Air Force Research Laboratory’s VIVID data set available from the Sensor Data Management System.47 The VIVID data are described in Ref. 48 with a video analysis in Ref. 49. The validation set consisted of 30 images extracted from the VIVID data set (Fig. 15). Images ranged from NIIRS 4 to almost NIIRS 7 and included a range of scene content, backgrounds, and viewing geometries. For a given compression method, each of these “parent” images were compressed at various compression ratios to produce multiple compression products. Ratings by four expert human observers quantified the delta NIIRS between the parent image and each compressed image. Likewise, CoDIFI generates a predicted NIIRS loss for each compressed image. The comparison of these two ratings for the images in Fig. 16 is the basis for the validation analysis. Analysis using H.264 compression will demonstrate the validation process. The approach is iterative in which validation analysis supports improvements in CoDIFI and subsequent analysis confirms the improvement in performance. The relationship between the CoDIFI-predicted NIIRS loss and the expert observer delta-NIIRS ratings demonstrates that the two values align well. A slope of 1 and an intercept of 0 would indicate perfect alignment between the CoDIFI predictions and the expert ratings. The regression analysis yields a slope of 0.972, which is not statistically different from 1 (). The intercept value of 0.15, which is statistically different from zero, is small in practical terms. Visual inspection of the data confirms the strong agreement shown by the regression model (Fig. 17) (Table 3). Table 3Regression analysis (R2=0.94).
8.ConclusionIn this paper, we presented the CoDIFI-NIIRS framework that can be employed to predict the compression-induced image quality loss in terms of NIIRS. The CoDIFI model is built on an automated image analytic that estimates line edge profiles on simulated edge images. The EPB approach in turn estimates the NIIRS degradation based on the derived GIDQE. Though our CoDIFI framework produces reasonable results, it is not fully validated. Our validation analysis demonstrates that CoDIFI predictions of NIIRS loss align well with expert ratings from human observers. Future efforts include assessing the timeliness of the methods for automated systems, extensions with video sequences subject to motion artifacts, blur,50 and resolution changes, and applicability to image fusion.51 AcknowledgmentsThis research was sponsored by the Dynamic-Data Driven Application System AFOSR grant and US Air Force Research Lab under Contract FA8750-16-C-0246. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of Air Force Research Laboratory, or the U.S. Government. ReferencesB. Kahler and E. Blasch,
“Predicted radar/optical feature fusion gains for target identification,”
in Proc. of the IEEE National Aerospace Electronics Conf. (NAECON),
(2010). Google Scholar
K. Koonsanit, C. Jaruskulchai and A. Eiumnoh,
“Band selection for dimension reduction in hyper spectral image using integrated information gain and principal components analysis technique,”
Int. J. Mach. Learn. Comput., 2
(3), 248
–251
(2012). https://doi.org/10.7763/IJMLC.2012.V2.124 Google Scholar
X. Shi et al.,
“Context-driven moving vehicle detection in wide area motion imagery,”
in Int. Conf. on Pattern Recognition (ICPR),
(2012). Google Scholar
E. Blasch et al.,
“Summary of methods in wide-area motion imagery (WAMI),”
Proc. SPIE, 9089 90890C
(2014). https://doi.org/10.1117/12.2052894 PSISDG 0277-786X Google Scholar
A Primer for Dissemination Services for Wide Area Motion Imagery, http://www.opengis.net/doc/bp/wami-primer/1.0 Google Scholar
Z. Tian et al.,
“Performance evaluation of distributed compressed wideband sensing for cognitive radio networks,”
in Int. Conf. on Info Fusion,
(2008). Google Scholar
D. Shen et al.,
“Network survivability oriented Markov games (NSOMG) in wideband satellite communications,”
(2014). https://doi.org/10.1109/DASC.2014.6979500 Google Scholar
D. Young et al.,
“Loss of interpretability due to compression effects as measured by the new video NIIRS,”
Proc. SPIE, 7529 75290V
(2010). https://doi.org/10.1117/12.838644 PSISDG 0277-786X Google Scholar
J. M. Irvine et al.,
“Evaluation of the tactical utility of compressed imagery,”
Opt. Eng., 41
(6), 1262
–1272
(2002). https://doi.org/10.1117/1.1475740 Google Scholar
J. Kerekes, A. Cisz and R. Simmons,
“A comparative evaluation of spectral quality metrics for hyperspectral imagery,”
Proc. SPIE, 5806
(2005). https://doi.org/10.1117/12.605916 PSISDG 0277-786X Google Scholar
J. M. Irvine,
“National imagery intelligence rating scale (NIIRS),”
The Encyclopedia of Optical Engineering, Marcel Dekker, New York
(2003). Google Scholar
J. C. Leachtenauer and R. G. Driggers, Surveillance and Reconnaissance Systems: Modeling and Performance Prediction, Artech House, Norwood, Massachusetts
(2001). Google Scholar
L. A. Maver, C. D. Erdman and K. Riehl,
“Imagery interpretability rating scales,”
in Digest of Technical Papers, Int. Symp. Society for Information Display,
117
–120
(1995). Google Scholar
J. C. Leachtenauer et al.,
“General image-quality equation for infrared imagery,”
Appl. Opt., 39
(26), 4826
–4828
(2000). https://doi.org/10.1364/AO.39.004826 APOPAI 0003-6935 Google Scholar
J. Kerekes and S. Hsu,
“Spectral quality metrics for VNIR and SWIR hyperspectral imagery,”
Proc. SPIE, 5425
(2004). https://doi.org/10.1117/12.542192 PSISDG 0277-786X Google Scholar
R. Driggers et al.,
“Synthetic aperture radar target acquisition model based on a National Imagery Interpretability Rating Scale to probability of discrimination conversion,”
Opt. Eng., 42
(7),
(2003). https://doi.org/10.1117/1.1580831 Google Scholar
Z. Liu et al.,
“Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative survey,”
IEEE Trans. Pattern Anal. Mach. Intell., 34
(1), 94
–109
(2012). https://doi.org/10.1109/TPAMI.2011.109 ITPIDJ 0162-8828 Google Scholar
E. Blasch, E. Bosse and D. A. Lambert, High-Level Information Fusion Management and Systems Design, Artech House, Norwood, Massachusetts
(2012). Google Scholar
L. Snidaro et al., Context-Enhanced Information Fusion: Boosting Real-World Performance with Domain Knowledge, AG Switzerland, Switzerland
(2016). Google Scholar
A. N. Steinberg et al.,
“Adaptive context assessment and context management,”
in Int. Conf. on Information Fusion,
(2014). Google Scholar
E. Blasch et al.,
“Image quality assessment for performance evaluation of image fusion,”
in Int. Conf. on Information Fusion,
(2008). Google Scholar
Y. Chen et al.,
“Image quality measures for predicting automatic target recognition performance,”
in IEEE Aerospace Conf.,
(2008). https://doi.org/10.1109/AERO.2008.4526432 Google Scholar
Y. Zheng et al.,
“Image quality (IQ) guided multispectral image compression,”
Proc. SPIE, 9871 98710C
(2016). https://doi.org/10.1117/12.2225532 PSISDG 0277-786X Google Scholar
K. Liu et al.,
“A cloud infrastructure for target detection and tracking using audio and video fusion,”
in IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW),
(2015). https://doi.org/10.1109/CVPRW.2015.7301299 Google Scholar
B. Liu et al.,
“Information fusion in a cloud computing era: a systems-level perspective,”
IEEE Aerosp. Electron. Syst. Mag., 29
(10), 16
–24
(2014). https://doi.org/10.1109/MAES.2014.130115 IESMEA 0885-8985 Google Scholar
E. Blasch, R. Breton and P. Valin,
“Information fusion measures of effectiveness (MOE) for decision support,”
Proc. SPIE, 8050
(2011). https://doi.org/10.1117/12.883988 PSISDG 0277-786X Google Scholar
E. Blasch, P. Valin and E. Bossé,
“Measures of effectiveness for high-level fusion,”
in Int. Conf. on Information Fusion,
(2010). https://doi.org/10.1109/ICIF.2010.5711858 Google Scholar
D. Gutchess et al.,
“Predicting the effectiveness of SAR imagery for target detection,”
Proc. SPIE, 8051 805110
(2011). https://doi.org/10.1117/12.882712 Google Scholar
J. M. Irvine and E. Nelson,
“Image quality and performance modeling for automated target detection,”
Proc. SPIE, 7335 73350L
(2009). https://doi.org/10.1117/12.818593 PSISDG 0277-786X Google Scholar
“Video-National Imagery Interpretability Rating Scale,”
MISB standard, http://www.gwg.nga.mil/misb/docs/standards/ST0901.2.pdf Google Scholar
J. M. Irvine et al.,
“Developing an interpretability scale for motion imagery,”
Opt. Eng., 46
(11), 117401
(2007). https://doi.org/10.1117/1.2801504 Google Scholar
J. M. Irvine, D. M. Cannon and S. A. Israel,
“Quantifying interpretability for motion imagery: applications to image chain analysis,”
in Int. Conf. on Information Fusion,
(2007). Google Scholar
D. Young et al.,
“Video National Imagery Interpretability Rating Scale criteria survey results,”
Proc. SPIE, 7307 73070G
(2009). https://doi.org/10.1117/12.816546 PSISDG 0277-786X Google Scholar
R. I. Hammoud et al.,
“Automatic association of chats and video tracks for activity learning and recognition in aerial video surveillance,”
Sensors, 14 19843
–19860
(2014). https://doi.org/10.3390/s141019843 SNSRES 0746-9462 Google Scholar
E. Blasch and B. Kahler,
“Application of VNIIRS for target tracking,”
Proc. SPIE, 9473 947306
(2015). https://doi.org/10.1117/12.2177543 PSISDG 0277-786X Google Scholar
J. C. Leachtenauer,
“National Imagery Interpretability Rating Scales: overview and product description,”
in Proc. of American Society of Photogrammetry and Remote Sensing Annual Meetings,
(1996). Google Scholar
J. C. Leachtenauer et al.,
“General image-quality equation: GIQE,”
Appl. Opt., 36 8322
–8328
(1997). https://doi.org/10.1364/AO.36.008322 APOPAI 0003-6935 Google Scholar
D. Griffith,
“General imagery quality equations,”
(2012) https://calval.cr.usgs.gov/wordpress/wp-content/uploads/Griffith_Doug_JACIEGIQERev3cor2sjs6with-Caveat.pdf Google Scholar
S. Blonski et al.,
“Spatial resolution characterization for aerial digital imagery,”
http://calval.cr.usgs.gov/JACIE_files/JACIE06/Files/39Blonsk.pdf Google Scholar
S. T. Thurman and J. R. Fienup,
“Application of the general image-quality equation to aberrated imagery,”
Appl. Opt., 49
(11), 2132
–2142
(2010). https://doi.org/10.1364/AO.49.002132 APOPAI 0003-6935 Google Scholar
T. Kim et al.,
“Automated image interpretability assessment by edge profile analysis of natural targets,”
in Opportunities for Emerging Geospatial Technologies, ASPRS,
(2010). Google Scholar
S. T. Thurman and J. R. Fienup,
“Analysis of the general image quality equation,”
Proc. SPIE, 6978 69780F
(2008). https://doi.org/10.1117/12.777718 PSISDG 0277-786X Google Scholar
T Kim, J.-I. Kim and D. Kim,
“Image-based estimation and validation of NIIRS for high-resolution satellite images,”
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.,
(B1), 1
–4
(2008). 1682-1750 Google Scholar
H.-M. Chen et al.,
“An investigation of image compression on NIIRS rating degradation through automated image analysis,”
Proc. SPIE, 9838 983811
(2016). https://doi.org/10.1117/12.2224631 PSISDG 0277-786X Google Scholar
L. Gibson et al.,
“User evaluation of differential compression for motion imagery,”
Proc. SPIE, 6246 624605
(2006). https://doi.org/10.1117/12.665200 PSISDG 0277-786X Google Scholar
G. O’Brien et al.,
“Metrics to estimate image quality in compressed video sequences,”
Proc. SPIE, 6546 65460A
(2007). https://doi.org/10.1117/12.718718 PSISDG 0277-786X Google Scholar
E. Blasch,
“AFRL VIVID Data Set,”
https://www.sdms.afrl.af.mil/index.php?collection=video_sample_set_1 Google Scholar
K. Erickson et al.,
“Evaluation of the VIVID confirmatory identification module,”
Proc. SPIE, 6566 65660B
(2007). https://doi.org/10.1117/12.719658 PSISDG 0277-786X Google Scholar
H. Ling et al.,
“Robust infrared vehicle tracking across target pose change using regularization,”
in Int. Conf. on Information Fusion,
(2010). Google Scholar
J. H. Cha et al.,
“On use of image quality metrics for perceptual blur modeling: image/video compression case,”
Opt. Eng., 57
(2), 023109
(2018). https://doi.org/10.1117/1.OE.57.2.023109 Google Scholar
Y. Zheng, E. Blasch and Z. Liu, Multispectral Image Fusion and Colorization, SPIE Press, Bellingham, Washington
(2018). Google Scholar
BiographyErik Blasch received his BS from MIT, multiple MS degrees, and PhD from Wright State University. He is a program officer at the Air Force Office of Scientific Research. He has compiled 700+ papers, 19 patents, and 4 SPIE tutorials in ATR, target tracking, and informaion fusion. His books include High-Level Information Fusion Management and Systems Design (Artech House, 2012), Context-Enhanced Information Fusion (Springer, 2016), and Multispectral Image Fusion and Colorization (SPIE, 2018). He is a member of ISIF, IEEE fellow, AIAA associate fellow, and SPIE fellow. Hua-mei Chen currently is a principal scientist at Intelligent Fusion Technology (IFT) Inc., Germantown, Maryland. He has 15+ years of experience in digital image processing. Prior to joining IFT, he was a senior research engineer at Signal Processing Inc., Rockville, Maryland, and an assistant professor in the Department of Computer Science and Engineering, the University of Texas at Arlington, where he focused on general 2-D/3-D rigid/nonrigid image registration problems. He received his PhD from Syracuse University, New York, in 2002. John M. Irvine is the chief scientist for Data Analytics at Charles Stark Draper Laboratory, Inc. (Draper). With over 40 years of experience, his research interests include signal and image processing, information fusion, image quality, data analytics, and generative data models. He has been the PI for multiple programs for DARPA, Air Force Research Laboratory, Night Vision Laboratory, and other sponsors. He serves on several government advisory panels, including the Defense Science Board. Prior to joining Draper, he was a technical fellow and deputy division manager at SAIC, and a senior scientist at the Environmental Research Institute of Michigan (ERIM). He has authored over two hundred journal and conference papers and holds a PhD in mathematical statistics from Yale University. Zhonghai Wang received his BS degree from Tianjin University, Tianjin, China, 1998, and PhD from Michigan Technological University, Michigan, USA, 2010. He has been with Intelligent Fusion Technology, Inc. (IFT), since March 2012, and currently is a senior project manager. Prior to IFT, he worked as a postdoctoral research scientist with the Electrical and Computer Engineering Department at Missouri University of Science and Technology, Rolla, Missouri, from March 2011 to February 2012. He was an engineer at China Aerospace Science & Industry Corporation from August 1998 to August 2005. His research interests include wireless localization, radar system, array signal processing and statistical signal processing. Genshe Chen received his BS, MS, and PhD degrees all from Northwestern Polytechnical University, Xian, China. He is the chief technological officer of Intelligent Fusion Technology, Inc., Germantown, Maryland, where he directs the research and development activities for the government services and commercial solutions. He is a senior member of SPIE. James Nagy received his BS from Western New England University and a MS degree in electrical engineering from National Technical University. He is a program manager at the Air Force Research Laboratory, Information Directorate located in Rome, New York. He has led or been on teams perfroming development of new and novel techniques for natural language processing, information extraction, latent relationship detection and knowledge discovery from multiple data sources using hard/soft fusion techniques. He has worked with numerous agencies, including DARPA, IARPA, NGA, and DTRA. He has contributed to numerous technical papers associated with these topics. Stephen Scott is an electronics engineer for the Air Force Research Laboratory, Rome, New York. His current focus is on test and analysis of advanced information fusion tools for warfighter systems, such as the Distributed Common Ground Station (DCGS). He received his BSEE from Carnegie Mellon University in 1987 and MSEE from Syracuse University in 1994. He is a member of the Institute of Electrical and Electronics Engineers (IEEE) and IEEE Signal Processing Society. His research interests include ground target tracking, maritime tracking, multitarget multisensor data fusion, data mining, and ground moving target exploitation and forensic analysis. |