1.

Introduction

High-spatial-resolution satellites (HSRS) images provide rich detailed information about land covers and have been used widely in different remote sensing applications.¹ However, the shadows in this type of image are obvious and cannot be ignored,² such as building shadows, vegetation shadows, river bank shadows, road shadows, and bridge shadows, consequently, shadow detection has become an indispensable image processing content in remote sensing information extraction.^1,3–5

Shadow appears in two different effects in HSRS images. On the one hand, shadows can be regarded as a kind of useful information in three-dimensional (3-D) reconstruction, building position recognition, and height estimation.^6,7 On the other hand, shadows present adverse influence in information extraction and applications,^8–10 such as reducing the accuracy of change detection and land cover classification. More obvious negative effects appear in cities, where many shadows are distributed and are often confused with dark land covers.

Scholars are devoted to improving the detection accuracy of shadow in remote sensing images^11–15 and have proposed many methods that can be divided into four types:⁴ model-based, physics-based, machine learning, and property-based. In a multispectral HSRS image with only four bands and a spatial resolution between 1 and 5 m, the obvious shadow limits the application of model-based,^16,17 physics-based,^4,18 and machine learning methods^19–24 because these methods need more parameters, auxiliary data, training samples, etc.

Property-based methods were widely used and have the advantage of not requiring in situ or ancillary data, taking into account the shadow properties generally directly deduced from the image data. Some examples of the method are simple histogram thresholding, invariant color model, and object-oriented algorithm. The used features include the image spectrum,^13,25,26 texture,^11,27–30 edge,^31–35 spatial context,^14,36,37 etc. Typical spectral features include the C3 component;³⁸ the difference between intensity and saturation of the hue, saturation, and intensity (HSI) model;³⁹ the normalized index;⁴⁰ the hue and brightness in color spaces such as HSI, hue, saturation, and value (HSV), hue, chroma, and value (HCV), luminance, hue, and saturation (YIQ), and luma, blue-difference chroma, and red-difference chroma (YCbCr);^40,41 the CIELCh color space.⁴² Texture features include the gray level co-occurrence matrix,^27,29 local binary pattern,²⁹ edge,²⁸ and Gabor filter.³⁰ Edge features include the ratio edge³² and weighted edge gradient ratio.³⁵ Spatial context features use the morphological building index and morphological shadow index to first determine the region of the building and shadow and then extract shadow by the distance between the building and shadow.^14,36,37 Shadow information can also be directly segmented by the threshold methods, such as the mean of the two peaks in the histogram⁸ and the Otsu method maximizing the difference between the classes⁴³ or by manual selection.⁴⁴

Most of the property-based methods are pixel-based and often work well. Studies suggest that the image features combined with the direction information and the positional relationship between the shadow and the object can improve shadow detection accuracy.^31,34 However, these methods are scene dependent and have a low degree of automation.

Taking advantage of high spatial resolution to segment the image pixels into “objects” and extract more information can often obtain higher precision.^45,46 This object-oriented method received a lot of attention in the shadow detection of HSRS images, and multiscale segmentation is most often used.^15,47–49 Related tools were provided by the typical remote sensing software such as ENVI,⁵⁰ eCognition,⁵¹ Definiens,⁵² and ERDAS Imagine.⁵³ However, object-oriented image processing often requires human–computer interaction and many tests to determine the relevant parameters and is difficult to convert small shadow patches into independent objects.¹⁵

In order to improve the accuracy and the automation degree of shadow detection in multispectral HSRS imagery, taking the most widely distributed building shadows in the city as the subject, this paper constructs a fast building shadow detection method by coupling the advantages of object-oriented and property-based methods. The new method uses the orthorectification remote sensing image as the input and can mask dark land cover that easily confused with building shadow. Technical innovations of the method include: (1) using multiple image features edge-based image segmentation instead of multiscale segmentation to obtain a more complete object, (2) using remote sensing indices to separate shadow from nonbuilding shadow regions, and (3) integrating the spectrum features with texture, geometric morphological, and context information to further remove the nonbuilding shadow areas.

The content is organized as follows. Section 2 describes the data set. Section 3 describes the proposed method in detail. Section 4 discusses the shadow detection result and accuracy. Section 5 gives the conclusions.

2.

Dataset

Satellite remote sensing images of urban region are often covered by various land covers and the number, length, and brightness of shadows in such images can vary widely. Water bodies, roads, and soils appear dark and their spectra are similar to that of the shadows in an image, increasing the difficulty of shadow detection.

The TripleSat-2 multispectral image on November 27, 2016, in Nanjing, China, was used in this paper. The TripleSat-2 orbital altitude is 651 km, its spatial resolution is 3.2 m, and it includes four bands: blue (B), green (G), red (R), and near-infrared (NIR). Six typical cases in one image were selected and are shown in Fig. 1. The case image size is $400\text{pixels}\times 300\text{pixels}$. The distribution characteristics of the shadows and dark land covers are shown in Table 1 and are used in Sec. 3.3.

Fig. 1

True-color composite images with 2% linear stretch, six typical cases from (a)–(f).

Table 1

Dark land covers in case images.

Among the dark land covers, water body is the first to be easily confused with building shadows, as seen in Figs. 1(a), 1(c), 1(e), and 1(f). Next is asphalt road and dark vegetation, as seen in Figs. 1(a), 1(b), 1(d), and 1(e). The shadows in (a), (c), and (d) are longer and easier to detect, whereas those in (b), (e), and (f) are short and are difficult to detect.

3.

Proposed Method

As the dark area formed by the sun’s rays project onto an opaque object, shadow is divided into a self-shadow (the part of the object that is not illuminated by direct light) and a cast shadow (the shadow projected by the object based on the direction of the light source),⁵⁴ and the latter can be further divided into the umbra and penumbra. The direct light is completely obscured in the umbra, but only a small portion of the direct light is blocked in the penumbra. The penumbra is usually located between the umbra and the nonshadow and may be ambiguous,⁴ resulting in a blurry boundary between the shaded and nonshaded areas.

In satellite remote sensing image, due to the limitations of the quality and spatial resolution, the presence of a cast shadow is obvious, while that of self-shadow is often weak. In a cast shadow, the penumbra accounts for a small portion only⁸ and is difficult to distinguish from the umbra. Therefore, the cast shadow was extracted in this paper.

The object-oriented shadow method consists of four parts: (1) extracting image features that can enhance the shadow information and then segmenting the image based on edge detection to obtain the object; (2) extracting the suspected shadow regions based on the statistical properties of the object; (3) refining the shadow combined the spectrum, texture, and geometric features with spatial context information to remove other nonbuilding shadows; and (4) postprocessing.

The orthorectification image with digital number (DN) is used as the input. The four bands were denoted as B, G, R, and NIR in order of increasing central wavelength.

3.1.

Object Segmentation

3.1.1.

Feature selection and extraction

The selected feature should highlight the shadow information and be positive to the image segmentation to obtain objects with the closed boundary. Previous studies^38,39,55 have shown that (1) the brightness (intensity) of the shadow area is low due to the occlusion of sunlight; (2) the atmospheric Rayleigh scattering effect is sensitive to the short-wavelength blue and violet bands, but insensitive to the NIR band; and (3) the C3 component of the shadow area has a high value and a high saturation. Therefore, six image features were selected from C1C2C3, the HSV color space, and the remote sensing index: C3, ${\mathrm{RATIO}}_{\mathrm{H}\_\mathrm{V}}$, ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$, NIR, the normalized difference vegetation index (NDVI), and the visible atmospherically resistant index (VARI).

• (1) C3

The C3 component of the C1C2C3 color space highlights the difference between shadow and greenish objects (water bodies, green areas, plastic runways, etc.)⁵⁴

Eq. (1)

$$\mathrm{C}3=\arctan \left[\frac{\mathrm{B}}{\max (\mathrm{R},\mathrm{G})}\right].$$

• (2) ${\mathrm{RATIO}}_{\mathrm{H}\_\mathrm{V}}$ and ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$

Shadow appears more clearly in the HSV color space than in the RGB color space. RGB is converted to HSV according the following equations:⁵⁶

Eq. (2)

$$H=\{\begin{array}{ll}\theta & (g\ge b)\\ 2\pi -\theta & (g<b)\end{array},$$

Eq. (3)

$$S=1-\frac{3*\min (r,g,b)}{r+g+b},$$

Eq. (4)

$$V=\frac{r+g+b}{3},$$

where $\theta =\arccos \frac{(r-g)+(r+g)}{2*\sqrt{{(r-g)}^2+(r-b)*(g-b)}}$, and $r$, $g$, and $b$ are the normalized R, G, and B band values, respectively. The values of $H$, $S$, and $V$ are all in the range from 0 to 1.

Based on the relationship between $H$ and $V$ in the HSV color space, Tsai constructed the ${\mathrm{RATIO}}_{\mathrm{H}\_\mathrm{V}}$ index⁴¹

Eq. (5)

$${\mathrm{RATIO}}_{\mathrm{H}\_\mathrm{V}}=\frac{H+1}{V+1}.$$

Based on the relationship between $S$ and $V$, Ma et al.⁵⁷ constructed the normalized saturation-value difference index (NSVDI)

Eq. (6)

$$\mathrm{NSVDI}=\frac{S-V}{S+V}.$$

Referring to the selected $S$ and $V$ in NSVDI, the ratio of $S$ to $V$ can be formed

Eq. (7)

$${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}=\frac{S}{V}.$$

A comparison in Fig. 2 showed that ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$ can highlight the shadow better than NSVDI. In Fig. 2, (a) is a local part of Fig. 1(b), (b)–(e) are images of ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$, NSVDI, and their edges. As seen in Fig. 2, ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$ better highlights the edges of the shadow and dark road.

Fig. 2

Comparison of ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$ and NSVDI: (a) local image with true-color composite, (b) ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$, (c) NSVDI, (d) edge of (b), and (e) edge of (c).

Figure 3 shows the relationship between ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$ and NSVDI in Figs. 2(b) and 2(c). At higher values, the ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$ index rises rapidly and presents a better discrimination.

• (3) NIR, NDVI, and VARI

Fig. 3

Relationship between NSVDI and ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$.

In the NIR band, it is easier to distinguish shadow from nonshadow than in the visible band.⁴ NDVI⁵⁸ can highlight the DN difference between a shadow area and vegetation.⁵⁹ VARI is less sensitive to atmospheric effects⁶⁰ and is used as a supplement to NDVI

Eq. (8)

$$\mathrm{NDVI}=\frac{\mathrm{NIR}-\mathrm{R}}{\mathrm{NIR}+\mathrm{R}},$$

Eq. (9)

$$\mathrm{VARI}=\frac{\mathrm{G}-\mathrm{R}}{\mathrm{G}+\mathrm{R}}.$$

3.1.2.

Image segmentation

The image features C3, ${\mathrm{RATIO}}_{\mathrm{H}\_\mathrm{V}}$, ${\mathrm{RATIO}}_{\mathrm{S}\_\mathrm{V}}$, NIR, NDVI, and VARI were first linearly stretched by 2% to enhance the information, and then their value ranges were changed to the range [0, 255].

The image segmentation procession is shown in Fig. 4. First, the edges of each image feature are detected and then the union set of these edges is taken as the image edge to ensure that the shadow region is not missed. Pixels are connected to get the segmented image if their edges touch.

Fig. 4

Example of image segmentation for shadow detection.

The Canny operator⁶¹ in MATLAB^® 2018b⁶² is used to detect the edges of each image feature. The edge’s union is postprocessed by the skeleton and bridge operator⁶² with a window size of $3\times 3$ to make the edge width one pixel. Finally, a pixel search is made around the outermost image boundary and the current pixel is set as an edge if it touches the edge.

The nonedge areas are the image object.

3.3.

Nonbuilding Shadow Area

A nonbuilding shadow area is shadow caused by objects other than buildings, consisting of dark land cover such as water bodies, buildings, soil, and roads. It is divided into two types: (1) type 1, the spectrum differs from that of building shadow, including dark buildings, dark soil, and dark roads. (2) Type 2, the spectrum is similar to that of the shadow, including dark water bodies and very dark land cover.

Type 1 area was extracted by the spectral features first; then S0, after being masked with type 1, was used to get the type 2 area by the multirule.

3.3.1.

Type 1 area

Rayleigh scattering is inversely proportional to the wavelength. The smallest difference in radiant energy between the shaded and nonshaded area appears in the blue band and the largest in the NIR band.⁶³ In addition, the shaded area has a higher saturation ($S$) and a lower value ($V$) in the HSV color space.³⁹ Based on these conclusions, the shadow and dark object separation index (SDSI) is proposed to expand the difference between the shadow and the nonshadow:

Eq. (10)

$$\mathrm{SDSI}=\alpha \frac{\mathrm{B}}{\mathrm{NIR}}+(1-\alpha )\frac{\mathrm{S}}{\mathrm{V}},$$

where B and NIR are the DNs of band B and band NIR in the original image, $S$ and $V$ are calculated by Eqs. (3) and (4), and the range of values of B/NIR and $S/V$ are all normalized to 0 to 1. $\alpha$ is an environmental parameter indicating the proportion of the shaded area in an image, $\alpha \in [0,1]$.

Confusion between building shadows and dark roads can be reduced by using a different $\alpha$. Taking the original image in Fig. 4 as an example, the shadow with the typical $\alpha$ is shown in Fig. 5. Figure 5(a) ($\alpha =0$) has the best dark removal effect. With an increase of $\alpha$, the effect worsens, and the building shadows on some vegetation are removed when $\alpha \ge 0.8$, as in Figs. 5(f) and 5(g). Section 4.2 further discusses the effect of $\alpha$ on the accuracy of shadow detection.

Fig. 5

Shadow distribution based on the typical $\alpha$ and SDSI. (a) Suspected shadow area S0 of image in Fig. 4, (b) $\alpha =0$, (c) $\alpha =0.2$, (d) $\alpha =0.4$, (e) $\alpha =0.6$, (f) $\alpha =0.8$, and (g) $\alpha =1$.

3.3.2.

Type 2 area

After masking the type 1 area from S0, the type 2 area is extracted by integrating spectral, textural, geometrical, and spatial context information.

In a high-spatial-resolution image, the shadows spectra often have the highest similarity to the spectrum of dark water bodies. According to the object boundary shape, a water body is divided into slender and nonslender.

• (1) Slender water body

A river is an example of a slender water body, as shown in Fig. 1(c). Its shape can be described by the ratio $d$ of the long-axis length $l_1$ and the short-axis length $l_2$ of the object in image

Eq. (11)

$$d=\frac{l_1}{l_2}.$$

For a slender water body, $d>T_d$ and $l_1>T_l$. The threshold $T_d$ is set to 10 and $T_l$ is set to 50 here.

• (2) Nonslender water body

A nonslender water body is shown in Figs. 1(a), 1(e), and 1(f), where (c) is the river, (e) is the pool, and (f) is the lake.

Spectral and textural features are used to extract this kind of water body. Compared with the shadow spectrum, the water body spectrum often appears as a larger value in the G band and a smaller value in the NIR band. Based on this, some remote sensing indices were designed to distinguish water bodies, such as (G-NIR)/(G + NIR) (normalized difference water index, NDWI),⁶⁴ G/NIR,⁶⁴ and G-NIR.

It is also possible to extract a water body by segmenting the G band and NIR band. At first, the image was segmented by the Otsu method⁴³ into the high- and low-value areas, then the intersection of the high-value area of G and the low-value area of NIR can be taken as the water body.

In general, the pixel value in the area of a water body tends to be uniform. This can be described by the sum of the standard deviations (SSD) of the pixel values of each band, where the SSD is low in water bodies and high in shadow areas.

Four methods are used to extract the shadow when a nonslender water body exists. (1) The NDWI low-value area intersects with the SSD high-value area. (2) The G/NIR low-value area intersects with the SSD high-value area. (3) The G-NIR low-value area intersects with the SSD high-value area. (4) The G low-value area is combined with the NIR high-value area, and then intersects with the SSD high-value area. The results are shown in Fig. 6.

Fig. 6

Water body extraction of case images (a), (e), and (f) from top to bottom by the four methods from left to right. In case (a), ponds distributed in the upper left and middle, and rivers in the upper left and right; in case (e), pools distributed in the middle; and in case (f), lake distributed in the lower left.

For shorter shadows in cases (e) and (f), NDWI and G/NIR are more suitable to distinguish shadows with water bodies. For longer shadows in case (a), method 4 is more suitable. More test cases containing water bodies validate this conclusion.

After masking the water body, the shadow accuracy in Fig. 6 is decreased in the order of method 1, method 2, method 3, and method 4. Method 4 may not extract the water body and the other methods may remove some shadows, but can extract the water body more completely.

The appropriate method depended on the image features of a specific image. For example, method 4 is the best for case image (a), whereas method 1 or 2 is the best for case images (e) and (f). As described in Sec. 3.1.1, the ratio G/NIR is more distinguishable than NDWI in the high-value areas. More multispectral test images of TripleSat-2 showed that method 2 or method 4 often achieves better accuracy.

In a city, buildings and water bodies are often surrounded by green plants, trees often grow on both sides of a road, and a building shadow is adjacent to the building. Therefore, the following rules were constructed (S1 denotes the image after removal of the water body):

• (1) Extract vegetation coverage (VC) area. $\mathrm{NDVI}>0$ is regarded as VC. In addition, the blue building and the vegetation appear similar in the NDVI image in some cases, and the obvious blue building area can be eliminated by $B<T_v$, where threshold $T_v$ is a value between the max DN of the vegetation area and the min DN of the blue building area. $T_v$ is not used in our six case images.

• (2) To eliminate edges, the VC is dilated by $5*5$ to obtain ${\mathrm{VC}}_{\text{dilate}}$, and S1 is dilated by $3*3$ to obtain $I_{\text{dilate}}$.

• (3) ${\mathrm{VC}}_{\text{dilate}}$ and $I_{\text{dilate}}$ are combined to obtain VI, which represents the collection of vegetation, shadows, and water bodies.

• (4) Traversing through region S1, most pixels in VI (usually 95% or more) are water and are assigned as 0, otherwise, the pixel is a shadow and assigned as 1.

The image S1 processed by the above steps is expressed as image S2.

4.

Results and Discussion

4.1.

Shadow Detection Accuracy

The shadow detection workflow is divided into four steps: step 1 image segmentation; step 2 extraction of the shadow area; step 3 nonbuilding shadow area removal; and step 4 postprocessing. Step 3 can be further divided into step 3.1, removal of type 1 area, and step 3.2, removal of type 2 area. The necessary steps include step 1, step 2, and step 4; step 3 is optional.

In the images in Fig. 1, cases (a), (c), and (e) were processed by step 1, step 2, step 3, and step 4; cases (b) and (d) by step 1, step 2, step 3.1, and step 4; and case (f) by step 1, step 2, step 3.2, and step 4.

The parameter $\alpha$ of SDSI in step 3 is 0.5, which means that the two components in SDSI are equally important. The images containing water bodies are cases (a), (e), and (f). According to Sec. 3.3.2, the water body removal method in case (a) is method 4, and that of cases (e) and (f) use method 2.

Taking manually identified shadows as the ground truth, precision ($P_s$), recall ($U_s$), and $F1$-score ($F_1$) were used to evaluate the detection accuracy as follows:

Eq. (12)

$$P_s=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},$$

Eq. (13)

$$U_s=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},$$

Eq. (14)

$$F_1=2\frac{P_s{U}_s}{P_s+U_s},$$

where TP denotes the number of pixels correctly detected as shadows, FN denotes the number of pixels in which shadows were detected as nonshadows, and FP denotes the number of pixels in which nonshadows were detected as shadows.

The results of the shadow detection in six case images are shown in Fig. 7 followed by the four steps. The shadow ground truth was obtained by visual interpretation and is shown in Fig. 8.

Fig. 7

Shadow detection procession for the four steps from left to right, for cases (a)–(f) from top to bottom.

Fig. 8

The actual shadows in cases (a)–(f) based on visual interpretation.

In Fig. 7, step 1 shows that the segmented image is consistent with the source image, and step 2 shows that SDSI can better eliminate the interference of other dark land covers. In step 3, all water bodies in the images, except in (a), are better removed. In step 4, the pond in the upper left corner of case (a) is removed by the rule of contextual information, and the river in cases (a) and (e) is removed by the rule of geometric shape.

The shadow detection accuracy is shown in Table 2. The average precision, recall, and $F1$-score were 85.6%, 88.6%, and 87%, respectively, and the ranges were 73.0% to 91.0%, 76.6% to 91.4%, and 74.7% to 91.2%, respectively.

Table 2

Shadow detection accuracy of case image (%).

Case	Precision	Recall	F1-score
(a)	88.5	89.0	88.8
(b)	90.9	91.5	91.2
(c)	91.0	88.4	89.7
(d)	84.4	94.1	89.0
(e)	73.0	76.6	74.7
(f)	85.5	91.7	88.5

According to the $F1$-score and visual comparison, cases (b) and (f) have the best accuracy without obvious errors or omissions. The omissions in cases (a) and (e) mainly appear in the area of light shadows and small shadows. Overextraction of the vegetation shadow appears in cases (a), (c), and (d). Case (c) also extracts the road shadow.

The error of the precision is mainly related to the vegetation shadow. The error of the recall is mainly due to the fact that the penumbra is often blurred on the image and is difficult to identify and extract. Owing to the skeleton and other operations in image segmentation, small shadows of $<3\text{pixels}$ in case images cannot be extracted.

The lowest accuracy appears in (e) where the shadow is short and small and there is a large area of dark soil and a dark water body as interference.

4.2.

Parameter Assignment

The shadow detection of Sec. 4.1, step 3 and step 4, are affected by parameters. Step 3 needs to determine the parameter $\alpha$ of SDSI. Step 4 needs to assign the number of pixels of the largest fragment and the largest hole according to the actual image.

The following is a discussion of SDSI and its parameter $\alpha$.

In remote sensing images, the sensitivity of dark land cover in different color spaces is different. We integrated an RGB color space and HSV color space to deal with this complex situation, and two components, B/NIR and $S/V$, were combined in SDSI. More tests showed that $S/V$ can better highlight the difference between shadows and dark roads in summer images, whereas B/NIR performs better in winter images.

Parameter $\alpha$ indicates the proportion of the shaded area representing the degree of influence of the segmentation of B/NIR and $S/V$ on the result. The larger $\alpha$ is, the greater the influence of B/NIR on the result is, and the smaller the influence of $S/V$ on the result is. The parameter $\alpha$ is changed from 0 to 1 with an increment of 0.1, and its relationship with shadow detection accuracy is shown in Fig. 9.

Fig. 9

Sensitivity of $\alpha$ in SDSI to shadows for cases (a)–(e) after suspected shadow detection.

SDSI can also be used as an index for directly separating shadows from other dark objects. Without the steps in Sec. 3.2, the shadow detection accuracy corresponding to different $\alpha$ is shown in Fig. 10.

Fig. 10

Sensitivity of $\alpha$ in SDSI to shadow for cases (a)–(e) ignoring suspected shadow.

Figure 9 shows that the sensitivity of SDSI varies in different images, and the precision of shadow extraction varies greatly with different $\alpha$. Figure 10 shows that SDSI can also achieve better results in images [e.g., cases (b) and (e)] without objects that are easily confused with shadows (e.g., water body), indicating that SDSI can also be used as an index of shadow detection.

4.3.

Comparison with Results of Multiscale Segmentation

The multiscale segmentation algorithm is one of the most widely used and is included in typical remote sensing software tools. Unlike the top-down image segmentation in this paper, multiscale segmentation is bottom-up.

We use eCognition⁵¹ to handle the multiscale segmentation. The simple yet robust estimation of scale parameter (ESP) tool enables fast and objective parametrization when performing image segmentation,⁶⁵ but for our experimental image with a spatial resolution of 3.2 m, it produced an insufficient segmentation. Thus, we still choose the segmentation parameters manually. After several tests, four multiscale segmentation parameters including scale, band weight, smoothness, and compactness are determined manually according to the visual display of a color image, and they are 40, 1:1:1:4, 0.1, and 0.5, respectively. The shadow distribution based on this segmentation is shown in Fig. 11 and the precision is listed in Table 3.

Fig. 11

Multiscale segmentation of image and shadow detection for cases (a)–(f).

Table 3

Shadow detection accuracy after multiscale segmentation (%).

Case	Precision	Recall	F1-score
(a)	81.3	68.5	74.4
(b)	87.6	71.3	78.6
(c)	86.6	62.7	72.7
(d)	82.1	69.1	75.0
(e)	55.5	61.1	58.2
(f)	76.6	69.0	72.6

The errors and omissions of shadows in Fig. 11 are consistent with that in Sec. 4.1. However, compared with Figs. 7, 8, and 11, multiscale segmentation cannot extract different scales of patches at the same time due to the different scales of various ground features, which easily leads to the omission of shadows and light shadows.

The average precision, recall, and $F1$-score of a shadow based on multiscale segmentation are 78.3%, 67%, and 71.9%, respectively, and their ranges are 55.5% to 87.6%, 61.1% to 71.3%, and 58.2% to 78.6%, respectively. Compared with the results in Table 2, the accuracy of the method is lower and the range is larger.

In high-spatial-resolution image applications, multiscale segmentation often achieves better results. However, for the shadow detection with 3.2-m spatial resolution image in this paper, the performance of multiscale segmentation is poor. The disadvantage of multiscale segmentation is that it requires many manual adjustments, and these adjustments need to take into account the scales of different regions in the image, which requires more experience in practical applications. In our method, the closed and sharp edge is used to determine the boundary of the object, which ensures the integrity of the ground object after segmentation and results in better performance.

5.

Conclusions

A method and workflow of building shadow detection in HSRS remote sensing image based on different image features and edge-based image segmentation were proposed in this paper. Our method is highly automated and has a fast computation speed and the performance of making small-size shadows into independent objects is better than that of multiscale segmentation.

The six case images from the TripleSat-2 multispectral sensor showed that average $F1$-score of shadow detection accuracy was 87%, indicating that the proposed method has a good reference value for shadow detection in a high-spatial-resolution remote sensing image. The extracted shadow can be used as a mask for invalid areas or for uncertainty analysis and can provide effective help for change detection, remote sensing classification, etc.

The adjustable parameter involved in this method is $\alpha$ of SDSI, which needs to be adjusted according to the image. The empirical value is 0.5.

The availability of the method and workflow for other sensor images needs more validation, including how to better combine the advantages of edge segmentation with those of region segmentation to detect shadows with higher accuracy. This needs further research.