Open Access
28 May 2022 Altitude-dependent probability of vertical, cloud-free line-of-sight from European Centre for Medium Range Weather Forecasting Re-Analysis-interim cloud cover
Author Affiliations +
Abstract

A method to calculate the altitude-dependent, vertical, cloud-free line-of-sight (CFLOS) using the fractional cloud cover from the European Centre for Medium Range Weather Forecasting Re-Analysis Interim (ERA-I) dataset has been developed. This method enables users of airborne and satellite collections of optical ground data to understand the statistical coverage limitations of these collection systems by informing them of global probabilities of CFLOS versus altitude as well as time of year. This method is accurate for regions between ±60  deg of latitude; it should not be applied to polar regions due to limitations in the underlying ERA-I data. Our CFLOS calculations have been compared to the results of CloudSAT/Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation analysis and with Moderate Resolution Imaging Spectroradiometer (MODIS) total cloud cover data. It is shown that the ERA-I reports on average less cloud cover by about 7.5% (absolute) for regions within   ±  60 deg of the equator relative to MODIS cloud cover retrievals. Our CFLOS calculation leverages the resolution and diversity of ERA-I that enables spatial coverage as well as frequency of occurrence CFLOS calculations for nearly all non-polar regions on earth.

1.

Introduction

The availability of ground observations of both active and passive airborne and satellite electro-optic and infrared (EO/IR) sensors can be limited by the sensor’s cloud-free line-of-sight (CFLOS) to the ground. Depending on the wavelength of interest, the clouds could be responsible for scattering and/or absorptive losses in addition to providing significant increases in thermal background radiation. To understand the availability and performance of airborne and satellite sensors, it is important to understand the spatial and temporal frequency of clouds and their structure from the ground to the top of the atmosphere. To assess the performance, we present a method to determine the altitude-dependent, vertical, cloud-free line-of-sight (CFLOS) using European Centre for Medium Range Weather Forecasting Re-Analysis Interim (ERA-I) cloud data. The uniqueness of this approach is that it enables spatial (lat/lon), as well as altitude dependence and temporal coverage of the variability of the global vertical, cloud-free line of sight. The Moderate Resolution Imaging Spectroradiometer (MODIS) satellite data enables global total cloud cover comparisons at high (lat/lon) resolution but provides no information with respect to the altitude dependence. Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO)/CloudSAT can be used to construct altitude-dependent CFLOS but is sparse in time with orbit repeats on the order of 16 days. The approach presented here utilizing ERA-I data enables global analytics on the native 0.7 deg field in addition to altitude dependence and temporal variation with model updates every 6 h for the entire globe. The following describes the methodology/approach, mathematical process model, and resultant global comparisons to MODIS and CALIPSO/CloudSAT. This method of utilizing the ERA-I cloud fields enables statistical views of cloud-free line of sight as a function of altitude as well as time of year for nearly anywhere on the globe.

Over the years, methods to determine the CFLOS have been developed that use statistical analysis of whole sky imaging (WSI) cameras15 to determine CFLOS from the ground to space. The works of Lund and Shanklin1,2 allowed for the creation of master probability matrices that describe the CFLOS from the ground to space for any elevation angle given total sky cover. As noted, this is for a surface observer to space and does not contain any information with respect to the altitude-dependent CFLOS. In addition, the WSI data have been processed to provide persistent statistics on CFLOS as well as calibrated whole sky irradiance values.6 The resultant data provide much needed inputs to statistical propagation analytics for active systems such as high-energy laser weapons7 as well as free-space optical communications8,9 and passive imaging systems for EO/IR detection as well as astronomical viewing. The results of WSI are quite useful for air to space or space to ground links but have limited utility in determining CFLOS as a function of slant and vertical paths that are less than the entire atmosphere. Other researchers10,11 utilized the results of Lund’s master probability matrices and cumulative clouds to construct CFLOS at limited altitudes. However, their data are highly dependent on relative cloud types and frequency that were determined by Lund. To improve this data, one can look across the globe at other sources of cloud information, but this is complicated by the sparseness of the data and diversity of these reporting metrics.12

An ongoing challenge has been to determine a method of calculating CFLOS as a function of altitude. A detailed analysis of CFLOS versus altitude as well as look angle was performed by Reinke et al.13 utilizing CloudSAT cloud profiling radar and coincident CALIPSO lidar data. The data have proven very useful for global trends and comparisons and is one of the best estimates of CFLOS as a function of both look angle as well as altitude. Unfortunately the CloudSAT data are sparse in time, as the repeat rate of the satellite is 16 days; thus, only long-term statistics can be evaluated via analysis of years of data. Here, we present a method to calculate CFLOS based on an isotonic interpolation of probabilities. We utilize the ERA-I dataset to construct altitude-dependent CFLOS and construct comparisons to CloudSAT CPR CFLOS13 as well as total cloud cover comparisons to MODIS data.14

2.

Overview of ERA-I Data Set

The probability of CFLOS has been analyzed using the European Centre for Medium Range Weather Forecasts Re-Analysis or ERA-Interim (aka ERA-I) dataset. The ERA-Interim dataset15,16 supplies cloud fraction (coverage) as a function of pressure/altitude. The first level starts at 0.1 hPA (65  km alt.) and increases in pressure to 1012 hPA (ground level) in a quasi-logarithmic fashion yielding higher spatial resolution near the ground. This results in vertical steps of hundreds of meters to 1 km for the first 10 km of altitude and 1- to 2-km steps up to 20 km of altitude. The altitude-dependent cloud fraction is the total cloud cover within each altitude bin when viewed from above (fractional areal coverage). When looking at the cloud fractional data on a global scale within the ERA-I dataset, no clouds were found above 20 km, which yields 39 useful vertical bins (ground to 20-km altitude). In addition to fractional cloud coverage versus altitude, ERA-I provides four other useful cloud data fields: the low (LO) cloud cover (clouds <2  km); medium (MED) cloud cover (2 to 6.5 km); high (HI) clouds (>6.5  km); and total cloud cover, which in our case is simply 1 minus the CFLOS from 20 km to the ground along a vertical path. The ERA-I data set that was utilized provided several cloud metrics on 0.7  deg×0.7  deg spatial grid with data every 6 h. The ERA-I data from 2017 was analyzed for the globe comprising: 1460×60×256×512 (time, pressure/altitude, latitude, and longitude) data points.

2.1.

Calculation of CFLOS from ERA-I

To determine the CFLOS, we need to take fractional cloud coverage as a function of altitude and construct a downward-looking probability of cloud obstruction (PCO), i.e., (1 − CFLOS), as we go up in altitude. This is done in a two-part process, described in brief here, and in more detail below. First, we compute the probability of cloud obstruction for the course LO, MED, and HI altitude regimes using the values provided by ERA-I in Table 1. This uses the combinatorics principle of inclusion and exclusion (PIE) to determine the PCO at each coarse altitude when looking down. Next, we calculate PCO(1) through PCO(39) for the high-resolution altitude bins 1 to 39. The calculations of PCO for these high-resolution altitude bins 1 to 39 are normalized with values calculated in the first step so that they are consistent with PCOLO, PCOMED, and PCOHI.

Table 1

PIE calculation for 3 levels from the low, mid, and high cloud fields provided by the ERA-I dataset.

3 Level PIE—Probability of Cloud Obscuration (1-CFLOS)Altitude range
PCO(0) = 00 to <2  km
PCOLO=PA2 to 6.5 km
PCOMED=PA+PB − (PAPB)6.5 to 20 km
PCOHI = PA + PB + PC − (PAPB) + (PAPC) + (PBPC) + (PAPBPC)Total cloud cover

STEP 1: Calculation of LO, MED, and HI altitude PCO

We employ the combinatorics PIE to compute PCOLO, PCOMED, and PCOHI. This is a method of combining probabilities—in our case, fractional cloud coverage as a function of altitude—to compute a singular probability. Our use of PIE is only valid for statistics where the individual events are not correlated, and so it cannot be applied directly to the higher spatial resolution cloud coverage (39 useable altitude bins) because the cloud structure is correlated on the vertical scale of hundreds of meters to perhaps as large as 2 km. There are methods to calculate PIE accounting for correlation in the data; however, they become computationally intensive and perhaps more importantly, rely very heavily on knowing the correlation between every single layer given by the 39 altitude bins. This would imply the user has a priori knowledge of the vertical cloud structure correlation not only seasonally (temporally) but also globally (spatially). In fact, one can show that a naïve direct application of PIE to the high-resolution bins will yield results that contradict the total cloud cover provided by ERA-I.

However, the principle of inclusion/exclusion can be used to combine the probability of a cloud obstruction (1-CFLOS) given by the ERA-I lower spatial resolution LO, MED, and HI cloud coverage values. This is approximately valid because the LO, MED, and HI clouds are vertically separated by a large enough spacing to be sufficiently uncorrelated to use PIE combinatorics. A detailed look at global cloud vertical and horizontal structure was performed by Guillaume et. al.17 utilizing CloudSAT data and a summary of the vertical cloud extent is shown in Table 2 of their work. The clouds present between each of the three coarse levels exhibit vertical structure on average that is less than the step size from LO to MED to HI in the ERA-I database. This provides for additional confidence in our assumption that the LO, MED, and HI clouds are sufficiently uncorrelated for combinatorics. Two notable exceptions are nimbostratus (NS) clouds with a mean thickness of 4.7 km17 and deep convection (DS) with a mean thickness of 9.3 km.17 In general these are LO probability cases as DS occurs <5% to 10% of the time18 and NS is rare except in polar regions >60  deg,18 a region we are not considering for our CFLOS calculation.

Table 2

ERA-I minus MODIS total cloud cover global statistics.

Total Cloud Cover Global Comparisons for Latitudes between ±60  deg
Month for 2017Mean (ERA-I minus MODIS)Std. dev. (ERA-I minus MODIS)
January−0.0750.128
February−0.0840.115
March−0.0750.106
April−0.0740.108
May−0.0710.100
June−0.0790.115
July−0.0790.120
August−0.0780.118
September−0.0700.108
October−0.0730.105
November−0.0750.110
December−0.0830.121

Figure 1 shows how we construct a three-level PCO by assessing the Venn diagram for each level. Starting at Cloud A (ERA-I LO cloud coverage), we simply determine the PCO to be the spatial (areal) coverage provided by Cloud A. As we go up through the atmosphere to a point just above Cloud B, we then need to determine the combination of probabilities of seeing a cloud PA+PB and subtract the region that has been double-counted PAPB. This approach is continued for our third level, and we thus have the probability of cloud obscuration for three distinct altitudes and is outlined in Table 1.

Fig. 1

The three-level PIE to determine the probability of seeing a cloud as a function of altitude.

JARS_16_2_028502_f001.png

An additional field, total cloud cover, reported in the ERA-I dataset was plotted against the PCO(HI) value, derived total cloud cover from combinatorics, and was found to be linear with a fit of x=0.999 y (R2>0.999), see Fig. 2. This further strengthens the argument that the clouds in the LO, MED, and HI cloud cover bins can be considered uncorrelated within the ERA-I data set.

Fig. 2

The total probability of seeing a cloud as derived from the three-level PIE approach is plotted against the total cloud field reported by the ERA-I data. The data shown represent nearly five million points around the contiguous United States for the year 2017 (1460 times, 38 latitudes, and 89 longitudes).

JARS_16_2_028502_f002.png

STEP 2: Calculation of PCO for high-resolution altitude bins 1 to 39

In our application, as we increase in altitude the PCO either remains the same or increases as we go through cloud layers. That is, the probability of obstruction monotonically increases, and this is a hard constraint. For this reason, we develop an isotonic interpolation model to approximate the vertical CFLOS from any of the altitude bins 1 to 39 looking down to the ground.

The isotonic interpolation is performed between the three boundary conditions: PCOLO, PCLOSMED, and PCOHI at altitudes of 2, 6.5, and 20 km, respectively. To perform this calculation, we take the cumulative sum of the fractional (area) cloud coverage altitude bins from with-in the ERA-I model from the ground to 2 km. Using the cumulative sum on the higher spatial resolution scale allows the data we actually do have to inform the rate at which our interpolator increases/accumulates probability between each of these coarse altitude bins. We then normalize the cumulative sum at the 2 km point to the 2-km PIE value—knowing that the cumulative sum of fractional cloud cover should equate to the cloud cover from the 2-km PIE point. That is, we enforce the boundary condition to ensure that the endpoints maintain correct cloud coverage values. In our case, the ground has no clouds and the 2-km point is equivalent to the LO cloud PCOLO value. The points between the ground and 2 km are then interpolated by the normalized cumulative sum from the ground to 2 km—resulting in a cloud obstruction profile (1-CFLOS) from 0 to 2 km in altitude.

Eq. (1)

cumulativeCloudFraction(alt)=ccf(alt)=altStartaltEndcloudFraction(alt).

This process is repeated from the 2- to 6.5-km mMED cloud level (normalizing the cumulative sum profile endpoints to the end point PIE values PCOLO (2 km) and PCOMED (6.5 km) and finally, for the 6.5- to 20-km cloud level (PCOHI). We concatenate the results and create a PCO (1-CFLOS) as a function of altitude with vertical resolutions equivalent to the original ERA-I data. These calculations are summarized by the piecewise defined interpolation function

Eq. (2)

PCO(alt)={ccf(alt)ccf(2  km)×(PCOLOPCOG)+PCO(0  km)alt[0,2]ccf(alt)ccf(6.5  km)×(PCOMEDPCOLO)+PCO(2  km)alt(2,6.5]ccf(alt)ccf(20  km)×(PCOHIPCOMED)+PCO(6.5  km)alt(6.5,20],
where ccf stands for the cumulativeCloudFraction as calculated using Eq. (1).

This calculation is shown graphically in Fig. 3, where the cloud fraction versus altitude from the ERA-I data on the left and the resultant CFLOS is shown on the right. In vertical regions where clouds exist the CFLOS reduces to account for the existence of clouds and when the fractional cloud coverage goes to zero, we see the CFLOS remain constant as a function of altitude until an additional cloud is found. There is a specific set of cases where this will result in inaccurate results: noteworthy is any region where we have two significant fractional cloud (>0.75) layers separated by a region of no clouds within either the LO, MED, or HI altitude zones. The cumulative summation and normalization will smooth over the first cloud, resulting in clearer conditions at the lower altitude bin than otherwise should be seen; fortunately, this is found to occur <1% of the time.

Fig. 3

(a) Fractional cloud coverage from ERA-I data. (b) Derived CFLOS using the three-level PIE and cumulative summation normalization.

JARS_16_2_028502_f003.png

3.

Process Model Justification

In the preceding section, we described an isotonic interpolation of probability. We now describe this modeling in a more rigorous fashion and provide a reasonable justification for its use.

3.1.

Orthographic Projection and Geometric Probability

The topic of geometric probability is multi-disciplined and covers several niche probability problem types.19 However, for our purposes, here, we are using the term geometric probability to describe a specific ratio of measures that might be found in the subject. Consider a cell, or element of volume if one prefers, Rijk in an over-approximating rectangular partition of a region R of the atmosphere. And let us then define the right rectangular solid

Eq. (3)

Skl,ku={(x,y,z)R|xi1xxi,yj1yyj,zkl<zzku},
which is the union of Rij(kl+1),,Rijku. The intersection of the clouds C with Skl,ku is simply Ckl,ku=CSkl,ku. The upper face Fku of the cylinder will be a rectangle in the plane z=zku. We orthographically project Ckl,ku onto Fku via the mapping Pkl,ku:Skl,kuFku:p=(x,y,z)(x,y,zku). Let μ denote the Lebesgue measure on Fku. Then, the geometric probability of cloud coverage for the cylinder Skl,ku is given as

Eq. (4)

qkl,ku=μ(Pkl,ku(Ckl,ku))μ(Fku),
so long as the projection Pkl,ku(Ckl,ku) is measurable. An interpretation of this probability is that it is the likelihood qkl,ku that sighting along a vertical ray from altitude kl to altitude ku will be blocked by the clouds, PCO. Let k0 be the altitude index of ground and define qk0,k0=0. We will let pkl,ku denote the complementary probability of CFLOS for Skl,ku.

3.2.

Bernoulli Processes

Let D={(x,y)|(x,y,z)Skl,ku}, a rectangle of side lengths Δx and Δy. Then, for any (x,y)D chosen uniformly at random, we will assign to Bkl,ku the value of 1 if we have CFLOS through Skl,ku and 0 else. {Bkl,ku} is then a two-parameter family of Bernoulli random variables with probability of ‘success’ pkl,ku and failure qkl,ku.

Recalling that Skl,ku=k=kl+1kuRijk and keeping firmly in mind the definitions made heretofore, it should be clear that there are strong relationships between members of the family {Bkl,ku} and the associated distribution parameters consistent with the underlying geometry. Indeed, we have in general the constructive relationship pkl,ku=P(Bkl,ku=1)=P(Bkl,k+Bk,ku=2). We will single out of the family the one step variables Bk1,k and denote them simply Bk. The recurrance probability relation is then calculated as

Eq. (5)

pkl,ku=P(Bkl,ku=1)=P(Bkl,ku1+Bku=2)==P(Bkl+1++Bku=kukl).

Contextualizing within our application, we now observe the following. Our problem data is the distributional parameterization {pk}k=kl+1ku of the Bernoulli process {Bk}k=kl+1ku, the fractional (area) cloud coverage altitude bins from with-in the ERA-I model. We are aiming to determine the distributional parameterization {pkl,k}k=kl+1ku of the Bernoulli process {Bkl,k}k=kl+1ku, and Eq. (5) provides the connection between what we are given and what we desire. The goal of our isotonic interpolation is in fact to model pkl,k in terms of the pl, l[klk], where [klk]kl+1,,k.

3.3.

Probability Models

We will now introduce reasonable arithmetic and geometric recurrence models for pkl,k in terms of the pl, l[klk], and then indicate why we proceed with the arithmetic model as the basis for our isotonic interpolation model and how this arithmetic model gets configured into an interpolant.

3.3.1.

Arithmetic model development

To develop the arithmetic model, we examine the complementary probability, which can be expressed using De Morgan’s law as

Eq. (6)

qkl,ku=P((Bkl,ku1=0)(Bku=0)),

Eq. (7)

=P(Bkl,ku1=0)+P(Bku=0)P((Bkl,ku1=0)(Bku=0)),

Eq. (8)

=P(Bkl,ku1=0)+[P(Bku=0)P(Bku=0|Bkl,ku1=0)P(Bkl,ku1=0)],
where the second equality follows from the PIE. Rewriting the last member of the equation in terms of distributional parameters and rearranging, we have the difference/recurrance relation qkl,kuqkl,ku1=d where

Eq. (9)

d=qkuP(Bku=0|Bkl,ku1=0)qkl,ku1.

We focus on difference field d. Appealing to the geometry qkuqkl,ku1 is a difference in the measures of the cloud projections of the ku partition strata and the cloud projections from the strata within a preceding right rectangular cylinder (a column of atmosphere below). Apparently, this difference can be negative whereas the difference field cannot. The probability P(Bku=0|Bkl,ku1=0) is the premodifier on the prior that appropriately conditions the field so that this never occurs. In the case of positive associations between the random variables Bku and Bkl,ku1, this probability is larger and reduces the difference field. In the case of negative associations between these random variables, the probability is slight and increases the difference field. In effect, which is the case amounts to whether we are dealing with the same clouds or not as we enter the ku’th strata. Let us assume that we are dealing with a portion of atmosphere of a size that suggests positive association between cload masses. The P(Bku=0|Bkl,ku1=0) then serves to diminish the positive attibution of the difference field to the accumulation captured by this difference equation.

The remaining issue is that we do not have access to any data in our problem to provide P(Bku=0|Bkl,ku1=0). This difference field modifier cannot be sensibly computed from the ql. For a given 0mp1, there are any given number of regions Pmp in Fl that yield the geometric probability mp. And while those regions cannot be dispersed, we know of no bases for an argument that they have centrality or tendency with respect to location in Fl. Generally speaking, we do not know what the blotches will look like or where they will be, so we cannot make inferences about the projection of a cloud distribution in a cell, or the cloud distribution in the cell itself, using ql. We lose the geometry in the measure of the projection. Based on this observation, we can only say that the extent to which qku may potentially contribute positively to the difference qkl,kuqkl,ku1 is directly proportional to qku. Defining aku as the proportionality constant, we posit

Eq. (10)

qkl,ku1=qkl,kuakuqku.

We assume consensus across strata with respect to the constant of proportionality, which we will denote by a. We have then the arithmetic recurrance model

Eq. (11)

qkl,ku1=qkl,kuaqku.

3.3.2.

Geometric model development

Our geometric model comes to us directly from the Conway–Maxwell–Poisson (CMP) binomial distribution.20 This distribution generalizes the Poisson binomial distribution, which characterizes Bernoulli sums, to the case were the probabilities of success are associated.21 We obtain from the CMP binomial distribution

Eq. (12)

pkl,ku=P(Bkl,ku1+Bku=2)=1Ckl,ku1νpkl,ku1pu,
where

Eq. (13)

Ckl,ku1ν=qkl,ku1qu+2ν1(qkl,ku1pu+pkl,ku1qu)+pkl,ku1pu.

If our CFLOS events over strata were independent, ν and Ckl,ku1ν=1, and we get a very clear-cut product relationship between pkl,ku and the pl. In the case of our assumed positively associated events, however, ν<1, which yields 0<C(kukl)ν<1, scaling the product up appropriately. This scale factor is nonlinear in ν and dependent on kl,ku. We have no clear synthesis method through which to obtain ν(kl,ku) from our problem data. This is a beautiful, but complicated model. But treating with it as we did with the difference model, we have, upon defining bku as the proportionality constant

Eq. (14)

pkl,ku=bkupkl,ku1pku.

Assuming consensus across strata in the correction factor, call it b, we have the geometric recurrance model

Eq. (15)

pkl,ku=bpkl,ku1pku.

3.4.

Sensitivity Analysis and Justification of Arithmetic Model Use

Solving the arithmetic recursion model yields us

Eq. (16)

qkn1,kn+1=qkn1,kn+l=kn+1kn+1alql,
while solution of the geometric recursion model provides

Eq. (17)

pkn1,kn+1=(l=kn+1kn+1bl)(l=kn+1kn+1pl)pkn1,kn.

Let us now compute the relative sensitivities of these recursion solutions with respect to their respective proportionality constants. In the case of the arithmetic recursion direct computation provides

Eq. (18)

akqkn1,kn+1qkn1,kn+1ak=akqkqkn1,kn+l=kn+1kn+1alql1,
when defined. Equality only holds in the case that qkn1,kn and all ql, lk, in the summation are 0. In the case of the geometric recursion direct computation provides

Eq. (19)

bkpkn1,kn+1pkn1,kn+1bk=1,
when defined.

The takeaway is the following. In assuming consensus in the proportionality constant, we are inducing a perturbation away from the more ideal strata dependent proportionality constants. The percent change in the correct probabilities due to a percent change in the parameters due to these pertubations is significantly less in the case of the arithmetic model than would be seen with the geometric model. Simply put, the geometric model is appreciably more sensitive to our gross practical simplifications. For this reason alone, we make use of the arithmetic model instead of the geometric model here, even though the latter has a more philosophically pleasing basis.

3.5.

Arithmetic Recurrence Model Use in Interpolation

Suppose then that we have a two-point boundary value problem with given initial condition qkn1,kn and final condition qkn1,kn+1 and that we seek qkn,k for k[knkn+1]. Let us return to our Eq. (16) solution to the arithmetic recurrance relation with the consensus assumption back in force. We have

Eq. (20)

qkn1,kn+1=qkn1,kn+al=kn+1kn+1ql,
from which we readily obtain

Eq. (21)

a=qkn1,kn+1qkn1,knl=kn+1kn+1ql,
and for k[knkn+1], we have

Eq. (22)

qkn,k=qkn1,kn+(qkn1,kn+1qkn1,knl=kn+1kn+1ql)l=kn+1kql,
as a isotonic interpolant between the boundary data. The final piece of mathematical detail is realizing that this is the interpolant used repeatedly along with accurate PIE generated boundary data to obtain isotonic interpolation Eq. (2).

4.

Comparisons: MODIS Total Cloud Comparison

The global total cloud cover from the ERA-I data was averaged on a monthly basis for 2017 and compared to the MODIS data14 for the same period by looking at the difference in cloud cover on a latitude and longitude grid. The 0.1deg MODIS data were interpolated and downsampled to the 0.7deg ERA-I data and directly compared by taking the difference between the ERA-I total cloud cover and MODIS total cloud cover. The mean difference was calculated for latitudes between ±60  deg and was found to be 0.074 with a standard deviation of 0.11. This confirms other researchers’ findings2224 that the ERA-I data are generally in good agreement with a bias toward clearer conditions and are relatively inaccurate at the poles. By utilizing the MODIS total cloud cover as a source of calibration data, we can feel confident in our PIE combinatorics approach to the ERA-I data set that uniquely enables global altitude-dependent CFLOS metrics with temporal variation as the cloud fields in ERA-I are updated every 6 h. Figure 4 shows the mean total cloud cover for the month of April 2017; on the left is the ERA-I data and on the right is the MODIS data. Figure 5 shows the difference in total cloud cover for December 2017—the color scale has been set using the mean as the center with the limits set to ±2 standard deviations. A summary of the total global cloud cover and and statistical comparisons to MODIS broken down by month is shown in Table 2. The monthly breakdown shows little variability in the relative total cloud cover for ERA-I and MODIS with a consistent bias of ERA-I toward marginally clearer conditions. See Appendix A for ERA-I total cloud cover minus MODIS total cloud cover for each month of 2017.

Fig. 4

Mean total cloud cover for April 2017. Left: ERA-I data analysis results from this study using PIE combinatorics and cumulative-summation normalization. Right: MODIS on NASA’s Terra Satellite.14

JARS_16_2_028502_f004.png

Fig. 5

Difference between ERA-I and MODIS total cloud cover for December 2017. The color scale has been set using the mean difference as the center and the limits set to ±2σ as described in Table 2 for December. See Appendix A for a detail of each month.

JARS_16_2_028502_f005.png

5.

Comparisons: CloudSAT Comparisons

We also compared the ERA-I derived vertical CFLOS to the CFLOS determined by Reinke et al.13 using CloudSAT data. Figure 6 shows the average over the month of April of the vertical CFLOS from 9.6-km altitude; on the left is the ERA-I-derived CFLOS and on the right is the CloudSAT-derived CFLOS. The data were then directly compared by taking the difference between the ERA-I and CloudSAT data. This was performed by interpolating the 0.7deg gridded ERA-I data to the 1deg gridded CloudSAT data. The mean difference was calculated for latitudes between ±60  deg and was found to be <0.025 for most altitudes below 10 km with larger differences at higher altitudes. The standard deviation was larger with σ0.2 for each altitude. As noted earlier in the comparisons to MODIS, the total cloud cover in ERA-I has a bias toward clearer conditions; this is also directly seen in the altitude-dependent CFLOS calculated with ERA-I and compared to CloudSAT/CALIPSO. Comparative statistics of CFLOS for each altitude are shown in Table 3. Analyzing the results of this table, we can see that the difference in CFLOS is seen to grow mostly between 11 to 16  km. The higher-altitude cirrus clouds within the ERA-I dataset are underrepresented relative to CloudSAT/CALIPSO and lead to lower total cloud cover when compared to MODIS. Figure 7 shows the difference in CFLOS for April from 9.6-km altitude—the color scale has been set using the mean as the center with the limits set to ±2 standard deviations. Upon careful inspection, one will notice that the CloudSAT/CALIPSO data have a linear structure that is clearly the result of the satellite’s orbit. Additional global plots of the altitude-dependent difference in ERA-I CFLOS and CloudSAT CFLOS can found in Appendix B.

Fig. 6

Probability of vertical CFLOS from 9.6-km altitude in April. (a) ERA-I data analysis results from this study with three-point PIE and cumulative-summation normalization. (b) CloudSAT/CALIPSO (Reinke et al.).25

JARS_16_2_028502_f006.png

Table 3

Difference in vertical CFLOS (ERA-I minus CloudSAT) for the month of April as a function of altitude for latitudes between ±60  deg.

Alt kmMean (ERA-I minus CloudSAT)Std. dev. (ERA-I minus CloudSAT)Alt kmMean (ERA-I minus CloudSAT)Std. dev. (ERA-I minus CloudSAT)
0.960.070.15510.56−0.0270.175
1.920.0010.17611.52−0.0320.177
2.88−0.0180.17712.48−0.0390.177
3.84−0.0200.17713.44−0.0480.175
4.80−0.0210.17714.40−0.0600.173
5.76−0.0200.17615.36−0.0720.174
6.72−0.0210.17516.32−0.0820.177
7.68−0.0160.17617.28−0.0880.179
8.64−0.0130.17618.24−0.0880.179
9.60−0.0250.17419.20−0.0880.179

Fig. 7

Difference between ERA-I and CloudSAT CFLOS for April from 9.6-km above the ground. The color scale has been set using the mean difference as the center and the limits set to ±2σ as described in Table 3 for altitude = 9.6 km. Appendix B contains global maps for additional altitude levels.

JARS_16_2_028502_f007.png

In addition to global analysis, a detailed look at the United States was performed for the ERA-I data from 2017. Figure 8 shows the mean CFLOS from 9.6-km altitude for 2017 over the United States. Outside of the southwest United States, one would expect to have a CFLOS to 9.6-km altitude approximately half of the time.

Fig. 8

The 2017 year-long mean vertical CFLOS from 9.6-km altitude—derived from the ERA-I cloud data as described herein.

JARS_16_2_028502_f008.png

6.

Temporal Analysis

One of the major advantages of using the ERA-I data is the 6 h temporal resolution that is available. Utilizing the time domain data, we are able to assess the frequency of occurrence of clouds around the globe. We approach this by calculating a family of percentile based curves. Each family of color curves represent the percentage of time that a certain CFLOS will be observed. Thus, the cloud-free line-of-sight can be thought of as the amount of ground or sky in a vertical slice one will observe that is cloud-free, while the color curves tell you how often you would expect to see that much. Figure 9 shows the CFLOS derived herein from the ERA-I data as a function of altitude. The left side is for the Johns Hopkins Applied Physics Laboratory facility located in Laurel, Maryland, and the right side of Fig. 9 shows Yuma, Arizona. The top plots are a result of aggregating the CFLOS curves for the summer (June, July, and August) and the bottom sets of curves are from aggregating the winter CFLOS (December, January, and February). The data show that in general, winter has more clouds in both Laurel, Maryland, and Yuma Arizona. If one focuses on the plot of the CFLOS for Yuma Arizona in the winter (bottom-right of Fig. 9), the lower altitudes are generally cloud-free. As we go up in altitude, the middle percentile curves start to turn toward lower CFLOS between 7 and 12 km, indicative of cloud layers. In any of these types of plots, the cloud layers become immediately obvious by looking for large drops in CFLOS as a function of altitude.

Fig. 9

Vertical CFLOS for summer and winter. Left: JHU/APL Laurel Maryland. Right: Yuma, Arizona. Winter in general tends to have the most clouds in the US. Large drops in CFLOS as a function of altitude are indicative of cloud layers. The families of colored curves represent the percentage of time one would expect to have a certain CFLOS to the ground.

JARS_16_2_028502_f009.png

7.

Conclusions

A method of calculating the CFLOS from the ERA-I data has been developed. The results of the ERA-I total cloud cover have been compared on a global scale to MODIS for latitudes between ±60  deg and in general are shown to under-predict clouds relative to MODIS with total cloud cover 7.5% less than MODIS with a standard deviation of 10%. Additionally, the CFLOS derived from the ERA-I dataset has been compared to CloudSAT CPR data as a function of altitude on a global scale for latitudes between ±60  deg and shows on average good agreement with a larger standard deviation of 0.2. In addition to spatial analysis of CFLOS (latitude, longitude, altitude), we have looked at the ERA-I data every 6 h over multiple years, thus enabling the end-user to look at the temporal distribution of CFLOS as well. The families of percentile-based curves were developed to represent the percentage of time one would expect to have a certain cloud-free-line-of-sight to the ground as a function of altitude. It is understood that the ERA-I cloud derivation is not perfect and in some rare (<1% of the time) instances it will over-predict the CFLOS for lower altitudes. However, our method of retrieving altitude-dependent CFLOS from ERA-I data enables many engineering tasks not previously possible. The ERA-I technique provides for vertical, altitude-dependent CFLOS on a global scale with temporal density; e.g., profiles every 6 h. CloudSAT/CALIPSO retrievals provide for very accurate measures of the vertical structure but are temporally sparse. The ability to look at the vertical structure on a more dense temporal scale provides for operational decisions with respect to airborne sensors for optimum altitudes of operation utilizing the statistics of CFLOS for day versus night, winter versus summer etc. Future iterations would include additional knowledge of vertical cloud coherence to refine the probability models.

8.

Appendix A: ERA-I Minus MODIS Total Cloud Cover

The global total cloud cover difference for ERA-I minus Modis has been analyzed to understand statistical variability over a years? worth of data. The monthly breakdown shows little variability in the relative total cloud cover for ERA-I and MODIS with a consistent bias of ERA-I toward marginally clearer conditions (Fig. 10).

Fig. 10

The ERA-I minus MODIS total Cloud Cover, monthly average for 2017. Color scale has been set to a zero mean difference with ±2σ limit. Statistics for each month are described in Table 2.

JARS_16_2_028502_f010.png

9.

Appendix B: ERA-I CFLOS Minus CloudSAT CFLOS

Global altitude dependent differential CFLOS for ERA-I and Cloudsat. Lower altitudes show very close agreement with altitudes above 10 km yielding marginally larger differences due to ERA-I under representing the higher altitude cirrus clouds (Fig. 11).

Fig. 11

The ERA-I CFLOS minus CloudSAT CFLOS as a function of altitude. Color scale has been set to a zero mean difference with ±2σ limit. Statistics for each altitude are described in Table 3.

JARS_16_2_028502_f011.png

Acknowledgments

The authors thank Don Reinke for supplying the CloudSAT/CPR CFLOS data for the month of April. The work presented here was funded by The Strategic Capabilities Office. The authors have no conflict of interests.

References

1. 

I. Lund and M. Shanklin, “Photogramettrically determined cloud-free lines-of-sight through the atmosphere,” J. Appl. Meteorol., 11 773 –782 (1972). https://doi.org/10.1175/1520-0450(1972)011<0773:PDCFLO>2.0.CO;2 Google Scholar

2. 

I. Lund and M. Shanklin, “Universal methods for estimating probabilities of cloud-free lines-of-sight through the atmosphere,” J. Appl. Meteorol., 12 28 –35 (1973). https://doi.org/10.1175/1520-0450(1973)012<0028:UMFEPO>2.0.CO;2 Google Scholar

3. 

I. Lund, “Persistence and recurrence probabilites of cloud-free and cloudy lines-of-sight through the atmosphere,” J. Appl. Meteorol., 12 1222 –1228 (1973). https://doi.org/10.1175/1520-0450(1973)012<1222:PARPOC>2.0.CO;2 Google Scholar

4. 

J. Shields et al., “New cloud free line of sight statistics measured with digital whole sky imagers,” Proc. SPIE, 5891 58910M (2005). https://doi.org/10.1117/12.615517 PSISDG 0277-786X Google Scholar

5. 

J. Shields et al., “Measurement and evaluation of cloud free line of sight with digital whole sky imagers,” in The Battlepace Atmos. and Cloud Impacts on Mil. Oper. (BACIMO), (2005). Google Scholar

6. 

J. E. Shields et al., “Day-night whole sky imagers for 24-h cloud and sky assessment: history and overview,” Appl. Opt., 52 (8), 1605 –1616 (2013). https://doi.org/10.1364/AO.52.001605 APOPAI 0003-6935 Google Scholar

7. 

D. Nahrstedt, “Cloud modeling for laser weapon propagation analysis,” Proc. SPIE, 4034 1 –13 (2000). https://doi.org/10.1117/12.391859 PSISDG 0277-786X Google Scholar

8. 

R. Venkat and D. W. Young, “Cloud-free line-of-sight estimation for free space optical communications,” Proc. SPIE, 8732 873205 (2013). https://doi.org/10.1117/12.2019269 PSISDG 0277-786X Google Scholar

9. 

R. Alliss and B. Felton, “The mitigation of cloud impacts on free space optical communications,” Proc. SPIE, 8380 83800S (2012). https://doi.org/10.1117/12.918403 PSISDG 0277-786X Google Scholar

10. 

L. Yunying and S. Litan, “Calculation of probability of cloud-free lines-of-sight at given heights in foshan, china,” Adv. Atmos. Sci., 16 297 –303 (1999). https://doi.org/10.1007/BF02973089 Google Scholar

11. 

S. Yu, K. Case and J. Chernick, “Methodologies and comparisons for lund’s two methods for calculating probability of cloud-free line-of-sight,” J. Clim. Appl. Meteorol., 25 389 –397 (1986). https://doi.org/10.1175/1520-0450(1986)025<0389:MACFLT>2.0.CO;2 JCAMEJ 0733-3021 Google Scholar

12. 

M. Free and B. Sun, “Time-varying biases in U.S. total cloud cover data,” J. Atmos. Ocean. Technol., 30 2838 –2849 (2013). https://doi.org/10.1175/JTECH-D-13-00026.1 JAOTES 0739-0572 Google Scholar

13. 

D. Reinke et al., “Probability of cloud-free line of sight (pcflos) derived from cloudsat cloud profiling radar and coincident lidar data,” in Preprints, 17th Conf. Satell. Meteorol., (2010). Google Scholar

14. 

R. Stockli, “Nasa’s Earth observatory - modis atmospheric imagery,” https://neo.sci.gsfc.nasa.gov/view.php?datasetId=MODAL2_M_CLD_FR Google Scholar

15. 

P. Berrisford et al., “The era-interim archive version 2.0,” (2011). https://www.ecmwf.int/node/8174 Google Scholar

16. 

D. Dee et al., “The era-interim reanalysis: configuration and performance of the data assimilation system,” Q. J. R. Meteorol. Soc., 137 553 –597 (2011). https://doi.org/10.1002/qj.828 QJRMAM 0035-9009 Google Scholar

17. 

A. Guillaume et al., “Horizontal and vertical scaling of cloud geometry inferred from cloudsat data,” J. Atmos. Sci., 75 2187 –2197 (2018). https://doi.org/10.1175/JAS-D-17-0111.1 JAHSAK 0022-4928 Google Scholar

18. 

K. Sassen and Z. Wang, “Classifying clouds around the globe with cloudsat radar: 1-year of results,” Geophys. Res. Lett., 35 1 –5 (2008). https://doi.org/10.1029/2007GL032591 GPRLAJ 0094-8276 Google Scholar

19. 

H. Solomon, Geometric Probability, Society for Industrial and Applied Mathematics(1978). Google Scholar

20. 

G. Shmueli et al., “A useful distribution for fitting discrete data: revival of the Conway–Maxwell–Poisson distribution,” J. R. Stat. Soc.: Ser. C (Appl. Stat.), 54 (1), 127 –142 (2005). https://doi.org/10.1111/j.1467-9876.2005.00474.x Google Scholar

21. 

J. B. Kadane, “Sums of possibly associated Bernoulli variables: the Conway–Maxwell–binomial distribution,” Bayesian Anal., 11 (2), 403 –420 (2016). https://doi.org/10.1214/15-BA955 Google Scholar

22. 

C. Naud, J. Booth and A. D. Genio, “Evaluation of the era-interim and merra cloudiness in the southern ocean,” J. Clim., 27 2109 –2124 (2014). https://doi.org/10.1175/JCLI-D-13-00432.1 JLCLEL 0894-8755 Google Scholar

23. 

M. Stengel et al., “Comparing era-interim clouds with satellite observation using a simplified satellite simulator,” Atmos. Chem. Phys, 18 17601 –17614 (2018). https://doi.org/10.5194/acp-18-17601-2018 ACPTCE 1680-7324 Google Scholar

24. 

M. Free, B. Sun and H. Yoo, “Comparison between total cloud cover in four reanalysis products and cloud measured by visual observations at U. S. weather stations,” J. Clim., 29 2015 –2021 (2016). https://doi.org/10.1175/JCLI-D-15-0637.1 JLCLEL 0894-8755 Google Scholar

25. 

D. Reinke, “Probability of cloud free line of sight,” (2021) https://www.cloudsat.cira.colostate.edu Google Scholar

Biography

Adam Willitsford received his BS degree in electrical and computer engineering from Lafayette College (2002) and his PhD in electrical engineering from The Pennsylvania State University (2008). He is a member of the Senior Professional Staff at the John’s Hopkins University Applied Physics Lab in the Optics and Photonics Group. In addition, he teaches fundamentals of lasers in the Engineering for Professionals program at John’s Hopkins University. His research interests include active and passive remote sensing and imaging systems as well as statistical analysis of EO/IR atmospheric propagation.

Gregory Hicks was awarded his PhD in applied mathematics from North Carolina State University (2003). He is member of the senior professional staff the John’s Hopkins University Applied Physics Lab and is the current supervisor of the Advanced Concepts Section of the Air and Missile Defense Sector’s Guidance, Navigation, and Control Group. His current research is principally focused on geometric and algebraic control synthesis methods for multi-agent systems, formal methods, stochastic systems, and optimization.

Walt Bowen received his bachelor’s and master’s degrees in electrical engineering from Virginia Tech in 1986 and 1988, respectively. He is currently a member of the principle professional staff at the Johns Hopkins University Applied Physics Laboratory in the Guidance, Control, Navigation, and Seekers group in the Force Projection Sector. He is the chief engineer of this group. His work currently focuses on the integration of optical and RF sensors into flight guidance systems.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Adam H. Willitsford, Gregory Hicks, and Walt Bowen "Altitude-dependent probability of vertical, cloud-free line-of-sight from European Centre for Medium Range Weather Forecasting Re-Analysis-interim cloud cover," Journal of Applied Remote Sensing 16(2), 028502 (28 May 2022). https://doi.org/10.1117/1.JRS.16.028502
Received: 2 November 2021; Accepted: 13 May 2022; Published: 28 May 2022
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
KEYWORDS
Clouds

MODIS

Satellites

Weather forecasting

Data modeling

Spatial resolution

Infrared imaging

Back to Top