2.1.

Instrument Design

The developed quantitative imaging device consists primarily of two commercially available components controlled by a computer: a digital LED miniprojector (Innoio, SmartBeam) and a 1.3-MP USB2 color camera (PointGrey, Firefly MV) with a C-mount 12-mm lens (Edmund Optics, UC series). The camera and projector are mounted in a fixed position relative to each other, as shown in Fig. 1, so that they have essentially the same focal plane and FOV. In this setup, we have a working distance of 15 cm, which gives us an FOV of $\sim 10\times 5\mathrm{cm}$. We are not able to control the projector digital micromirror device directly, so it was employed as a second monitor where the spatially modulated patterns are projected full-screen as images on it. This is a common practice in other low-cost SFDI systems.^22,23 The camera was controlled with the software development kit (SDK) provided by the manufacturer (PointGrey, FlyCapture SDK), and the whole acquisition procedure was controlled with Python scripts. Since it is not possible to synchronize the camera with the projector, the exposure time during acquisition should be a multiple of the inverse of the frame-rate ($1/60\mathrm{Hz}$) in order to avoid flickering in the image intensity.²³ In order to block specularly reflected reflected light, two cross-polarisers have been applied in front of the camera, and the projector rotated by 90 deg with respect to each other.

Fig. 1

Schematic of the acquisition device. A mini projector is used to shine a sinusoidal pattern on the tissue, then a CMOS digital camera captures the reflected light. Both components are controlled by a computer, and cross-polarizers are placed in front of them to remove specular reflected light.

2.2.

Spectral Characterization

The spectral specifications of these devices are not known, and there might also be variability between components in the same series. For this reason, it is necessary to characterize them individually, to know their exact spectral features. The main spectral bands and the cross-channel bands have been characterized using two different approaches, one using laboratory equipment and another using alternative low-cost components. This way it was possible to compare the results obtained with the two methods and determine if the low-cost approach is reliable enough to be used in research.

2.2.1.

Laboratory approach

First, an intensity calibration of a spectrometer (Avantes, ULS2048-RS-USB2) was performed, using a calibrated broadband light source (Avantes, Avalight-HAL-CAL-Mini) of which we know the exact intensity over the emission spectrum. To characterize the LEDs, pure red, green, and blue images, respectively, were projected on a 99% reflectance target (Labsphere, calibrated reflectance standard), the reflected light was captured with an optic fiber, and the spectral signature was measured with the intensity calibrated spectrometer (Fig. 2). The spectrum of the three LEDs was normalized to the maximum value of all three bands, in order to preserve the relative intensity ratios.

Fig. 2

Schematics of the setup used to calibrate the projector: each individual LED shines light on a 99% reflectance reference. The reflected light is captured by an optic fiber and its spectrum is measured with an intensity calibrated spectrometer.

To measure the characteristic of the camera filters, the following setup was used (Fig. 3): an optic fiber was used to collect light from a light source and concentrate it on a small spot, pointed directly at the camera. A transmission holographic grating (500 grooves/mm) was then placed in front of the camera to separate the wavelengths of the light along a line, and we have acquired images to process. First, a mercury lamp (StellarNet Inc., SL2 calibration lamp) was used to calibrate the wavelengths, as its spectrum has several discrete peaks at known wavelengths that can be used as a reference. To calibrate the intensity, the calibrated broadband white light source was used (Avantes, Avalight-HAL-CAL-Mini). To obtain the characteristic of the red, green, and blue filters on the CMOS sensor of the camera, the intensity profiles of the diffracted light have been analyzed in the red, green, and blue channels of the white light pictures, normalizing their value to the maximum value in all three channels.

Fig. 3

Schematics of the setup used to calibrate the camera: a light source is shone with an optic fiber through a grating, which separates the light in its spectral components along a line. The intensity of the diffracted light is then measured with the camera. A mercury lamp is used for wavelength calibration and a broadband light for intensity calibration.

Finally, the spectral features of the LEDs and camera that were obtained from this characterization procedure have been combined. The three LED spectra have been multiplied by each of the three color filters function, obtaining this way nine different combinations (the three main color bands and six cross-channel bands).

2.2.2.

Low-cost approach

In the second procedure, the intensity of the cross-channels has been measured directly with the following experimental setup (as presented in Fig. 4). A transmission grating was placed in front of the projector, and a black image with a 1-px vertical line of a pure color (red, green, or blue) was projected on a 99% reflectance target. The grating diffracts the wavelengths horizontally allowing us to interpret the spectral distribution of each LED on the camera directly. The light intensity profiles in each of the color channels of the acquired pictures are a measure of the combined effect of the color filter function and the LED spectrum. A wavelength calibration procedure was performed using two laser pointers of known wavelengths (532 and 650 nm), in a manner similar to the mercury lamp in Sec. 2.2.1, and the signals have been normalized to the maximum value of the each of the three primary LED channels for comparison.

Fig. 4

Schematic of the setup used for characterizing the system directly. The projector is used to shine a vertical line of each individual LED color on a 99% reflectance target. The light passes through a grating, which separates its spectral components, and it is captured by the camera. A wavelength calibration is then performed using two laser pointers, in a way similar to Fig. 3.

2.3.

SFDI Acquisition Procedure

To perform SFDI, multiple sinusoidal patterns are projected sequentially on the tissue, and the remitted light (i.e., diffuse reflectance) is measured using the camera. Most commercial projectors, however, use an internal gamma correction to match the nonlinear response of the human eye, and they intentionally alter the brightness levels so that the output is nonlinear. Since pure sinusoidal patterns would be distorted by this gamma correction, it has been characterized and compensated, using the procedure described in previous papers.^22,23 For the following data processing steps, it is necessary to measure the amplitude of the AC component of the modulated reflectance. To do so, we employ a multiphase demodulation technique in the spatial domain.⁴ This technique requires the acquisition of three sinusoidal signals with a spatial phase shift of 0 deg, 120 deg, and 240 deg. A simple GUI was developed to control the camera parameters (such as the exposure time), the number of spatial frequencies ($f_x$) to generate (at least two) and to start the acquisition in either single acquisition mode or continuous acquisition mode. In this study, patterns at four spatial frequencies (0, 0.0333, 0.05, $0.1{\mathrm{mm}}^{-1}$) have been employed. For each $f_x$, images were projected at each RGB color and three phases, sequentially, for a total of nine images per $f_x$. Then, the individual color channels from the camera needed for our application (3 RGB channels and 2 cross-channels) were extracted and saved in grayscale in uncompressed 8-bit.bmp format. The total acquisition time for an entire sequence was between 10 and 20 seconds, depending on the camera exposure time. The system has not been optimized for speed, however.

2.4.

Data Processing

The data were processed using the methods described in Ref. 4. The first step to perform is to demodulate the images to obtain the AC amplitude of the signal. Equation (1) has been applied pixel by pixel, where $I_1$, $I_2$, and $I_3$ are the images at a specific $f_x$ and spectral band, to obtain the AC amplitude ($M_{\mathrm{AC}}$):

The second step is the calibration. An SFDI dataset has been acquired under the same experimental conditions on a reference calibration phantom with known optical properties. The aim is to decouple the diffuse reflectance ($R_d$) of the tissue from that of the light source intensity ($I_0$) and the transfer function of the instrument (${\mathrm{MTF}}_{\text{system}}$). To obtain the diffuse reflectance of the tissue, Eq. (2) was applied at each $\lambda$ and $f_x$:

The third step is the fitting for optical properties. An inverse solver approach was used to minimize errors between the measured $R_{d,\text{tissue}}f(x)$ and forward Monte Carlo simulations of $R_d(f_x)$ for specific pairs of ${\mu }_a$ and ${\mu }_s^{\prime }$. In this investigation, an iterative Nelder–Mead optimization algorithm (from the scipy.optimize library) then searches in the parameters’ space for a global minimum that minimizes the difference and gives as a result a pair of (${\mu }_a$, ${\mu }_s^{\prime }$) values.

2.5.

Optical Phantom Validation Measurements

The device performance was first tested on tissue-simulating liquid phantoms and compared to a validated SFDS system,^21,25,26 used as a reference. Sixteen phantoms were manufactured by combining four different concentrations of Intralipid^®, which was used as a scattering agent,^27,28 and four concentrations of India ink, which was used as an absorber.^27,29 The Intralipid^® concentrations were (0.5%, 1%, 1.5%, 2%), giving measured values of ${\mu }_s^{\prime }$ comprised in the range 0.5 to $2.5{\mathrm{mm}}^{-1}$ @625 nm. The India ink was first sonicated to ensure an homogeneous distribution of the pigment and was added in concentrations of (13, 27, 40, and $53\mathrm{mg}/\mathrm{L}$), giving measured absorption values in the range 0.025 to $0.25{\mathrm{mm}}^{-1}$ @625 nm. A dataset of 60 images (three phases, four spatial frequencies, and five spectral bands) was acquired on all 16 phantoms. Analogously, a dataset of 15 signals (three phases, five spatial frequencies, spanning the spectrum range 400 to 700 nm) was acquired in parallel with the SFDS system. The spatial frequencies used in the SFDS system are (0, 0.05, 0.1, 0.15, 0.2)${\mathrm{mm}}^{-1}$. Under the same experimental conditions, two datasets were also acquired for calibration with the SFDI and SFDS systems on a reference silicon phantom with known optical properties. The exposure time was adjusted for each measurement so that the measured signal would use the entire dynamic range of the camera while not causing the sensor to saturate.

2.6.

In Vivo Measurements

In vivo measurements were performed on healthy skin of a volunteer with both the SFDI and SFDS system. Written informed consent was obtained from the subject. The arm was laid on an armrest, and a dataset was acquired first with the SFDI system and afterward with the SFDS system on the forearm of the subject and on a calibration silicon phantom, like in sec. 2.5. The measurements were performed with the approval of Linköping ethics committee (Dnr 2018/282-31).

3.1.

Characterization of Spectral Bands

The nine spectral bands characterized in Sec. 2.2 are presented in Fig. 5. The naming convention for the spectral bands uses two letters: the first one is referring to the spectral band emitted by the projector (red, green, or blue) and the second one to the filter band of the camera (likewise, red, green, or blue). The solid line is the data derived from the laboratory approach (Sec. 2.2.1), whereas the dashed line is derived from the low-cost approach (Sec. 2.2.2). Based on the data in Fig. 5, two of the cross-channel bands (GB and GR) were determined to be the strongest candidates for our application in term of signal-to-noise ratio (SNR). The BG band also had an intensity level comparable to the GR band, but the absorption of blue light in tissue is very strong, which further degrades the actual SNR of the band. For this reason, we decided not to include the BG band in this study. The signal from the remaining three bands (BR, RB, and RG) was too weak to be useful in this implementation. The central wavelength (${\lambda }_0$) and full width half maximum (FWHM) of the nine bands were calculated and the main five are reported in Table 1. Since the spectral bands might have a complex shape, these values were calculated by doing a weighted average of the signals, rather than by fitting to a Gaussian function.

Fig. 5

Plot of the nine combinations of the LED and camera spectral features (three main bands on the diagonal and six cross-channel bands). The solid lines are the signals calculated with the laboratory approach, and the dashed line is calculated with the low-cost approach. The rows represent the LEDs color (from the top: blue, green, and red) while the columns represent the camera filters (from the left: blue, green, and red).

Table 1

List of the five main spectral combinations in this study, obtained with the two approaches. The values are expressed as (λ0±12FWHM). The naming convention is a letter that indicates the color of the LED followed by a letter that indicates the color of the filter.

3.2.

Optical Phantom Validation Measurements

The data were processed with the procedure describe in Sec. 2.4, obtaining a map of ${\mu }_a$, ${\mu }_s^{\prime }$ at five wavelengths with the SFDI system and the whole optical spectrum in a single central location with the SFDS system. Since the phantoms are homogeneous, an average of the optical properties over the whole FOV of the SFDI system was calculated in order to compare it with the SFDS data. Likewise, since the SFDS data are a continuous spectrum, in order to compare it with the SFDI data, spectral properties at the five wavelengths were emulated from the SFDS data using a method described in a previous publication.¹⁴ The comparison can be seen in Fig. 6, where a linear regression between the values of ${\mu }_a$, ${\mu }_s^{\prime }$ in SFDI and SFDS data is calculated. The color-coded clusters on the graph represent the four different levels of absorption (upper image) or scattering (lower image), which were measured on the phantoms. Each cluster consists of 20 data points, corresponding to the four phantoms that have the same optical property (e.g., same ${\mu }_a$, but different ${\mu }_s^{\prime }$), measured at the five spectral bands. The $R^2$ and root mean square error (RMSE) values are reported ($R^2=0.987$, $\mathrm{RMSE}=0.006$ for ${\mu }_a$, $R^2=0.994$, $\mathrm{RMSE}=0.078$ for ${\mu }_s^{\prime }$).

Fig. 6

Comparison of the measured values of (a) ${\mu }_a$ and (b) ${\mu }_s^{\prime }$ between the SFDI and the SFDS systems in the five bands, measured on optical phantoms with four levels of absorption and scattering. The dashed lines are the linear regression lines ($R^2$ in figure), the dotted lines are the unity lines.

3.3.

In Vivo Measurements

The data from the in vivo measurements were processed in a similar way at the same three spatial frequencies 0, 0.05, $0.1{\mathrm{mm}}^{-1}$ both for SFDI and SFDS. This choice of a subset of the acquired $f_x$ was made considering the layered nature of skin. This way we are sampling the same volume of tissue at each $f_x$ with both systems. The results are shown in Fig. 7. The SFDI data points (red circles) are averaged spatially over the entire image. The SFDS spectrum (dashed line) is averaged spectrally for comparison, by doing a weighted average over the five bands of interest. We obtain five data points (blue diamonds), which emulate what the SFDS system would detect over a broad band, as described in a previous publication.¹⁴

Fig. 7

Comparison of the (a) ${\mu }_a$ and (b) ${\mu }_s^{\prime }$ values measured in vivo on human forearm skin using SFDI (red circles) and SFDS (dashed line). Five data points are then emulated from the SFDS data, by doing a weighted average over the five bands (blue diamonds).