1.

Introduction

Infection following trauma is one of the most prevalent and challenging complications faced by orthopedic surgeons. Inadequate tissue perfusion plays a critical role in this complication since poorly perfused bone can be a nidus for bacterial biofilm formation creating resistance to antibiotics.^1,2 These biofilms make treatment of infections difficult as they increase the bacteria’s ability to evade the immune system, share antibiotic resistant genes, and proliferate additional planktonic bacteria.³ Because of this, management of open fractures and fracture-related infection relies on aggressive and thorough debridement in an effort to remove all poorly perfused bone.^4,5 However, there is a limited number of imaging tools to objectively inform bone perfusion and guide debridement.

In contrast to other clinical imaging modalities, such as dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI)^6,7 and combined positron emission tomography/computed tomography (PET/CT),⁸ indocyanine green (ICG)-based dynamic contrast-enhanced fluorescence imaging (DCE-FI) presents an intraoperative solution. Unlike these other methods, which cannot be commonly utilized in the operating room, DCE-FI has demonstrated an ability to objectively assess bone perfusion during the surgery and provide surgical guidance for the debridement of poorly perfused tissues.^9–11

Although DCE-FI shows both feasibility and ease of utilization in the surgical setting, motion artifacts resulting from patient’s involuntary respiration and other mechanical disturbances during imaging data acquisition can potentially lead to errors in quantitative bone perfusion assessment. Previous studies have utilized marker tracking and edge detection for automated motion compensation in fluorescence imaging.^12,13 Nevertheless, these methods may not be suitable for DCE-FI during orthopedic surgery as marker tracking relies on additional hardware support, and edge detection can be highly sensitive to complex noises introduced by intraoperative interference.^14–16 In contrast, mutual information (MI)¹⁷ can be implemented for retrospective correction and has shown favorable robustness to intraoperative interference as it evaluates image similarity based on whole image intensities instead of specific landmarks.^18–20

While MI offers a better solution for reducing the motion artifact in intraoperative fluorescence imaging, the conventional MI calculation that using a fixed reference may fall short for DCE-FI in intraoperative orthopedic imaging. This is due to the extensive heterogeneity of fluorescence intensities arising from bone’s deep three-dimensional structure and coupled with the dynamic changes in perfusion levels that occur across the imaging field over time. To overcome this limitation, a frame-by-frame MI-based motion correction approach for DCE-FI has been developed and validated using tissue phantoms and 45 intraoperative imaging sessions on 15 amputation cases. The results of the motion correction demonstrated a significant reduction in motion artifacts. The alignment of image frames at different time points within each imaging session resulted in improved kinetic curves and enhanced perfusion-related parameters for identifying bone perfusion levels.

2.

Materials and Methods

2.1.

Imaging Acquisition

Figure 1 presents the fluorescence imaging system and the imaging setup for our fluorescence imaging guide during open orthopedic surgery. Near-infrared (NIR) lights were used to excite ICG injected intravenously, thereby enabling intraoperative assessment of bone perfusion. The image data were acquired using a SPY Elite Fluorescence Imaging System [Fig. 1(a)] (Stryker, Kalamazoo, MI). The system was positioned at a 30-cm working distance from the surgical field and utilized 805 nm NIR light for ICG excitation. A charge-coupled device camera with an 820 to 900 nm bandpass filter was employed for fluorescence light imaging. Additionally, to capture white-light images, an RGB camera was connected to the system via a beam splitter [Fig. 1(b)]. This setup allowed simultaneous capture of both fluorescence and white-light images.

Fig. 1

(a) Fluorescence imaging system and (b) imaging setup for open orthopedic surgery.

During each imaging session, a white-light image was captured first, followed by a fluorescence video recorded at a rate of 3.75 frames per second for 1024 frames (maximum frame number). Each video frame has an $18\times 13.5\mathrm{cm}$ field of view (FOV) with $1024\times 768\text{pixels}$ and a bit depth of 8. Further system specifications were described in previous publications.^10,11,21,22

2.3.

Phantom Validation

To verify the efficacy of our MI-based motion correction in mitigating the mechanical disturbances from the imaging device, we conducted validation experiments using an ICG-equivalent phantom (QUEL Imaging, White River Junction, VT).²⁴ As shown in Fig. 2(a), this phantom containing nine 8-mm wells filled with varying ICG equivalent concentrations, ranging from $1\mu \mathrm{M}$ (top left) to 0 (bottom right). To simulate typical mechanical disturbances, we induced random vibrations in the imaging head of the SPY Elite Fluorescence Imaging System by manually tapping the articulating arm [Fig. 1(a)]. The imaging FOV underwent oscillations for several cycles before returning to a stationary status. Fluorescence imaging of the phantom was conducted over a 90-s duration, with manual tapping at 0, 30, and 60 s, ensuring sufficient time for a vibration to complete. The tapping intensity was varied to assess the power of the correction against different levels of mechanical vibrations.

Fig. 2

Photos, images, and a schematic diagram of the image analysis and display using a phantom as an example. (a) Photo and (b) fluorescence image of the phantom. Nine 8-mm wells are filled with silicon material with various ICG-equivalent concentrations ranging from $1\mu \mathrm{M}$ (top left) to 0 (bottom right). (c) Overlaid image of two fluorescence frames with a 1 mm horizontal displacement for 300 nM (top) and 100 nM wells (bottom), with intensity differences highlighted by green and magenta. (d) Schematic diagram of the kinetic parameters for assessing bone perfusion.

To further examine the correction’s capability in addressing the motions due to the patient’s involuntary respiration, we conducted another validation study using the ICG-equivalent phantom placed on the lower leg of a normal subject in a supine position on a table. The subject was instructed to contract and relax leg muscles to mimic breathing motions observed in intraoperative images obtained from patient cases.

The intensity variations due to the motion artifacts in the images were assessed. The areas inside the 100 and 300 nM wells were designated as the regions of interest (ROIs), from which the average ICG intensity of the circular ROI was extracted before and after the correction.

3.

Results

Figure 3 demonstrates the efficacy of motion correction based on MI for mitigating motion artifacts caused by mechanical disturbances and patient respiration. Given the partial fluorescence saturation in the $1\mu \mathrm{M}$ phantom well, our analysis concentrates on the 300 and 100 nM wells, where the fluorescence intensities align with the ranges observed in bone during our clinical studies. Panels (a) and (b), as well as (c) and (d), display the mean intensity fluctuations due to mechanical disturbances [Figs. 3(a) and 3(b)] and simulated respiration [Figs. 3(c) and 3(d)], respectively, within the ROIs of the 300 and 100 nM wells. The blue and red curves represent the intensity fluctuations pre- and postcorrection, respectively. As shown in Figs. 3(a) and 3(b), the high-frequency intensity fluctuations in 300 nM well caused by mechanical disturbances were reduced by 93%, from 31 RFUs down to below 2 RFUs [Fig. 3(a)], and the fluctuations in the 100 nM well were reduced by 91%, from 11 RFUs to under 1 RFU [Fig. 3(b)]. The motion correction also significantly lowered the intensity fluctuations due to respiration—from a peak of 45 RFUs to 11 RFUs (a 76% reduction) for the 300 nM well [Fig. 3(c)], and from 16 RFUs to 4 RFUs (a 75% reduction) for the 100 nM well [Fig. 3(d)].

Fig. 3

Phantom study results demonstrating motion artifact reduction from mechanical disturbances and simulated respiration. (a) and (b) The average intensity fluctuations within the ROI for the 300 and 100 nM wells, respectively, before (blue) and after (red) applying the motion correction. (c) and (d) The changes in average intensity due to simulated respiratory artifacts within the ROI for the 300 and 100 nM wells, respectively, also before (blue) and after (red) correction.

Figure 4 displays the overlay of two tibial fluorescence images captured at 150 and 250 s, respectively. The patient, a 53-year-old male, underwent amputation and an osteotomy at the tibial diaphysis. The fluorescence intensity of the two images is color-coded in green (150 s) and purple (250 s) for comparison before [Fig. 4(a)] and after correction with fixed-reference [Fig. 4(b)] or frame-by-frame reference [Fig. 4(c)]. As indicated in the corresponding zoom-in images of the yellow-box areas in Figs. 4(a)–4(c), the initial mismatch of $\sim 2.5\mathrm{mm}$ in the boundaries of the bone and soft tissue is observed before the correction [Fig. 4(d)]. After the fixed-reference correction, the images were overcorrected with a 2 mm mismatch [Fig. 4(e)]. However, the frame-by-frame correction significantly improved the alignment of the two images [Fig. 4(f)].

Fig. 4

An example of applying MI-based motion correction to a patient’s DCE-FI. (a)–(c) The overlay of two frames captured at 150 and 250 s after the start of imaging. The green and purple colors represent the images captured at 150 and 250 s, respectively. (a) Before correction, (b) after fixed-reference correction, and (c) after frame-by-frame correction. (d)–(e) Zoom-in images of the yellow dashed-line areas indicated in panels (a)–(c), respectively.

Figure 5 demonstrates the joint histograms of the two frames in Fig. 4 before and after MI-based corrections with fixed-reference and frame-by-frame methods. Following the correction, the grayscale intensity distribution becomes concentrated toward the diagonal line, indicating a reduction in joint entropy and an increase in marginal entropy, resulting in higher MI values. Indeed, compared to the MI value of 1.66 before the correction [Fig. 5(a)], the MI values of 1.94 [Fig. 5(b)] and 2.05 [Fig. 5(c)] after correction indicate a significant reduction in motion artifacts. Additionally, the frame-by-frame reference approach shows better performance compared to the fixed-reference approach.

Fig. 5

Joint histograms of the two frames in Fig. 4: (a) before correction, (b) after fixed-reference correction, and (c) frame-by-frame correction.

Figure 6 presents the kinetic curves before and after frame-by-frame motion correction on DCE-FI of the same patient case discussed above. The white-light image exhibits an osteotomy at the middle of the tibia [Fig. 6(a)]. A fluorescence image captured at one minute after ICG injection is superimposed on a white-light image [Fig. 6(b)]. Four circular ROIs with a 10 mm diameter—labeled p1 (magenta) and p2 (green) for the proximal region toward the head, and d1 (yellow) and d2 (blue) for the distal region toward the foot—were positioned at equidistant points along the axis from the feet to the head, spanning the entire imaging FOV of the tibias. The kinetic curves corresponding to these ROIs are presented in Fig. 6(c), color-coded to match the ROIs. The dashed and solid lines represent kinetic curves before and after the motion correction, respectively. The motion-induced intensity surges were smoothed out, and the general trend of ICG bolus wash-in and wash-out was restored after correction.

Fig. 6

MI-based motion correction improved the kinetic curves: (a) white-light image of the surgical view; (b) with fluorescence image overlaid and four circular ROIs drawn, two on the proximal region ($p1$, $p2$) and two on the distal region ($d1$, $d2$); and (c) kinetic curves of the four ROIs in corresponding colors, where dashed and solid lines represent kinetic curves before and after the correction, respectively.

Figure 7 presents the kinetic parameters extracted from the four ROIs in Fig. 6(b) before and after the frame-by-frame correction. Due to the osteotomy damaging endosteal blood supply in distal region, perfusion-related kinetic parameters ($I_{\max }$, IS, and TTP) in ROIs of $d1$ and $d2$ were expected to be significantly reduced ($I_{\max }$ and IS) or increased (TTP), compared to those in ROIs of $p1$ and $p2$. However, as shown in Fig. 7, before correction, these differences were not evident (left portion of the bar graph in each subfigure). In contrast, after the correction, $I_{\max }$ and IS in $p1$ and $p2$ (solid line) were higher than those in $d1$ and $d2$ (dashed line), whereas proximal TTPs became much lower than those in distal ROIs. These drastic changes in perfusion-related parameters demonstrate a lower perfusion in the distal regions after the motion correction, indicating the successful restoration of kinetic parameters.

Fig. 7

MI-based motion correction improved the perfusion-related parameters before and after the correction, in each of the proximal and distal circular ROI shown in Fig. 6(b): (a) average $I_{\max }$, (b) average IS, and (c) average TTP.

The automated MI-based motion correction was applied to all images in 45 imaging sessions (baseline, osteotomy, and circumferential periosteal/soft tissue stripping) of 15 amputation cases. Figure 8 shows the total corrected displacements calculated by summing the absolute values of correction between each frame in $x$- and $y$-axes of each imaging session. The range of total displacements corrected varied from 0 to 23.9 mm [mean: 4.3 mm; standard deviation (SD): 5.3 mm] and 0 to 50.8 mm (mean: 9.6 mm; SD: 13.8 mm) along the $x$- and $y$-axes, respectively. The average displacement in $y$-axis was more than double of that in $x$-axis, indicating more substantial motion in $y$-direction in this amputation cohort.

Fig. 8

The total corrected displacements of each imaging session in (a) $x$- and (b) $y$-axes.

4.

Discussion

In addition to clinical imaging modalities such as DCE-MRI^6,7 and PET/CT,⁸ other imaging modalities such as photoacoustic imaging,^25,26 laser speckle contrast imaging,²⁷ and laser Doppler flowmetry²⁸ have been adapted for in vivo blood flow assessment across various medical applications. Photoacoustic imaging stands out by providing superior surface spatial resolution and capturing angiographic images of tissue. However, it relies on a high-power, short-pulse, and wavelength-tunable laser source to irradiate tissue, and it uses multiple ultrasound transducers to detect the resulting pressure waves. Nevertheless, this approach has limitations related to imaging depth, laser power safety considerations, and the complexities and costs associated with solid-state laser systems, which hinder its widespread clinical adoption for assessing bone perfusion in real patients.²⁹ Additionally, its imaging accuracy can be uncertain because the ultrasound signal is affected by the complexities of light absorption and scattering under multiwavelength illumination.²⁹ Laser speckle contrast imaging, on the other hand, can effectively capture relative blood flow in relatively clean and flat tissue surfaces based on speckle pattern analysis. However, it faces challenges when applied to intraoperative orthopedic imaging due to the surface roughness and deep 3D curvature of bones during surgery. Laser Doppler flowmetry is an invasive technique that provides blood flow information at specific location, but it does not be suitable in conditions where contact with the study area is avoided such as surgical interventions.³⁰ In contrast, as shown in this study, although ICG-based DCE-FI relies on external contrast agents to generate useful images, it can capture the varies blood perfusion level of the human bone due to the different levels of the damage intraoperatively, to realize the surgical guidance for orthopedic surgery.

Various techniques have been developed to remove motion artifact in dynamic imaging,^31,32 including normalized MI, which reduces the impact of overlapping areas,^33–35 and regional MI, which integrates spatial information of pixel intensity distribution.^23,36,37 Comparing these approaches, our simple frame-by-frame MI-based approach developed in this study is particularly advantageous for intraoperative DCE-FI in orthopedic surgery due to several reasons. First, it demonstrates robustness to clinical interferences and eliminates the need for any additional hardware or manual selection. Second, it effectively addresses the high heterogeneity of fluorescence intensity across the entire bone area within the imaging FOV, resulting in improved image processing stability.

Due to the large number of imaging frames that require processing for each case, and the limited processing time allowed intraoperatively, we have made some assumptions and simplifications to reduce processing time while maintaining effectiveness. Given the rigid nature of bone, we assumed that the corrections can be achieved through rigid transformation. Additionally, we assume that all motions occurred at the same depth and were limited to planar translations, neglecting minor motions such as rotation and movements in depth for this application. Moreover, temporal down-sampling at an appropriate level was employed to accelerate the processing time and reduce accumulated errors in the frame-by-frame registration process.

For the analysis presented in Figs. 6 and 7, ROIs were designated to optimally represent perfusion across the entire tibial region of each patient under varying bone conditions while ensuring methodological consistency and avoiding a biased selection that would only showcase the most significant or dramatic improvements. The imaging FOV of the tibia was evenly divided into four sections from the feet toward the head, with a 10 mm diameter ROI centrally located in each section. Two ROIs were placed on the distal end and two on the proximal end, respectively. This arrangement accounts for the average coronal dimensions of the tibia and helps to mitigate potential edge artifacts. These ROIs are indicative of the system’s general performance in capturing key perfusion metrics, such as $I_{\max }$, IS, and TTP. Additionally, we extracted and compared the average time-intensity curves of the tibia’s entire proximal and distal halves, both pre- and postcorrection. The trends observed corroborate the improvements postcorrection, similar to those depicted in Fig. 6(c). The utilization of four smaller ROIs, as opposed to two larger ones, yielded more precise insights into the correction’s efficacy across varying perfusion levels within the tibia.

As observed in Fig. 8, the corrected displacements were significantly reduced in the later cases (4 to 15) compared to the early cases (1 to 3). This improvement can be attributed to the enhanced surgical techniques adopted in subsequent cases. Specifically, surgeons took greater care to avoid nonessential clinical interventions during imaging sessions, such as minimizing movement of retractors and carefully suctioning away blood from the surface with minimal tissue contact.

In Fig. 3, we assess the effectiveness of MI-based motion correction for minimizing motion artifacts caused by mechanical disturbances and patient respiration. The analysis focused on wells with 100 and 300 nM ICG-equivalent concentrations, selected based on their fluorescence intensities matching those observed in the tibia via the SPY Elite Fluorescence Imaging System and correlating with the extent of bone damage. Figure 9 presents a comparison of ICG intensity in the tibial bone area of 15 patient’s postlower extremity amputation against that in phantom wells with equivalent ICG concentrations. Figure 9(a) shows the average histogram distributions across various imaging sessions: baseline (top), after osteotomy (middle), and following circumferential periosteal/soft tissue stripping (bottom), with 90% confidence intervals shaded. The diagonal lines mark the area within the modified full width at half maximum (mFWHM)³⁸ of the average histograms. As depicted in Fig. 9(a), the ICG intensities within the mFWHM range from 18 to 176 RFUs. Using the calibration curve of fluorescence intensity versus ICG-equivalent concentrations from the validation phantom in Fig. 9(b), these ICG intensity ranges correspond to concentrations between 49 and 540 nM.

Fig. 9

Comparative analysis of ICG intensity in the tibial bone region of 15 patients’ postlower extremity amputation against phantom wells with ICG-equivalent concentrations. (a) Average histogram distributions from multiple imaging sessions display baseline (top), postosteotomy (middle), and postcircumferential periosteal/soft tissue stripping (bottom) measurements, with the 90% confidence intervals shaded. Diagonal lines highlight the area within the mFWHM of the average histograms. (b) Relationship between fluorescence intensity and ICG-equivalent concentrations for the validation phantom. The shaded region delineates the range of ICG intensity corresponding to mFWHM of the average histograms and associated concentrations.

To illustrate our approach, we focused on case 2, where a significant amount of motion artifacts was observed. The comparison of boundary alignment of bone to soft tissue (Fig. 4) provides a straightforward and compelling demonstration of the improvement in imaging quality.

Furthermore, the changes in joint histograms before and after the correction (Fig. 5) demonstrate the efficacy of the MI-based approach, especially with the frame-by-frame reference approach.

Quantitative analysis results in Figs. 6 and 7 revealed that the kinetic behaviors of tibial tissues were restored after the removal of motion artifacts. Although there are no standard perfusion references corresponding to the damages caused by osteotomy or soft tissue stripping, we assumed that the distal area would be less perfused than the proximal area after osteotomy. This assumption is based on the original higher vascularization in the proximal half of the tibia,³⁹ and the subsequent reduction in distal perfusion due to the osteotomy, which cut off both periosteal and endosteal blood supply toward the distal region. The precorrection levels of perfusion indicated the significance of motion artifact removal in DCE-FI to achieve intraoperative and objective perfusion assessments.

For our phantom validations, we simulated the two most common types of motion encountered in intraoperative DCE-FI imaging during orthopedic surgery, namely, mechanical disturbances and breathing motions. While our imaging system operates at a video frame rate of 3.75 Hz, which effectively captures breathing artifacts at approximately 0.3 Hz, it falls short in adequately capturing high-frequency disruptive mechanical motions that can occur within a single frame. Additionally, even though our correction process is applied on a frame-by-frame basis and incorporates permanent displacements as the cumulative sum of interframe displacements, it is essential to acknowledge that as these displacements accumulate during the correction process, significant permanent displacement can lead to a portion of the bone area shifting out of the FOV, potentially causing the loss of information in that area due to the correction process. Nonetheless, the results presented in Fig. 3 demonstrate that our method effectively mitigates high-frequency artifacts to an acceptable level.

Our optimization involves searching for the maximum MI using a kernel of $3\times 3\text{pixels}$ instead of an exhaustive grid search. With this optimized correction between two frames, the registration can be achieved within seconds. However, since the approach needs to be implemented frame-by-frame and each dataset contains 1024 frames, the total processing time for an entire dataset is $\sim 20\min$. Reflecting on our previous success with utilizing parallel processing to markedly reduce DCE-FI processing times from 20 min in Matlab to under a minute using Python’s multiprocessing package, applying a similar strategy to MI calculations will enable the real-time image registration feedback. This advancement could potentially cut down the time required to provide image guidance in orthopedic surgeries to a few minutes.^40,41

5.

Conclusion

The results from this study demonstrate that automatic MI-based correction reduces motion artifact during the DCE-FI imaging data acquisition and significantly improve the quantitative assessments of bone perfusion.