Recent Foundation Models have begun to yield remarkable successes across various downstream medical imaging applications. Yet, their potential within the context of multi-view medical image analysis remains largely unexplored. This research aims to investigate the feasibility of leveraging foundation models for predicting breast cancer from multiview mammograms through parameter-efficient transfer learning (PETL). PETL was implemented by inserting lightweight adapter modules into existing pre-trained transformer models. During model training, the parameters of the adapters were updated while the pre-trained weights of the foundation model remained fixed. To assess the model's performance, we retrospectively assembled a dataset of 949 patients, with 470 malignant cases and 479 normal or benign cases. Each patient has four mammograms obtained from two views (CC/MLO) of both the right and left breasts. The large foundation model with 328 million (M) parameters, finetuned with adapters comprising only 3.2M tunable parameters (about 1% of the total model parameters), achieved a classification accuracy of 78.9% ± 1.7%. This performance was competitive but slightly inferior to a smaller model with 36M parameters, finetuned using traditional methods, which attained an accuracy of 80.4% ± 0.9%. The results suggest that while foundation models possess considerable potential, their efficacy in medium-sized datasets and in transitioning from single-view to multi-view image analysis, particularly where reasoning feature relationships across different mammographic views is crucial, can be challenging. This underscores the need for innovative transfer learning approaches to better adapt and generalize foundation models for the complex requirements of multi-view medical image analysis.
Fourier ptychography microscopy (FPM) is a computational imaging technique that enables high resolution and large FOV simultaneously. For FPM, multiplexed LED illumination can significantly improve the efficiency of image data acquisition at the cost of deteriorated quality in the reconstructed images. In this study, we aim to evaluate the imaging quality of multiplexed FPM with different illumination configurations. For this purpose, a prototype FPM microscope was developed, which was equipped with a 4×/0.1 NA objective lens. This prototype was used to test 1 LED conventional, 2 LED multiplexed, and 4 LED multiplexed FPM illumination configurations on a standard USAF 1951 resolution target and a cytology sample. Modulation transfer function (MTF) curves were generated from the reconstructed images to quantitatively compare the performance of different LED combinations. The results demonstrated that the resolution target image reconstructed using 1 LED illumination raw images can resolve up to 912.3 lp/mm, but it decreased to 812.7 lp/mm and 724.1 lp/mm when 2 LED and 4 LED illumination were adopted, respectively. The corresponding MTF curves indicate decreased contrast on most spatial frequencies when comparing reconstructed results between multiplexed (2/4 LED) and conventional illumination configurations. Accordingly, the quality of reconstructed clinical cytology sample images decreases as the number of LEDs per image increases. However, all of them have satisfactory quality for most clinical applications. This preliminary study provides useful information to facilitate the development of multiplexed illumination FPM imaging systems in the future.
This study aims to investigate the effectiveness of a self-supervised deep learning based noise reduction algorithm at improving the detectability of phantom images acquired from the phase-sensitive breast tomosynthesis (PBT) system.
An ACR mammography phantom and three different Contrast Detail (CD) phantoms were used in experiments. Each phantom is 5cm in thickness and fabricated with materials simulating 50% glandular tissue and 50% adipose tissue. The phantoms were imaged by 59kV and 89kV with varying levels of external filtrations. The x-ray exposure was adjusted so that the average glandular dose was consistently to be 1.3 mGy throughout the imaging.
A noise reduction algorithm was applied to the images. The algorithm being evaluated is a state-of-the-art self-supervised single image denoising approach that can prioritize the preservation of fine-grained image structures while performing noise removal.
The contrast-to-noise (CNR) ratio was measured to conduct objective analysis. Additionally, an observer performance study was conducted in which observers were shown the images from each phantom in a randomized order before and after the denoising algorithm was applied. The observers rated the detectability of each image by identifying the minimum perceptible feature.
The results indicate some improvement from the objective studies; however, in the subjective observer studies, no improvement was observed in the detectability of the ACR images, and limited improvement was observed in the detectability of the CD phantom images.
Molecular ultrasound imaging is used to image the expression of specific proteins on the surface of blood vessels using the conjugated microbubbles (MBs) that can bind to the targeted proteins, which makes MBs ideal for imaging the protein expressed on blood vessels. However, how to optimally apply MBs in an ultrasound imaging system to detect and quantify the targeted protein expression needs further investigation. To address this issue, objective of this study is to investigate feasibility of developing and applying a new quantitative imaging marker to quantify the expression of protein markers on the surface of cancer cells. To obtain a numeric value proportional to the amount of MBs that bind to the target protein, a standard method for quantification of MBs is applying a destructive pulse, which bursts most of the bubbles in the region of interest. The difference between the signal intensity before and after destruction is used to measure the differential targeted enhancement (dTE). In addition, a dynamic kinetic model is applied to fit the timeintensity curves and a structural similarity model with three metrics is used to detect the differences between images. Study results show that the elevated dTE signals in images acquired from the targeted (MBTar) and isotype (MBIso) are significantly different (p<0.05). Quantitative image features are also successfully computed from the kinetic model and structural similarity model, which provide potential to identify new quantitative image markers that can more accurately differentiate the targeted microbubble status.
This study aims to investigate the impact of external filtration on image quality and exposure time of an in-line phase-contrast x-ray breast imaging prototype.
A Contrast-Detail phantom is imaged by 59 kV and 89 kV systems with a CCD camera and varying filter thicknesses, ranging from 1.0 mm to 3.3 mm of aluminum. The average glandular radiation dose is set to 1.3 mGy throughout the experiment, regardless of imaging parameters. The Contrast-Detail (CD) curves are generated from the reading results of three experienced observers. The Contrast-to-Noise-Ratio (CNR) is calculated for objective comparisons. The results show that the beam hardening with 1.3 and 2.5 mm aluminum filters in the 59 kV system provides the most desirable CNRs and CD curves, whereas a 3.3 mm aluminum might be a preferable external filtration in the 89 kV system. It can be concluded that the 59 KV beam, filtered by a 1.3 mm aluminum, is a better choice, as it results in comparable image quality and a 35% shorter exposure time. On the other hand, the 89 KV beam filtered by 3.3 mm aluminum results in higher image quality at the expense of slightly increased acquisition time. The prolonged acquisition effect on the image blurring should be evaluated in patient studies where the object is not immobile like imaging phantoms.
Ovarian carcinoma is the most lethal malignancy in all kinds of gynecologic cancers, and radiomics based image marker is an effective tool for the early-stage prediction of the chemotherapies applied on ovarian cancer patients. This investigation aims to compare and evaluate the predicting performance of the 2D and 3D radiomics features. During the experiment, the tumors were first segmented from the CT slices, based on which a total of 1032 2D radiomics features and 1595 3D radiomics features were extracted. These features are related to tumor shape, density and texture properties. Next, a least absolute shrinkage and selection operator (LASSO) feature selection method was adopted to determine optimal features clusters for 2D and 3D feature pools respectively, which were used as the input of support vector machine (SVM) based prediction models. During the experiment, a total of 99 cases were selected from a previously established dataset at our medical center. The model performance was assessed by receiver operating characteristic (ROC) curve. The results indicated that the 2D and 3D feature based models achieved an area under the curve (AUC) of 0.85±0.03 and 0.89±0.02, while the overall accuracies were 0.76 and 0.81 respectively. These results indicate that the overall performance of the 3D feature is higher than the 2D features. But the sensitivity of the 2D model is higher at some certain specificity range. This study initially reveals the difference between the 2D and 3D features, which should be meaningful for the optimization of the radiomics based clinical decision support tools.
Metaphase chromosome karyotyping plays an important role in the diagnosis of certain cancers and some genetic diseases by detecting chromosome abnormalities. For this technique, high magnification objective lens is used to ensure the chromosome’s band pattern sharpness, but the small field of view (FOV) of the lens makes the imaging of chromosomes very tedious and time consuming. The purpose of this study is to verify the use of the Fourier ptychography microscopy (FPM) system in high-resolution karyotyping. Based on our former study, we further expanded the theoretical NA of the FPM system to 1.11 with a 20×/0.4 NA objective lens and higher illumination angles. To evaluate the resolving power of the FPM system, a 1951 USAF resolution target was imaged to create the modulation transfer function (MTF) curves. The performance of the FPM system was also assessed by imaging chromosomes acquired from blood and bone marrow pathological samples. The results were compared with a conventional 100×/1.45 NA oil immersion objective lens. The MTF curves demonstrate that the contrast of the FPM system is inferior but close to the 100× objective lens (1.45 NA). As compared to the images acquired by the 100×/1.45 NA oil immersion objective lens, the chromosome images recovered by the FPM system contain all the band patterns, despite the loss of some fine details. This study initially verified that the high NA FPM system can guarantee the sharpness of chromosome band patterns as the conventional high magnification oil immersion objective lens, while enabling a large FOV without the utilization of oil immersion medium.
The study aims to develop a novel computer-aided diagnosis (CAD) scheme for mammographic breast mass classification using semi-supervised learning. Although supervised deep learning has achieved huge success across various medical image analysis tasks, its success relies on large amounts of high-quality annotations, which can be challenging to acquire in practice. To overcome this limitation, we propose employing a semi-supervised method, i.e., virtual adversarial training (VAT), to leverage and learn useful information underlying in unlabeled data for better classification of breast masses. Accordingly, our VAT-based models have two types of losses, namely supervised and virtual adversarial losses. The former loss acts as in supervised classification, while the latter loss aims at enhancing the model’s robustness against virtual adversarial perturbation, thus improving model generalizability. To evaluate the performance of our VAT-based CAD scheme, we retrospectively assembled a total of 1024 breast mass images, with equal number of benign and malignant masses. A large CNN and a small CNN were used in this investigation, and both were trained with and without the adversarial loss. When the labeled ratios were 40% and 80%, VAT-based CNNs delivered the highest classification accuracy of 0.740±0.015 and 0.760±0.015, respectively. The experimental results suggest that the VAT-based CAD scheme can effectively utilize meaningful knowledge from unlabeled data to better classify mammographic breast mass images.
Significance: Searching analyzable metaphase chromosomes is a critical step for the diagnosis and treatment of leukemia patients, and the searching efficiency is limited by the difficulty that the conventional microscopic systems have in simultaneously achieving high resolution and a large field of view (FOV). However, this challenge can be addressed by Fourier ptychography microscopy (FPM) technology.
Aim: The purpose of this study is to investigate the feasibility of utilizing FPM to reconstruct high-resolution chromosome images.
Approach: An experimental FPM prototype, which was equipped with 4 × / 0.1 NA or 10 × / 0.25 NA objective lenses to achieve a theoretical equivalent NA of 0.48 and 0.63, respectively, was developed. Under these configurations, we first generated the system modulation transfer function (MTF) curves to assess the resolving power. Next, a group of analyzable metaphase chromosomes were imaged by the FPM system, which were acquired from the peripheral blood samples of the leukemia patients. The chromosome feature qualities were evaluated and compared with the results accomplished by the corresponding conventional microscopes.
Results: The MTF curve results indicate that the resolving power of the 4 × / 0.1 NA FPM system is equivalent and comparable to the 20 × / 0.4 NA conventional microscope, whereas the performance of the 10 × / 0.25 NA FPM system is close to the 60 × / 0.95 NA conventional microscope. When imaging the chromosomes, the feature qualities of the 4 × / 0.1 NA FPM system are comparable to the results under the conventional 20 × / 0.4 NA lens, whereas the feature qualities of the 10 × / 0.25 NA FPM system are better than the conventional 60 × / 0.95 NA lens and comparable to the conventional 100 × / 1.25 NA lens.
Conclusions: This study initially verified that it is feasible to utilize FPM to develop a high-resolution and wide-field chromosome sample scanner.
The purpose of this study is to develop a novel computer-aided diagnosis (CAD) scheme to facilitate breast mass classification, which is based on the latest transferring generative adversarial networks (GAN) technology. Although GAN is one of the most popular techniques for image augmentation, it requires a relatively large original dataset to achieve satisfactory results, which may not be available for most of the medical imaging tasks. To address this challenge, we developed a novel transferring GAN, which was built based on the deep convolutional generative adversarial networks (DCGAN). This novel model was first pre-trained on a dataset of non-mass mammogram patches. Then the generator and the discriminator were fine-tuned on the mass dataset. A supervised loss was integrated with the discriminator, such that it can be used to directly classify the benign/malignant masses. We retrospectively assembled a total of 25,000 non-mass patches and 1024 mass images to assess this model, using classification accuracy and receiver operating characteristic (ROC) curve. The results demonstrated that our proposed approach improved the accuracy and area under the ROC curve (AUC) by 6.0% and 3.5% respectively, when compared with the classifiers trained without conventional data augmentation. This investigation may provide a new perspective for researchers to effectively train the GAN models on medical imaging tasks with limited datasets.
This study aims to utilize the primary tumor characteristics from CT images to detect lymph node (LN) metastasis for accurately categorizing locally advanced cervical cancer patients (LACC). In clinical practice, LN metastasis is a critical indicator for patients’ prognostic assessment, which is usually investigated by PET/CT (i.e., positron emission tomography/computed tomography) examination. However, the high cost of the PET/CT imaging modality limits its application and also leads to heavy financial burden on patients. Thus it is clinically imperative to develop an economic solution for the LN metastasis identification. For this purpose, a novel image marker was developed, which is based on the primary cervical tumors segmented from CT images. Accordingly, a total of 99 handcrafted features were computed, and an optimal feature set was determined by Laplacian Score (LS) method. Next, a logistic regression model was applied on the optimal feature set to generate a likelihood score for the identification of LN metastasis. Using a retrospective dataset that contains a total of 82 LACC patients, this new model was trained and optimized by leave one out cross validation (LOOCV) strategy. The marker performance was assessed by receiver operator characteristic curve (ROC). The results indicate that the area under the ROC curve (AUC) of this identification model was 0.774±0.050, which demonstrates its strong discriminative power. This study may be able to provide gynecologic oncologists a CT image based low cost clinical marker to identify LN metastasis occurred on LACC patients.
Automatic classification of epithelium and stroma regions on histopathological images is critically important in digital pathology. Although many studies have been conducted in this research area, few investigations have been focused on model generalizability between different types of tissue samples. The objective of this study is to initially verify the classification effectiveness of a sufficiently optimized transfer model. Accordingly, two datasets were assembled, which contain 157 breast cancer images (Dataset I) and 11 ovarian cancer images (Dataset II), respectively. A computer aided detection (CAD) scheme was developed for this classification task. The scheme first divided each image into small regions of interest (ROI) containing only epithelium or stroma tissues, using multi-resolution super-pixel algorithm. Then, a total of 26 quantitative features were computed for each ROI, which were used as the input of five different machine learning classifiers, namely, linear support vector machine (SVM), linear discriminant analysis (LDA), logistic regression, decision tree and k-nearest neighbors (KNN). The scheme was trained and optimized on Dataset I, and five-fold cross validation strategy was utilized for performance evaluation. After the scheme was sufficiently optimized on Dataset I, it was applied “as is” on dataset II. The results of the breast cancer dataset show that linear SVM achieved the highest classification accuracy of 0.910. When applied on the 11 ovarian cancer cases (Dataset II), the SVM model achieved an average classification accuracy of 0.744. This preliminary study initially demonstrates the model transfer performance for epithelium-stroma classification task.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.