|
Manual image segmentations are naturally subject to inaccuracies related to systematic errors (due to the tools used, eye-hand coordination, etc.). This was noted earlier when a simplified accuracy scale was proposed [1]. This scale arbitrarily divides a given range of values of the Kappa measurement parameter into classes: almost perfect (>0.80), substantial (0.61 - 0.80), moderate (0.41 - 0.60), fair (0.21 - 0.40), slight (0.00 - 0.21) and poor (< 0.00). However, the determination of threshold values between classes is not entirely clear and seems to be application-dependent. This is particularly important for images in which the tumor-normal tissue boundary can be very indistinct, as is observed in ultrasound imaging of the most common cancer in women - breast cancer [2]. In machine learning, there is an ongoing contest over the values of performance indicators obtained from new neural network architecture without accounting for any ground truth bias. This raises the question of what relevance, from a segmentation quality point of view, a gain at the level of single percentages has [3] if the references have much greater uncertainty. So far, research on this topic has been limited. The relationship between the segmentations of breast tumors on ultrasound images provided by three radiologists and those obtained using deep learning model has been studied in [4]. Unfortunately, the indicated segmentation contour sometimes varied widely in all three cases. A cursory analysis by multiple physicians, which focused only on the Kappa coefficient in the context of physicians’ BI-RADS category assignments, was conducted in the [5]. In this article, we present a preliminary analysis of the accuracy of experts’ manually prepared binary breast cancer masks on ultrasound images and their impact on performance metrics commonly used in machine learning. In addition, we examined how tumor type or BI-RADS category [6] affects the accuracy of tumor contouring.
|
