Medical image classification poses significant challenges in real-world scenarios. One major obstacle is the scarcity of labelled training data, which hampers the performance of image-classification algorithms and generalisation. Gathering sufficient labelled data is often difficult and time-consuming in the medical domain, but deep learning (DL) has shown remarkable performance, although it typically requires a large amount of labelled data to achieve optimal results. Transfer learning (TL) has played a pivotal role in reducing the time, cost, and need for a large number of labelled images. This paper presents a novel TL approach that aims to overcome the limitations and disadvantages of TL that are characteristic of an ImageNet dataset, which belongs to a different domain. Our proposed TL approach involves training DL models on numerous medical images that are similar to the target dataset. These models were then fine-tuned using a small set of annotated medical images to leverage the knowledge gained from the pre-training phase. We specifically focused on medical X-ray imaging scenarios that involve the humerus and wrist from the musculoskeletal radiographs (MURA) dataset. Both of these tasks face significant challenges regarding accurate classification. The models trained with the proposed TL were used to extract features and were subsequently fused to train several machine learning (ML) classifiers. We combined these diverse features to represent various relevant characteristics in a comprehensive way. Through extensive evaluation, our proposed TL and feature-fusion approach using ML classifiers achieved remarkable results. For the classification of the humerus, we achieved an accuracy of 87.85%, an F1-score of 87.63%, and a Cohen’s Kappa coefficient of 75.69%. For wrist classification, our approach achieved an accuracy of 85.58%, an F1-score of 82.70%, and a Cohen’s Kappa coefficient of 70.46%. The results demonstrated that the models trained using our proposed TL approach outperformed those trained with ImageNet TL. We employed visualisation techniques to further validate these findings, including a gradient-based class activation heat map (Grad-CAM) and locally interpretable model-independent explanations (LIME). These visualisation tools provided additional evidence to support the superior accuracy of models trained with our proposed TL approach compared to those trained with ImageNet TL. Furthermore, our proposed TL approach exhibited greater robustness in various experiments compared to ImageNet TL. Importantly, the proposed TL approach and the feature-fusion technique are not limited to specific tasks. They can be applied to various medical image applications, thus extending their utility and potential impact. To demonstrate the concept of reusability, a computed tomography (CT) case was adopted. The results obtained from the proposed method showed improvements.
医学影像分类在现实场景中面临重大挑战。主要障碍之一是标注训练数据的稀缺，这制约了影像分类算法的性能与泛化能力。在医学领域，获取充足的标注数据往往困难且耗时，而深度学习虽展现出卓越性能，但通常需要大量标注数据才能达到最佳效果。迁移学习在减少时间、成本及对大量标注图像需求方面发挥了关键作用。本文提出一种新颖的迁移学习方法，旨在克服基于ImageNet数据集（该数据集属于不同领域）的迁移学习所特有的局限性。我们提出的迁移学习方法包括在大量与目标数据集相似的医学图像上训练深度学习模型，随后使用少量标注医学图像进行微调，以利用预训练阶段获得的知识。我们特别关注涉及肌肉骨骼放射影像（MURA）数据集中肱骨和手腕的医学X射线成像场景，这两类任务在精确分类方面均面临显著挑战。采用所提迁移学习方法训练的模型被用于特征提取，并通过特征融合训练多个机器学习分类器。我们整合这些多样化特征，以全面呈现各类相关特性。经广泛评估，我们提出的迁移学习方法与基于机器学习分类器的特征融合策略取得了显著成果：在肱骨分类中实现87.85%准确率、87.63% F1分数和75.69%科恩卡帕系数；手腕分类中获得85.58%准确率、82.70% F1分数和70.46%科恩卡帕系数。结果表明，采用本迁移学习方法训练的模型性能优于基于ImageNet的迁移学习模型。我们进一步通过可视化技术验证这些发现，包括基于梯度的类别激活热图（Grad-CAM）和局部可解释模型无关解释（LIME）。这些可视化工具为所提迁移学习方法相比ImageNet迁移学习具有更高准确性提供了佐证。此外，在各种实验中，本迁移学习方法比ImageNet迁移学习表现出更强鲁棒性。重要的是，所提迁移学习方法与特征融合技术不局限于特定任务，可应用于多种医学影像场景，从而扩展其效用与潜在影响。为验证可重用性概念，我们采用计算机断层扫描（CT）案例进行验证，该方法所得结果显示出性能提升。
Deep Transfer Learning with Enhanced Feature Fusion for Detection of Abnormalities in X-ray Images
肿瘤(癌症)患者之家