Downloads

https://doi.org/10.53941/aim.2024.100004
Zhang, Y., Jiang, N., Xie, Z., Cao, J., & Teng, Y. Ultrasonic Image’s Annotation Removal: A Self-supervised Noise2Noise Approach. AI Medicine. 2024. doi: https://doi.org/10.53941/aim.2024.100004
Forthcoming Issue
Copyright (c) 2024 by the authors.
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Article

Ultrasonic Image’s Annotation Removal: A Self-supervised Noise2Noise Approach

Yuanheng Zhang 1, Nan Jiang 2, Zhaoheng Xie 3, Junying Cao 2,*, and Yueyang Teng 1,*

1   College of Medicine and Biological Information Engineering, Northeastern University, Shenyang 110016, China
2   The Department of Ultrasound, General Hospital of Northern Theater Command, Shenyang 110169, China
3   The Institute of Medical Technology, Peking University, Beijing 100191, China
*  Correspondence: shenzongchaosheng@163.com (J.C.); tengyy@bmie.neu.edu.cn (Y.T.)

Received: 11 March 2024; Revised: 25 May 2024; Accepted: 28 May 2024; Published: 17 July 2024

 

Abstract: Accurately annotated ultrasonic images are vital components of a high-quality medical report. Hospitals often have strict guidelines on the types of annotations that should appear on imaging results. However, manually inspecting these images can be a cumbersome task. While a neural network could potentially automate the process, training such a model typically requires a dataset of paired input and target images, which in turn involves significant human labor. This study introduces an automated approach for detecting annotations in images. This is achieved by treating the annotations as noise, creating a self-supervised pretext task and using a model trained under the Noise2Noise scheme to restore the image to a clean state. We tested a variety of model structures on the denoising task against different types of annotation, including body marker annotation, radial line annotation, etc. Our results demonstrate that most models trained under the Noise2Noise scheme outperformed their counterparts trained with noisy-clean data pairs. The costumed U-Net yielded the most optimal outcome on the body marker annotation dataset, with high scores on segmentation precision and reconstruction similarity. Our approach streamlines the laborious task of manually quality-controlling ultrasound scans, with minimal human labor involved, making the quality control process efficient and scalable.

Keywords:

image restoration Noise2Noise segmentation U-Net ultrasonic

References

  1. Kulshrestha, A.; Singh, J. Inter-hospital and intra-hospital patient transfer: Recent concepts. Indian J. Anaesth. 2016, 60, 451–457. DOI: https://doi.org/10.4103/0019-5049.186012
  2. Jackson, P.; Chenal, C. Ultrasonic Imaging System with Body Marker Annotations. Google Patents. US Patent 9,713,458, 25 July 2017.
  3. Li, H.; Xiong, P.; An, J.; Wang, L. Pyramid attention network for semantic segmentation. arXiv 2018, arXiv:1805.10180.
  4. Huang, Q.; Xia, C.; Wu, C.; Li, S.; Wang, Y.; Song, Y.; Kuo, C.-C.J. Semantic segmentation with reverse attention. arXiv 2017, arXiv:1707.06426. DOI: https://doi.org/10.5244/C.31.18
  5. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial networks. Commun. ACM 2020, 63, 139–144. DOI: https://doi.org/10.1145/3422622
  6. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.-Y.; Berg, A.C. Ssd: Single Shot Multibox Detector; Springer: Amsterdam, The Netherlands, 2016; pp. 21–37. DOI: https://doi.org/10.1007/978-3-319-46448-0_2
  7. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. DOI: https://doi.org/10.1109/CVPR.2016.91
  8. Lehtinen, J.; Munkberg, J.; Hasselgren, J.; Laine, S.; Karras, T.; Aittala, M.; Aila, T. Noise2noise: Learning image restoration without clean data. arXiv 2018, arXiv:1803.04189.
  9. Kashyap, M.M.; Tambwekar, A.; Manohara, K.; Natarajan, S. Speech denoising without clean training data: a noise2noise approach. arXiv 2021, arXiv:2104.03838. DOI: https://doi.org/10.21437/Interspeech.2021-1130
  10. Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. DOI: https://doi.org/10.1109/CVPR.2015.7298965
  11. Minaee, S.; Boykov, Y.Y.; Porikli, F.; Plaza, A.J.; Kehtarnavaz, N.; Terzopoulos, D. Image segmentation using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 44, 3523–3542. DOI: https://doi.org/10.1109/TPAMI.2021.3059968
  12. Chen, L.-C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587.
  13. Chaurasia, A.; Culurciello, E. Linknet: Exploiting encoder representations for efficient semantic segmentation. In Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 10–13 December 2017; pp.1–4. DOI: https://doi.org/10.1109/VCIP.2017.8305148
  14. Li, R.; Zheng, S.; Zhang, C.; Duan, C.; Su, J.; Wang, L.; Atkinson, P.M. Multiattention network for semantic segmentation of fine-resolution remote sensing images. IEEE Trans. Geosci. Remote Sens. 2021, 60, 1–13. DOI: https://doi.org/10.1109/TGRS.2021.3093977
  15. Iakubovskii, P. Segmentation Models Pytorch. Available online: https://github.com/qubvel/segmentation_models.pytorch (accessed on 17 April 2024).
  16. Chlap, P.; Min, H.; Vandenberg, N.; Dowling, J.; Holloway, L.; Haworth, A. A review of medical image data augmentation techniques for deep learning applications. J. Med. Imaging Radiat Oncol. 2021, 65, 545–563. DOI: https://doi.org/10.1111/1754-9485.13261
  17. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation, In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Springer: Berlin, Germany; pp. 234–241. DOI: https://doi.org/10.1007/978-3-319-24574-4_28
  18. Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Granada, Spain, 20 September 2018; Springer: Berlin, Germany; pp. 3–11. DOI: https://doi.org/10.1007/978-3-030-00889-5_1
  19. Ibtehaz, N.; Rahman, M.S. Multiresunet: Rethinking the u-net architecture for multimodal biomedical image segmentation. Neur. Netw. 2020, 121, 74–87. DOI: https://doi.org/10.1016/j.neunet.2019.08.025
  20. Ponomarenko, N.; Silvestri, F.; Egiazarian, K.; Carli, M.; Astola, J.; Lukin, V. On between-coefficient contrast masking of dct basis functions. In Proceedings of the Third International Workshop on Video Processing and Quality Metrics for Consumer Electronics, Scottsdale, AZ, USA, 13–15 January 2007; Volume 4.
  21. Tieleman, T.; Hinton, G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neur. Netw. Mach. Learn. 2012, 4, 26–31.
  22. RMSprop. Available online: https://pytorch.org/docs/stable/generated/torch.optim.RMSprop.html#torch.optim.RMSprop (accessed on 12 October 2022).