Research progress of deep learning applications in mass spectrometry imaging data analysis

doi:10.3724/SP.J.1123.2023.10035

Abstract

Abstract:

Mass spectrometry imaging (MSI) is a promising method for characterizing the spatial distribution of compounds. Given the diversified development of acquisition methods and continuous improvements in the sensitivity of this technology, both the total amount of generated data and complexity of analysis have exponentially increased, rendering increasing challenges of data postprocessing, such as large amounts of noise, background signal interferences, as well as image registration deviations caused by sample position changes and scan deviations, and etc. Deep learning (DL) is a powerful tool widely used in data analysis and image reconstruction. This tool enables the automatic feature extraction of data by building and training a neural network model, and achieves comprehensive and in-depth analysis of target data through transfer learning, which has great potential for MSI data analysis. This paper reviews the current research status, application progress and challenges of DL in MSI data analysis, focusing on four core stages: data preprocessing, image reconstruction, cluster analysis, and multimodal fusion. The application of a combination of DL and mass spectrometry imaging in the study of tumor diagnosis and subtype classification is also illustrated. This review also discusses trends of development in the future, aiming to promote a better combination of artificial intelligence and mass spectrometry technology.

Key words: mass spectrometry imaging (MSI), deep learning, neural network, data analysis

CLC Number:

O658

HUANG Dongdong, LIU Xinyu, XU Guowang. Research progress of deep learning applications in mass spectrometry imaging data analysis[J]. Chinese Journal of Chromatography, 2024, 42(7): 669-680.

Figures/Tables 8

References

[1]	Luxembourg S L, Mize T H, McDonnell L, et al. Anal Chem, 2004, 76(18): 5339
[2]	Buchberger A R, DeLaney K, Johnson J, et al. Anal Chem, 2017, 90(1): 240
[3]	Mellinger A L, Muddiman D C, Gamcsik M P. J Proteome Res, 2022, 21(8): 1800
[4]	Palmer A, Phapale P, Chernyavsky I, et al. Nat Methods, 2017, 14(1): 57
[5]	Duhamel M, Drelich L, Wisztorski M, et al. Nat Commun, 2022, 13(1): 6665
[6]	Li F, Luo Q. Chinese Journal of Chromatography, 2024, 42(2): 150
[6]	李芳, 罗茜. 色谱, 2024, 42(2): 150
[7]	Hou J J, Zhang Z J, Wu W Y, et al. Acta Pharmacol Sin, 2022, 43(12): 3096
[8]	Randall E C, Emdal K B, Laramy J K, et al. Nat Commun, 2018, 9(1): 4904
[9]	Dong Y, Aharoni A. Nat Pro Rep, 2022, 39(7): 1510
[10]	Zou Y, Tang W, Li B, et al. Int J Food Microbiol, 2022, 371: 109675
[11]	Guo X, Wang X, Tian C, et al. Talanta, 2023, 264: 124721
[12]	Kim J Y, Seo E S, Kim H, et al. Nat Commun, 2017, 8(1): 2113
[13]	Kompauer M, Heiles S, Spengler B. Nat Methods, 2017, 14(1): 90
[14]	Porta Siegel T, Hamm G, Bunch J, et al. Mol Imaging Biol, 2018, 20(6): 888
[15]	Coskun A F, Han G, Ganesh S, et al. Nat Commun, 2021, 12(1): 789
[16]	LeCun Y, Bengio Y, Hinton G. Nature, 2015, 521(7553): 436
[17]	He J, Huang L, Tian R, et al. Anal Chim Acta, 2018, 1015: 50
[18]	Huang L, Mao X, Sun C, et al. Anal Chim Acta, 2019, 1077: 183
[19]	Hu H, Laskin J. Adv Sci (Weinh), 2022, 9(34): e2203339
[20]	Song X, Zang Q, Zhang J, et al. Anal Chem, 2023, 95(17): 6775
[21]	Thomas S A, Race A M, Steven R T, et al. IEEE SSCI, 2016: 1
[22]	Inglese P, McKenzie J S, Mroz A, et al. Chem Sci, 2017, 8(5): 3500
[23]	Abdelmoula W M, Lopez B G, Randall E C, et al. Nat Commun, 2021, 12(1): 5544
[24]	Abdelmoula W M, Stopka S A, Randall E C, et al. Bioinformatics, 2022, 38(7): 2015
[25]	Ma W, Luo L, Liang K, et al. Anal Bioanal Chem, 2023, 415(14): 2819
[26]	Belthangady C, Royer L A. Nat Methods, 2019, 16(12): 1215
[27]	Helminiak D, Hu H, Laskin J, et al. Electron Imaging, 2021, 33(15): 2901
[28]	Ronneberger O, Fischer P, Brox T. MICCAI. (2015-05-18). http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net
[29]	Hu H, Helminiak D, Yang M, et al. ACS Meas Sci Au, 2022, 2(5): 466
[30]	Li D, Qian Y, Yao H, et al. Anal Chem, 2023, 95(29): 10879
[31]	Shen X, Shao W, Wang C, et al. Brief Bioinform, 2022, 23(5): bbac331
[32]	Wang H, Yin Y, Zhu Z J. Anal Chem, 2023, 95(16): 6533
[33]	Behrmann J, Etmann C, Boskamp T, et al. Bioinformatics, 2018, 34(7): 1215
[34]	Janßen C, Boskamp T, Hauberg-Lotte L, et al. Proteom Clin Appl, 2022, 16(4): e2100068
[35]	Guo D, Föll M C, Volkmann V, et al. Bioinformatics, 2020, 36: i300
[36]	Isberg O G, Giunchiglia V, McKenzie J S, et al. Metabolites, 2022, 12(5): 455
[37]	Guo L, Dong J, Xu X, et al. Anal Chem, 2023, 95(3): 1924
[38]	Ovchinnikova K, Stuart L, Rakhlin A, et al. Bioinformatics, 2020, 36(10): 3215
[39]	Chollet F. 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1251
[40]	Zhang W, Claesen M, Moerman T, et al. Anal Bioanal Chem, 2021, 413(10): 2803
[41]	Hu H, Bindu J P, Laskin J. Chem Sci, 2022, 13(1): 90
[42]	Tan M, Li Q. ICML, 2019, 97: 6105
[43]	Race A M, Sutton D, Hamm G, et al. Anal Chem, 2021, 93(6): 3061
[44]	Janßen C, Boskamp T, Le'Clerc Arrastia J, et al. Cancers (Basel), 2022, 14(24): 6181
[45]	Haque M I U, Mukherjee D, Stopka S A, et al. J Am Soc Mass Spectr, 2023, 34(2): 227
[46]	Connolly L, Jamzad A, Kaufmann M, et al. J Imaging, 2021, 7(10): 203
[47]	Balakrishnan G, Zhao A, Sabuncu M R, et al. IEEE Trans Med Imaging, 2019: 1788
[48]	Guo A, Chen Z, Li F, et al. GigaScience, 2022, 12: giad021
[49]	Dong C, Loy C C, He K, et al. IEEE PAMI, 2016, 38: 295
[50]	Guo L, Zhu J, Wang K, et al. Anal Chem, 2023, 95(25): 9714
[51]	Liao T, Ren Z, Chai Z, et al. Nat Mach Intell, 2023, 5(6): 656
[52]	Guo G, Papanicolaou M, Demarais N J, et al. Nat Commun, 2021, 12(1): 3241
[53]	Sun C, Wang A, Zhou Y, et al. Nat Commun, 2023, 14(1): 2692
[54]	Zhou Y, Jiang X, Wang X, et al. J Pharm Anal, 2023, 13(8): 851
[55]	Kleppe A, Skrede O J, De Raedt S, et al. Nat Rev Cancer, 2021, 21(3): 199
[56]	Mukhopadhyay S, Katzenstein A L. Am J Surg Pathol, 2011, 35(1): 15
[57]	Kriegsmann M, Casadonte R, Kriegsmann J, et al. Mol Cell Proteomics, 2016, 15(10): 3081
[58]	Boskamp T, Lachmund D, Oetjen J, et al. Biochim Biophys Acta Proteins Proteom, 2017, 1865(7): 916
[59]	Yamada H, Xu L, Eto F, et al. J Am Soc Mass Spectr, 2022, 33(9): 1607
[60]	Xie Y R, Castro D C, Rubakhin S S, et al. Nat Methods, 2024, 21(3): 521

Function	Neural networks	Model	Sample/ Dataset	Acquisition methods	Description	Refs.
Data prepro- cessing	AE	unsuper- vised	rat brain	MALDI	the first exploration of deep learning in MSI data dimensionality reduction	[21]
	AE	unsuper- vised	human colorectal carcinoma	DESI	unsupervised parameterization dimensionality reduction method established by combining DNN and t-SNE	[22]
	VAE	unsuper- vised	METASPACE	MALDI, DESI, FT-ICR	fully connected VAE for unsupervised peak learning of different instrument/sample datasets	[23]
	massNet, VAE	-	mouse brain spongio- blastoma	MALDI	scalable massNet framework for directly learning of features from high-dimensional data	[24]
	Res-Net	-	mouse muscle, human colorectal carcinoma	MALDI	Res-Net model based on channel selection to directly extract and characterize features	[25]
Image recon- struction	U-Net, MLP	supervised	mouse uterus, kidney	nano-DESI	sparse dynamic sampling planning and image reconstruction by U-Net CNN	[27, 29]
	U-Net-GAN	-	rat brain spongioblas- toma, kidney	MALDI	customize sampling unit and adversarial learning to optimize and improve accuracy of image reconstruction	[30]
	VGG	supervised	maternal plasma	ESI	irstly proposed to construct pseudo-images by using multi-dimensional information of LC-MS	[31]
	Res-Net	-	human esophagus squamous cell carcino- ma serum	ESI	image blocks based custom multi-channel approach to optimize pseudo-imaging accuracy	[32]
Image segmentation	IsotopeNet, Res-Net	supervised	pancreatic/squamous cell carcinoma, lung/ pancreatic tumor	MALDI	the first application of Res-Net in feature extraction and ROI labeling of MSI	[33]
	IsotopeNet	-	human non-small cell lung cancer	MALDI	a tumor classification system integrating DL and LDA classification algorithms	[34]
	MIL-CNN	semi-super- vised	human renal cell carci- noma, bladder cancer	MALDI, DESI	semi-supervised MXL framework for tissue level annotation to realize tumor sub-tissue labeling and classification	[35]
	MIL-CNN	semi-super- vised	human breast cancer	DESI	application of MIL in cancer diagnosis with high accuracy	[36]
	AE, CNN	unsuper- vised	mouse fetus, human breast cancer	MALDI	dc-DeepMSI model and DL algorithm based data reduction and feature clustering	[37]
Spatial clustering	Xception	semi-super- vised	METASPACE	MALDI	Xception network based semi-supervised Pi model showed best molecular colocalization ability	[38]
	Xception, ANN	unsuper- vised	human lymph nodes, mouse kidney	MALDI	Xception network based neural ion channel realized more accurate spatial clustering	[40]
	EffcientNet	self-supervised	METASPACE	MALDI	contrast learning based CNN model realized unannotated molecular colocalization	[41]
Multimodal fusion	DenseNet	supervised	mouse adenocarcino- ma	DESI	DenseNet based features annotation for automated tumor ROI division	[43]
	IsotopeNet, U-Net	-	human non-small cell lung cancer	MALDI	combining of U-Net and IsotopeNet for ROI analysis and tumor annotation of MSI	[44]
	Res-Net	semi-super- vised	human prostate	MALDI	similarity learning between H&E imaging and MSI data by Res-Net	[45]
	SiameseNet, U-Net	unsuper- vised	human prostate	DESI	MassReg model containing U-Net annotation learning and SiameseNet output	[46]
	DCNN	unsuper- vised	mouse kidney	DESI	the multimodal fusion strategy realized the custom feature extracting and matching through DCNN	[48]
	CNN	unsuper- vised	mouse brain, human liver cancer	DESI	spatial transformation network based DeepFERE model for high-resolution images construction	[50]
	GAN	-	mouse brain	MALDI	MOSR model by combining multiple networks to build the mapping relationship and predict ultra-high-resolution images	[51]

Function	Neural networks	Model	Sample/ Dataset	Acquisition methods	Description	Refs.
Data prepro- cessing	AE	unsuper- vised	rat brain	MALDI	the first exploration of deep learning in MSI data dimensionality reduction	[21]
	AE	unsuper- vised	human colorectal carcinoma	DESI	unsupervised parameterization dimensionality reduction method established by combining DNN and t-SNE	[22]
	VAE	unsuper- vised	METASPACE	MALDI, DESI, FT-ICR	fully connected VAE for unsupervised peak learning of different instrument/sample datasets	[23]
	massNet, VAE	-	mouse brain spongio- blastoma	MALDI	scalable massNet framework for directly learning of features from high-dimensional data	[24]
	Res-Net	-	mouse muscle, human colorectal carcinoma	MALDI	Res-Net model based on channel selection to directly extract and characterize features	[25]
Image recon- struction	U-Net, MLP	supervised	mouse uterus, kidney	nano-DESI	sparse dynamic sampling planning and image reconstruction by U-Net CNN	[27, 29]
	U-Net-GAN	-	rat brain spongioblas- toma, kidney	MALDI	customize sampling unit and adversarial learning to optimize and improve accuracy of image reconstruction	[30]
	VGG	supervised	maternal plasma	ESI	irstly proposed to construct pseudo-images by using multi-dimensional information of LC-MS	[31]
	Res-Net	-	human esophagus squamous cell carcino- ma serum	ESI	image blocks based custom multi-channel approach to optimize pseudo-imaging accuracy	[32]
Image segmentation	IsotopeNet, Res-Net	supervised	pancreatic/squamous cell carcinoma, lung/ pancreatic tumor	MALDI	the first application of Res-Net in feature extraction and ROI labeling of MSI	[33]
	IsotopeNet	-	human non-small cell lung cancer	MALDI	a tumor classification system integrating DL and LDA classification algorithms	[34]
	MIL-CNN	semi-super- vised	human renal cell carci- noma, bladder cancer	MALDI, DESI	semi-supervised MXL framework for tissue level annotation to realize tumor sub-tissue labeling and classification	[35]
	MIL-CNN	semi-super- vised	human breast cancer	DESI	application of MIL in cancer diagnosis with high accuracy	[36]
	AE, CNN	unsuper- vised	mouse fetus, human breast cancer	MALDI	dc-DeepMSI model and DL algorithm based data reduction and feature clustering	[37]
Spatial clustering	Xception	semi-super- vised	METASPACE	MALDI	Xception network based semi-supervised Pi model showed best molecular colocalization ability	[38]
	Xception, ANN	unsuper- vised	human lymph nodes, mouse kidney	MALDI	Xception network based neural ion channel realized more accurate spatial clustering	[40]
	EffcientNet	self-supervised	METASPACE	MALDI	contrast learning based CNN model realized unannotated molecular colocalization	[41]
Multimodal fusion	DenseNet	supervised	mouse adenocarcino- ma	DESI	DenseNet based features annotation for automated tumor ROI division	[43]
	IsotopeNet, U-Net	-	human non-small cell lung cancer	MALDI	combining of U-Net and IsotopeNet for ROI analysis and tumor annotation of MSI	[44]
	Res-Net	semi-super- vised	human prostate	MALDI	similarity learning between H&E imaging and MSI data by Res-Net	[45]
	SiameseNet, U-Net	unsuper- vised	human prostate	DESI	MassReg model containing U-Net annotation learning and SiameseNet output	[46]
	DCNN	unsuper- vised	mouse kidney	DESI	the multimodal fusion strategy realized the custom feature extracting and matching through DCNN	[48]
	CNN	unsuper- vised	mouse brain, human liver cancer	DESI	spatial transformation network based DeepFERE model for high-resolution images construction	[50]
	GAN	-	mouse brain	MALDI	MOSR model by combining multiple networks to build the mapping relationship and predict ultra-high-resolution images	[51]