Abstract:In order to solve the small angle of the endoscopic view, large changes of target scale and mutual occlusion during endotracheal intubation in difficult airways, a multi-modal intelligent target detection algorithm for endotracheal intubation based on deep learning was proposed with a combination of endoscopic images and carbon dioxide concentration information. First, the traditional YOLOv3 network was improved. It adopted parallel multi-branch dilated convolution block with different dilated rates to extract more information, and up-sampled the output features and concatenated these tensors. Second, the vector localization algorithm was applied to locate the center of the target according to the difference of multi-channel carbon dioxide concentration, so as to further correct the center position of the boundary box predicted by YOLOv3 network. It helped improve the accuracy of small target detection and assisted to locate airway. Finally, a prototype of the new multimodal endotracheal intubation assistant device was developed based on the proposed algorithm, and tested in a simulated airway to verify its feasibility. The effect of the prototype was satisfactory with a median operation time of 15.5 s and a success rate of 97.3%. The study shows that the multi-modal intelligent target detection algorithm for endotracheal intubation based on deep learning has good operation results and can effectively assist endotracheal intubation.