关键词: 多模态大模型, 视觉? 语言模型, 毁伤评估, 思维树, 毁伤推理

Abstract:

Aiming at the problems that the existing damage assessment technology mainly focuses on the extraction of change information, the difficulty of damage feature discrimination, and the low degree of automation of damage reasoning, a complex target damage assessment method based on multimodal large scale model is proposed by using the technical advantages of multimodal large model for reference. Firstly, the physical damage analysis model based on visual-language model is designed, and the hallucination suppression model based on SegGPT is designed by using the damage semantic analysis technology of computer vision to optimize the accuracy of physical parts damage analysis. Then, combined with damage rule knowledge injection, a damage reasoning model integrated with damage tree is designed, and a target detection and recognition algorithm based on small samples is proposed to ensure the automatic realization of damage reasoning. Finally, the results of damage analysis and reasoning are presented visual tables in the form of Markdown. Experimental results show that the proposed algorithm can automatically judge the damage position and damage degree of complex targets, which the accuracy of physical damage assessment is improved by approximately 29.5% compared to the Qwen2.5 method, and can achieve the reasoning ability from local to overall. simultaneously, for images of up to 4 000×2 492 pixels, the damage assessment can be completed within 1 min, leading to good real-time performance.

Key words: multimodal large model, vision-language model, damage assessment, tree of thought （ToT）, damage reasoning