Aiming at the fuzziness of the assessment result of the damage effect of the building and the problem of strong subjectivity of the weight setting of the fuzzy comprehensive evaluation method, an improved method is proposed. By selecting the characteristics of the high resolution remote sensing image before and after the damage of the building, the evaluation index set is constructed. The conversion formula is defined to get the value of the evaluation index. Calculate the evaluation index variation coefficient and get the weight constraint. Using the stochastic multi-criteria acceptability analysis-2 (SMAA-2) idea to optimize the deterministic SMAA method, the probability set of the damage grade assessment is synthesized from the bottom to the top. The assessment results are obtained according to the assessment criteria. The method only uses the image information, and the assessment process does not need to set weights in the quantitative setting, so the subjectivity is reduced. The effectiveness of the method is verified by practical examples.