Disaster damage assessment is an important basis for the objective assessment of the social impacts of disasters and for the planning of recovery and reconstruction. It is also an important research field with regard to disaster mitigation and risk management. Quantitative assessment of physical damage refers to the determination of the physical damage state of the exposed elements in a disaster area, reflecting the aggregate quantities of damages. It plays a key role in the comprehensive damage assessment of major natural hazard-induced disasters. The National Disaster Reduction Center of China has established a technical work flow for the quantitative assessment of disaster physical damage using remote sensing data.This article presents a quantitative assessment index system and method that can be integrated with high-resolution remote sensing data, basic geographical data, and field survey data. Following the 2014 Ludian Earthquake in Yunnan Province, China, this work flow was used to assess the damage to buildings, roads, and agricultural and forest resources, and the assessment results were incorporated into the Disaster Damage Comprehensive Assessment Report of the 2014 Ludian Earthquake for the State Council of China. This article also outlines some possible improvements that can be addressed in future work.