Addressing the current lack of a unified and scalable system for assessing the effectiveness of rural black and odorous water treatments in China, we developed an evaluation framework using hierarchical analysis and structural entropy weight method. This framework incorporates five key evaluation themes: water quality assessment, source control, endogenous treatment, ecological restoration, and project management. Embedded in this approach is an adaptable index library, designed to accommodate the variability stemming from the diverse nature of black and odorous water bodies. This evaluation framework is empirically applied to eight distinct rural black and odorous water bodies in Hubei Province, representing four major classes of rural pollution: residential pollution, livestock and poultry pollution, plantation pollution, and industrial pollution. These sites have all undergone remediation processes. Results exhibit significantly effective treatment for two water bodies and effective treatment for a further six. These findings provide a testament to the practicality of the proposed evaluation system for assessing the effectiveness of remediation strategies across a range of rural black and odorous water bodies.