A Fraction Skill Score-based precipitation forecast jumpiness index and its application

Based on two sets of precipitation forecast results of the CMA-BJ model (operational and assimilated forecast) on 9 August 2021 and operational results on 4 June 2022 and 17 September 2023, combined with subjective analysis, four objective evaluation indexes (uncertainty, root-mean-square error, inconsistency index and forecast jumpiness index based on Fraction Skill Score (FSS)) are used to quantitatively evaluate the inconsistency characteristics of precipitation forecast. The analysis of the above three precipitation processes shows that the FSS-based forecast jumpiness index can not only identify the three forecast moments at which the precipitation forecast decreased significantly in the operational results on 9 August 2021 and 4 June 2022, but also distinguish them from the cases in which precipitation forecast is relatively stable based on the assimilated forecast on 9 August 2021 and the operational result on 17 September 2023. As the forecast moment gradually approaches the latest forecast, the jumpiness index either increases or maintains a large value and small fluctuation on the whole, indicating that the 15 continuous precipitation forecast results gradually converge or are roughly similar to the latest forecast, which is relatively consistent with the results of subjective analysis. Since the forecast jumpiness index focuses on precipitation probability in the selected time window, it is not affected by the value, and thus it can more directly reflect spatial distribution characteristics of precipitation forecast in the continuous forecast results of the model. This makes it have a unique advantage for analyzing the overall evolution characteristics of precipitation process. The other three indexes are insufficient to characterize the inconsistency of precipitation forecast. For example, the uncertainty and the root-mean-square error are significantly affected by the value of precipitation forecast, and the uncertainty is not able to reflect temporal characteristics of the forecast inconsistency. The inconsistency index changes greatly as the forecast time rolls from time to time. According to the criteria of the inconsistency index, many jumpiness cases that deviate from results of the subjective analysis are identified.