Validation data and Forecast data error

[priya-sharma-07]

@jweyn could you please suggest a file that actually converts the validation data in the format of forecast data. because in forecast data the format is (f_hour, sample, lat, lon, varlev) and in validation data the format is (time, lat, lon, varlev).
actually, we are testing the model on the ERA5 dataset subregion(2D data). we are able to train the model with a
loss of 0.0234. but for prediction, we are not getting expected results. Maybe we are not able to compare validation data and forecast data correctly. it would be great if you could suggest a solution.

[ashish615]

Even I am facing the same issue while calculating error i am not able to reproduce results as per reported. I am forecasting for the 7 days and 6h interval. Histogram represents the number of instances with given rmse value(ranging from 1 to 15). Please Help me with this.

[jweyn]

Hi @priya-sharma-07, to convert the validation data to the same format as the model prediction, first understand that the "sample" dimension corresponds to the time of forecast initialization while the "f_hour" dimension corresponds to the number of forecast hours in the future relative to the initialization time. So for example, sample=2018-01-01 and f_hour=108 means the validation would be at time=2018-01-04 12:00. There is a method that converts training data to a forecast-like array in DLWP.verify.verification_from_series.

Note that this conversion will result in some NaNs for the latest initialization times at the end of a time series, since the forecast will be for some future time that hasn't occurred (isn't in the original data).

[jweyn]

@ashish615 , I'm not sure I understand what is in your figure or what your issue is. Does the above comment help?