New templates for reforecast in section 4 #280

sebvi · 2024-11-10T15:16:37Z

Initial request

The following two templates are available to encode reforecasts:

Code	Description
60	Individual ensemble reforecast control and perturbed at a horizontal level or in a horizontal layer at a point in time
61	Individual ensemble reforecast control and perturbed at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

These 2 templates contains the following set of keys for reforecasts:

number of octets	description
2	Year of Model Version Date
1	Month of Model Version Date
1	Day of Model Version Date
1	Hour of Model Version Date
1	Minute of Model Version Date
1	Second of Model Version Date

We propose to add additional templates which are combinations of existing section 4 templates (2, 12, 99, 100, 103, 104) for specific products extended by the reforecast specific part from templates 60 and 61.

Amendment details

ADD to code table 4.0 Product definition template number

Code	Description
137	Derived reforecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time
138	Derived reforecast based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval
139	Reforecast at a horizontal level or in a horizontal layer at a point in time for waves selected by period range.
140	Individual ensemble reforecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for waves selected by period range
141	Reforecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions
142	Individual ensemble reforecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions

ADD TEMPLATE 4.137, Derived reforecast based on all ensemble members at a horizontal level or in a horizontal layer at a point in time

Octet	Number of octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15- 16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19- 22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25- 28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Derived Forecast (see Code table 4.7)
36-39	4	Number of Forecasts in Ensemble
40-41	2	Year of Model Version Date
42	1	Month of Model Version Date
43	1	Day of Model Version Date
44	1	Hour of Model Version Date
45	1	Minute of Model Version Date
46	1	Second of Model Version Date

ADD TEMPLATE 4.138, Derived reforecast based on all ensemble members at a horizontal level or in a horizontal layer in a continuous or non-continuous time interval

Octet	Number of octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Generating Process (see code table 4.3)
13	1	Background Process
14	1	Generating Process Identifier
15- 16	2	Hours After Data Cut-off
17	1	Minutes After Data Cut-off
18	1	Indicator of Unit of Time Range (see code table 4.4)
19- 22	4	Forecast Time
23	1	Type of First Fixed Surface (see code table 4.5)
24	1	Scale Factor of First Fixed Surface
25- 28	4	Scaled Value of First Fixed Surface
29	1	Type of Second Fixed Surface (see code table 4.5)
30	1	Scale Factor of Second Fixed Surface
31-34	4	Scaled Value of Second Fixed Surface
35	1	Derived Forecast (see Code table 4.7)
36-39	4	Number of Forecasts in Ensemble
40-41	2	Year of Model Version Date
42	1	Month of Model Version Date
43	1	Day of Model Version Date
44	1	Hour of Model Version Date
45	1	Minute of Model Version Date
46	1	Second of Model Version Date
47-48	2	Year of end of overall time interval
49	1	Month of end of overall time interval
50	1	Day of end of overall time interval
51	1	Hour of end of overall time interval
52	1	Minute of end of overall time interval
53	1	Second of end of overall time interval
54	1	n - number of time range specifications describing the time intervals used to calculate the statistically processed field
55-58	4	Total number of data values missing in statistical process
		59 - 70 Specification of the outermost (or only) time range over which statistical processing is done
59	1	Statistical process used to calculate the processed field from the field at each time increment during the time range (see code table 4.10)
60	1	Type of time increment between successive fields used in the statistical processing (see code table 4.11)
61	1	Indicator of unit of time for time range over which statistical processing is done (see code table 4.4)
62-65	4	Length of the time range over which statistical processing is done, in units defined by the previous octet
66	1	Indicator of unit of time for the increment between the successive fields used (see code table 4.4)
67-70	4	Time increment between successive fields, in units defined by the previous octet
		71 - nn These octets are included only if n > 1, where nn = 59+ 12 x n
71-82	12	As octets 59 to 70, next innermost step of processing
83-nn	n/a	Additional time range specifications, included in accordance with the value of n. Contents as octets 59 to 70, repeated as necessary

ADD TEMPLATE 4.139, Reforecast at a horizontal level or in a horizontal layer at a point in time for waves selected by period range.

Octet	Number of octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Wave Period Interval (see code table 4.91)
13	1	Scale Factor of Lower Wave Period Limit
14 -17	4	Scaled Value of Lower Wave Period Limit
18	1	Scale Factor of Upper Wave Period Limit
19-22	4	Scaled Value of Upper Wave Period Limit
23	1	Type of Generating Process (see code table 4.3)
24	1	Background Process
25	1	Generating Process Identifier
26-27	2	Hours After Data Cut-off
28	1	Minutes After Data Cut-off
29	1	Indicator of Unit of Time Range (see code table 4.4)
30-33	4	Forecast Time
34	1	Type of First Fixed Surface (see code table 4.5)
35	1	Scale Factor of First Fixed Surface
36-39	4	Scaled Value of First Fixed Surface
40	1	Type of Second Fixed Surface (see code table 4.5)
41	1	Scale Factor of Second Fixed Surface
42-45	4	Scaled Value of Second Fixed Surface
46-47	2	Year of Model Version Date
48	1	Month of Model Version Date
49	1	Day of Model Version Date
50	1	Hour of Model Version Date
51	1	Minute of Model Version Date
52	1	Second of Model Version Date

ADD TEMPLATE 4.140, Individual ensemble reforecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for waves selected by period range

Octet	Number of octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12	1	Type of Wave Period Interval (see code table 4.91)
13	1	Scale Factor of Lower Wave Period Limit
14 -17	4	Scaled Value of Lower Wave Period Limit
18	1	Scale Factor of Upper Wave Period Limit
19-22	4	Scaled Value of Upper Wave Period Limit
23	1	Type of Generating Process (see code table 4.3)
24	1	Background Process
25	1	Generating Process Identifier
26-27	2	Hours After Data Cut-off
28	1	Minutes After Data Cut-off
29	1	Indicator of Unit of Time Range (see code table 4.4)
30-33	4	Forecast Time
34	1	Type of First Fixed Surface (see code table 4.5)
35	1	Scale Factor of First Fixed Surface
36-39	4	Scaled Value of First Fixed Surface
40	1	Type of Second Fixed Surface (see code table 4.5)
41	1	Scale Factor of Second Fixed Surface
42-45	4	Scaled Value of Second Fixed Surface
46	1	Type of Ensemble Forecast (see code table 4.6)
47-50	4	Perturbation Number
51-54	4	Number of Forecasts in Ensemble
55-56	2	Year of Model Version Date
57	1	Month of Model Version Date
58	1	Day of Model Version Date
59	1	Hour of Model Version Date
60	1	Minute of Model Version Date
61	1	Second of Model Version Date

ADD TEMPLATE 4.141, Reforecast at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions

Octet	Number of Octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12-13	2	Wave Direction Number
14-15	2	Number of Wave Directions – ND
16-17	2	Wave Frequency Number
18-19	2	Number of Wave Frequencies – NF
20	1	Type of Generating Process (see code table 4.3)
21	1	Background Process (defined by originating centre)
22	1	Generating Process Identifier (defined by originating centre)
23-24	2	Hours After Data Cut-off (see Note)
25	1	Minutes After Data Cut-off
26	1	Indicator of Unit of Time Range (see code table 4.4)
27-30	4	Forecast Time
31-32	2	Year of Model Version Date
33	1	Month of Model Version Date
34	1	Day of Model Version Date
35	1	Hour of Model Version Date
36	1	Minute of Model Version Date
37	1	Second of Model Version Date
38	1	Scale Factor of Wave Directions
		The following entry of 4 octets is repeated ND times. (ND is defined in octets 14-15)
39 + (ND-1)4 –42 + (ND-1)4	4	Scaled Values of Wave Directions
43 + (ND-1)*4	1	Scale Factor of Wave Frequencies
		The following entry of 4 octets is repeated NF times. (NF is defined in octets 18-19)
44 +(ND-1)4 + (NF-1)4-47 +(ND-1)4 + (NF-1)4	4	Scaled Values of Wave Frequencies

ADD TEMPLATE 4.142, Individual ensemble reforecast, control and perturbed, at a horizontal level or in a horizontal layer at a point in time for wave 2D spectra with explicit list of frequencies and directions

Octet	Number of Octets	Description
10	1	Parameter Category (see code table 4.1)
11	1	Parameter Number (see code table 4.2)
12-13	2	Wave Direction Number
14-15	2	Number of Wave Directions – ND
16-17	2	Wave Frequency Number
18-19	2	Number of Wave Frequencies – NF
20	1	Type of Generating Process (see code table 4.3)
21	1	Background Process (defined by originating centre)
22	1	Generating Process Identifier (defined by originating centre)
23-24	2	Hours After Data Cut-off (see Note)
25	1	Minutes After Data Cut-off
26	1	Indicator of Unit of Time Range (see code table 4.4)
27-30	4	Forecast Time
31	1	Type of Ensemble Forecast (see code table 4.6)
32-35	4	Perturbation Number
36-39	4	Number of Forecasts in Ensemble
40-41	2	Year of Model Version Date
42	1	Month of Model Version Date
43	1	Day of Model Version Date
44	1	Hour of Model Version Date
45	1	Minute of Model Version Date
46	1	Second of Model Version Date
47	1	Scale Factor of Wave Directions
		The following entry of 4 octets is repeated ND times. (ND is defined in octets 14-15)
48 + (ND-1)4 –51 + (ND-1)4	4	Scaled Values of Wave Directions
52 + (ND-1)*4	1	Scale Factor of Wave Frequencies
		The following entry of 4 octets is repeated NF times. (NF is defined in octets 18-19)
53 +(ND-1)4 + (NF-1)4-56 +(ND-1)4 + (NF-1)4	4	Scaled Values of Wave Frequencies

Comments

No response

Requestor(s)

Sebastien Villaume (ECMWF)
Robert Osinski (ECMWF)

Stakeholder(s)

ECMWF

Publication(s)

Manual on Codes (WMO-No. 306), Volume I.2, GRIB code table 4.0 (update)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.137 (create)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.138 (create)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.139 (create)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.140 (create)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.141 (create)
Manual on Codes (WMO-No. 306), Volume I.2, GRIB Template 4.142 (create)

Expected impact of change

None

Collaborators

No response

References

No response

Validation

No response

amilan17 · 2024-11-13T13:52:25Z

https://github.com/wmo-im/tt-tdcf/wiki/2024.11.13.tt.tdcf notes:

Sebastien introduced the proposal
@wmo-im/tt-tdcf please review and comment on this proposal

sebvi · 2024-12-02T09:52:52Z

Please find attached some sample files and a version of ecCodes capable of encode/decode the new proposed templates.
eccodes-2.40.0-Source.tar.gz
FT2025-1_280.tar.gz

amilan17 · 2024-12-03T14:37:20Z

ttps://github.com/wmo-im/tt-tdcf/wiki/2024.11.12.tt.tdcf notes:
@sebvi branch is ready for updates;

sebvi · 2025-01-10T11:22:23Z

branch is updated

amilan17 · 2025-01-14T14:01:06Z

https://github.com/wmo-im/et-data/wiki/2025.01.14.et.data notes:
@SibylleK will validate

rdosinski · 2025-01-15T09:51:56Z

Please find attached ASCII files with section 4 grib_dumps as discussed in the meeting from 14/01/2025 :
dumps_280.tar.gz

SibylleK · 2025-01-22T16:45:39Z

@sebvi could you also update the examples in this issue for PDT 137 - 142 with the GRIB messages Robert used for the dumps?
That would help the validation of this PDTs a lot.

SibylleK · 2025-01-28T16:48:29Z

I updated some entries in the branch.
I.e. correct octet counts in PDT 4.137 and add the missing octet entries in PDT 4.141.
Regarding PDT 4.138 I delete the octet count = 12 in row no 41, as the "next innermost step .." is not a 12 byte long entry but a repitation of previous entries.

With these corrections of the csv files in the branch I was able to decode the provided GRIB examples.
See: dwd_dumps_280.zip

The values in the dumps of PDT 137 and 138 are the same as in the provided ECMWF dumps.

But I cannot compare the entries for PDT 139 - 142 with the ECMWF dumps, as it seems that the GRIB examples are different files (with most entries set to 0 or missing instead of more significant entries as in the dumps).

But maybe it is nonetheless sufficient for the validation?

amilan17 · 2025-01-29T13:38:18Z

But maybe it is nonetheless sufficient for the validation?

@sebvi @SibylleK
The branches are QC'd and ready to merge, pending validation confirmation.

….0 (#300) * issue #280 adding templates 4.137 to 4.142 and update of code table 4.0 * Update GRIB2_Template_4_137_ProductDefinitionTemplate_en.csv Some incorrect OctetCount entries corrected * Update GRIB2_Template_4_138_ProductDefinitionTemplate_en.csv Delete OctetCount = 12 for the "next innermost step of processing", as it is not a 12 byte long entry. * Update GRIB2_Template_4_141_ProductDefinitionTemplate_en.csv Add OctetCount entries * Update GRIB2_Template_4_137_ProductDefinitionTemplate_en.csv Correct typo in month of model .. * minor editorial --------- Co-authored-by: Sebastien Villaume <[email protected]> Co-authored-by: Sibylle Krebber <[email protected]>

sebvi self-assigned this Nov 10, 2024

amilan17 added this to GRIB2 Amendments Nov 11, 2024

github-project-automation bot moved this to Submitted in GRIB2 Amendments Nov 11, 2024

amilan17 added this to the FT2025-1 milestone Nov 12, 2024

amilan17 added ocean and removed ocean labels Nov 13, 2024

amilan17 moved this from Submitted to In progress in GRIB2 Amendments Nov 13, 2024

sebvi added a commit that referenced this issue Dec 20, 2024

issue #280 adding templates 4.137 to 4.142 and update of code table 4.0

891e6f4

amilan17 moved this from In progress to In validation / review in GRIB2 Amendments Jan 14, 2025

amilan17 mentioned this issue Jan 23, 2025

issue #280 adding templates 4.137 to 4.142 and update of code table 4.0 #300

Merged

amilan17 linked a pull request Jan 23, 2025 that will close this issue

issue #280 adding templates 4.137 to 4.142 and update of code table 4.0 #300

Merged

amilan17 moved this from In validation / review to Validated in GRIB2 Amendments Feb 6, 2025

amilan17 closed this as completed Feb 7, 2025

github-project-automation bot moved this from Validated to Ready for FT approval procedure in GRIB2 Amendments Feb 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New templates for reforecast in section 4 #280

New templates for reforecast in section 4 #280

sebvi commented Nov 10, 2024 •

edited by amilan17

Loading

amilan17 commented Nov 13, 2024

sebvi commented Dec 2, 2024 •

edited

Loading

amilan17 commented Dec 3, 2024

sebvi commented Jan 10, 2025

amilan17 commented Jan 14, 2025

rdosinski commented Jan 15, 2025 •

edited

Loading

SibylleK commented Jan 22, 2025

SibylleK commented Jan 28, 2025

amilan17 commented Jan 29, 2025

New templates for reforecast in section 4 #280

New templates for reforecast in section 4 #280

Comments

sebvi commented Nov 10, 2024 • edited by amilan17 Loading

Initial request

Amendment details

Comments

Requestor(s)

Stakeholder(s)

Publication(s)

Expected impact of change

Collaborators

References

Validation

amilan17 commented Nov 13, 2024

sebvi commented Dec 2, 2024 • edited Loading

amilan17 commented Dec 3, 2024

sebvi commented Jan 10, 2025

amilan17 commented Jan 14, 2025

rdosinski commented Jan 15, 2025 • edited Loading

SibylleK commented Jan 22, 2025

SibylleK commented Jan 28, 2025

amilan17 commented Jan 29, 2025

sebvi commented Nov 10, 2024 •

edited by amilan17

Loading

sebvi commented Dec 2, 2024 •

edited

Loading

rdosinski commented Jan 15, 2025 •

edited

Loading