Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Schema and example drafts for the KM3NeT alert programm #194

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Schema and example drafts for the KM3NeT alert programm #194

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f4c7dff
4.2.0
Sep 6, 2024
49a093a
Back to development
Sep 6, 2024
0946e61
KM3NeT schema and example drafts
Sep 6, 2024
4113c4b
Update package.json
VincentLuminy Sep 6, 2024
8dc13df
Update gcn/notices/km3net/test/medal_ranking_alert.example.json
VincentLuminy Sep 10, 2024
1e255af
Revert the (unwanted) bump of version
Sep 10, 2024
b87ce06
Adress reviewer coment: Improve description of 'prob_evt_topology'
Sep 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions gcn/notices/km3net/test/medal_ranking_alert.example.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"$schema": "https://gcn.nasa.gov/schema/main/gcn/notices/km3net/test/medal_ranking_alert.schema.json",
"mission": "KM3NeT",
"instrument": "ARCA028",
"messenger": "Neutrino",
"packet_type": 999,
"alert_tense": "test",
"alert_type": "initial",
"alert_datetime": "2024-09-01T12:01:00.00Z",
"analysis_pipeline": "exceptionnal_evt_arca",
VincentLuminy marked this conversation as resolved.
Show resolved Hide resolved
"description": "KM3NeT online analysis, bronze candidate neutrino observation.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it specific field, like medal_rank, instead of description. It is useful property, should be machine-readable.

"event_name": ["KM240901A"],
"trigger_time": "2024-09-01T12:00:00.00Z",
"ra": 13.82,
"dec": 19.01,
"ra_dec_error": 0.9,
"healpix_url": "https://www.km3net.org/about-km3net/open-access/",
"far": 8.029e-8,
"additional_info": "Track only / Track+Shower analysis. Up-going / All-sky selection. Analysis pipeline event selection tuned to select X event per month in average.",
"triggering_evts": [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How this ra,dec is different from the event above?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case (where the alert is produced by only one event) there is no difference.
But we have analysis pipeline where several events (coming from the same direction in a short time window, but individually below our threshold to send alert) would be identified as a signal of interest and lead to alert creation.
In that case, the coordinates in "triggering_evts" are the coordinates of our individual events, while the "ra", "dec" and "ra_dec_error" of the alert body correspond to the most probable direction.

We wanted to have an alert message as generic as possible, this is why we decided to duplicate the information, but I am open to suggestions if you have.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thanks.
Localization (ra,dec) is used at two places.

(---)"ra": 13.82,
"dec": 19.01,
"ra_dec_error": 0.9,
"healpix_url": "https://www.km3net.org/about-km3net/open-access/",
"far": 8.029e-8,
"additional_info": "Track only / Track+Shower analysis. Up-going / All-sky selection. Analysis pipeline event selection tuned to select X event per month in average.",
"triggering_evts": [
{
"trigger_time": "2024-09-01T12:00:00.00Z",
(---) "ra": 13.82,
"dec": 19.01,

I would suggest, either put all localizations in trigger_evts (ra, dec) list, Or create two examples files for one schema. One for alert, another for several events analysis pipeline.
It's creating confusion, example, whether above one is follow-up event.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it could be confusing. I will create a second example file with several events in the alert. We are also discussing to rename "triggering_evts" into something more explicit ("hits_information" or "internal_triggers_information" or something else). I will commit the change as soon as we reach an internal agreement.
The descriptions in the schema file will be updated as well (both for the alert coordinates and the triggering_evts variable).

{
"trigger_time": "2024-09-01T12:00:00.00Z",
"ra": 13.82,
"dec": 19.01,
"ra_dec_error": 0.9,
"cos_zenith": -0.95,
"prob_evt_topology": "(Starting) Track / Shower",
"n_triggered_pmt": 586
}
],
"known_sources": "No search / No match / [List]"
}
142 changes: 142 additions & 0 deletions gcn/notices/km3net/test/medal_ranking_alert.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
{
"$id": "https://gcn.nasa.gov/schema/main/gcn/notices/km3net/test/medal_ranking_alert.schema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"unevaluatedProperties": false,
"title": "KM3NeT/Gold-Silver-Bronze Alerts",
"description": "Candidate neutrino observation reported from the KM3NeT online analysis pipeline.",
"allOf": [
{
"$ref": "../../core/Reporter.schema.json",
"description": "Instrument (ARCAxxx/ORCAxxx) issuing the alert. Mission=KM3NeT; Messenger=Neutrino."
},
{
"$ref": "../../core/Alert.schema.json",
"description": "Tense role (test/current observation), type (initial/update/retraction) and datetime of the alert generation"
},
{
"$ref": "../../core/Event.schema.json",
"description": "Event ID (eg 1A. Nb increment for each new alert, letter increment for update/retraction of the nb). Event_Name (KM3-yymmddL) if appropriate. "
},
{
"$ref": "#/$defs/space_time_coord",
"description": "Most probable Localization (Time, Position, Error and Skymap if available)."
},
{
"$ref": "../../core/Statistics.schema.json",
"description": "FAR and other statistic related to the event probability."
},
{
"$ref": "../../core/AdditionalInfo.schema.json",
"description": "Description of the pipeline used to produce the alert."
}
],
"properties": {
"$schema": true,
"packet_type": {
"type": "number",
"description": "packet_type provided by GCN as for the old GCN format, associatied to notice_type/topic Gold=, Silver=, Bronze=."
Copy link
Member

@Vidushi-GitHub Vidushi-GitHub Sep 14, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this associated with your pipeline?
Or, GCN Classic? As we will not rely on the old system in the future. And I am afraid that we have packet_type with new Notices.

@jracusin do we have packet type with new system?
KM3NeT had following concern over the email: "Additionally, my collaboration has requested that we include a packet_type in our alerts. Would it be possible to provide us with three distinct packet_type values (one for each of the Gold, Silver, and Bronze alerts), similar to how it was done with the classical VOEvent messages?"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you stated, we would like to be able to give a packet_type in the notice, like it was done in GCN Classic. The number here are random for example, but will be updated with the accurate value if we have one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Vidushi-GitHub @jracusin would it be possible to have a packet_type even if our notices are not part of GCN classic ?
I know it is not needed from a technical point of view with Kafka, but we want to provide this information for compatibility with pipelines already existing in experiments susceptible to receive our alerts.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VincentLuminy We'd rather not reference back to GCN Classic notice formats. We can definitely find a way to list your classification of "Gold, Silver, Bronze", but "packet_type" isn't the most descriptive or intuitive keyword.

You could use the classification in the statistics core schema, though that is designed to have a probabilistic classification for each choice as a dictionary.

You could use "additionalInfo" like IceCube.

Or you could create a custom schema keyword like you have done, but maybe name it something more intuitive like "alert_criteria" or "alert_credibility".

},
"description": { "type": "string", "description": "Alert description." },
"analysis_pipeline": {
"type": "string",
"description": "Analysis pipeline outputing the alert (eg exceptionnal_evt_arca, multiplets_orca, ...)."
},
"triggering_evts": {
"type": "array",
"items": { "$ref": "#/$defs/km3event" },
"description": "Array of KM3NeT event space-time coord, topology and energy info."
},
"known_sources": {
"type": ["array", "string"],
"items": {
"anyOf": [
{ "$ref": "#/$defs/knownsrc" },
{ "$ref": "#/$defs/mmcoincevent" }
]
},
"description": "No search / No match / Array of known astrophysical sources present in the KM3NeT event RoI (not exhaustive) OR reference and position for a coincident time-variable source."
}
},
"required": [
Copy link
Member

@Vidushi-GitHub Vidushi-GitHub Sep 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you wish you can skip "required" option, it will enforce all fields must be there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it to help producer and receiver to understand what they can expect. If you don't see any disadvantage of having it, I would like to keep it.
However if you strongly advise to remove it, I will.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your pipeline will produce all the fields for each notices, then fine.
One can read at schema browser (https://gcn.nasa.gov/docs/schema/v4.1.0/gcn/notices), for what to expect, even we will soon create and announce mission page soon after your feedback.
eg: https://gcn.nasa.gov/missions/einstein-probe
It will put constraint on your pipeline, it's upto you.

"$schema",
"packet_type",
"description",
"mission",
"instrument",
"messenger",
"alert_tense",
"alert_type",
"alert_datetime",
"analysis_pipeline",
"event_name",
"trigger_time",
"ra",
"dec",
"ra_dec_error",
"far",
"triggering_evts",
"known_sources"
],
"$defs": {
"prob_evt_topology": {
VincentLuminy marked this conversation as resolved.
Show resolved Hide resolved
"type": "string",
"description": "track, shower, starting event."
},
"cos_zenith": {
"type": "number",
"description": "Value of the cosine of the angle between the event direction and the vertical -1 correspond to up-going event while 1 is down-going."
},
"n_triggered_pmt": {
"type": "number",
"description": "Number of PMT triggered by the event (energy proxy)."
},
"space_time_coord": {
"type": "object",
"allOf": [
{ "$ref": "../../core/DateTime.schema.json" },
{ "$ref": "../../core/Localization.schema.json" }
],
"description": "Time and position of a source."
},
"km3event": {
"type": "object",
"allOf": [{ "$ref": "#/$defs/space_time_coord" }],
"properties": {
"prob_evt_topology": { "$ref": "#/$defs/prob_evt_topology" },
"cos_zenith": { "$ref": "#/$defs/cos_zenith" },
"n_triggered_pmt": { "$ref": "#/$defs/n_triggered_pmt" }
},
"required": [
"trigger_time",
"ra",
"dec",
"ra_dec_error",
"prob_evt_topology",
"cos_zenith",
"n_triggered_pmt"
],
"description": "[Date_Event, RA, Dec, Radius_Error, Topology, Zenith, NTrigPmts] of a KM3NeT event."
},
"knownsrc": {
"type": "object",
"allOf": [
{ "$ref": "../../core/FollowUp.schema.json" },
{ "$ref": "../../core/Localization.schema.json" },
{ "$ref": "../../core/Distance.schema.json" },
{ "$ref": "../../core/Redshift.schema.json" }
],
"required": ["reference", "ra", "dec"],
"description": "[Name, Catalog, URL, RA, Dec, Distance (if available)] of each source (not exhaustive) in the KM3NeT event RoI"
},
"mmcoincevent": {
"type": "object",
"allOf": [
{ "$ref": "../../core/FollowUp.schema.json" },
{ "$ref": "#/$defs/space_time_coord" }
],
"required": ["ref_type", "ref_instrument", "trigger_time"],
"description": "[Type, Instrument, Notice_Link (if available), Time, Position (if available)] for each GCN/Chime alerts in coincidence with KM3NeT event / Burst in LCs provided by MAXI or Swift-BAT for a time variabile source."
}
}
}
4 changes: 2 additions & 2 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading