RFC: iNews compatiblity with segments

[PascalViauRC]

About me

This RFC is posted on behalf of CBC/Radio-Canada

Use case

In ENPS, it's possible to group multiple stories together to form a segment. The MOS Gateway sends the stories to Sofie with a slug value with the format "SEGMENT NAME; STORY NAME". The Sofie Core then merges the stories together to form the segments and keeps track of the changes coming from ENPS.

In iNews, there is no such thing as a « segment ». We can force individual story slugs to use the format "SEGMENT NAME; STORY NAME", but that means we have to duplicate the segment name in each story. There is no way to assign a single value to multiple stories.

So the two challenges are : first, how can we define a segment coming from iNews and second, how can we name it using a single value in one of the stories of the segment ?

Proposal

I've noticed that Sofie uses the "SEGMENT NAME;" to name the segment, but the segment grouping is not entirely done using the segment name as an id. For example, two groups of stories separated by an empty story will form two segments in the rundown, even if they use the same segment id.

This is a great behavior, and that means there is already a notion of « separator », an empty story being the separator in that case. So one way of creating a segment would be to use an empty story as a separator, even for stories with no segment name like the one coming in from iNews. One step further would be to define in the blueprint which characters in the slug should identify the story acting as a separator. For example, the separator could be a story with an empty slug "" or with the slug characters "=====================".

As for the naming of the segment, we coud define in the blueprint a pattern to extract the name from the story slugs. For exemple, we could define a start character "[" and an end character "]" so if the string "[SEGMENT 1]" is found in one of the story slugs of the segment, it will be identified and parsed as the segment name "SEGMENT 1".

There are probably a lot of ways to do that, so I'm open to suggestions !

Process

The Sofie Team will evaluate this RFC and open up a discussion about it, usually within a week.

• RFC created
• Sofie Team has evaluated the RFC
• A workshop has been planned
• RFC has been discussed in a workshop
• A conclusion has been reached, see comments in thread

[nytamin]

Thank you for submitting this RFC!
(If you haven’t already, please give our contribution guidelines a read.)

We've done a quick huddle in the Sofie Team, and we'd like to invite you to a workshop where we'll outline a plan for how this could be implemented, on Tuesday, January 23rd at 15:00 CET (14:00 GMT, 9:00 EST).

If anyone else wishes to join the workshop, please email me at [email protected]

[PeterC89]

Just putting a couple of quick notes here ahead of the workshop.
In our NCS setup the labelling is thus:

roID: MY.NCS.ID.MOS;XX_0.00000000

Then for a given story it's:
storyID: MY.NCS.ID.MOS;XX_0.00000000;XX_1.11111111,2.22222222.2

Where the numbers reflect the internal numbering system of the NCS and the first part of the storyID is the same as the roID

The storySlug may be something like Title but may also be like Title-Subtitle depending on the columns used in the NCS.

The way this translates into Sofie currently is that each story or 'line' in the NCS becomes a separate segment which is our desired behaviour.

It would be great to have this configurable as it means the behaviour then could be changed regardless of NCS and could even be changed for different studios if desired.

[nytamin]

Meeting notes, Workshop January 23rd

Ideas for how to group stories

There are several possible ways to group stories into segments in the NCS, some ideas are:

• Having a delimiter in the story Slug using ";" ("segmentName;storyName") (this is how NRK does it today).
• Add an empty story line to indigate separate segments.
• Add a story line with the slug "============" to separate segments.
• Mark story lines with a value to group adjacent lines (with the same value) together.

It should be noted that whatever solution we come up with, it should be possible to keep existing behaviour, which includes "no grouping of stories at all".

The current data flow

In Sofie, there are two data flows as of today:

MOS

1. RunningOrder and Story updates sent over MOS (via Sofie MOS Gateway).
2. In Sofie Core: Stories are grouped into IngestSegments & IngestParts (using the ";" delimiter in the Story slug).
3. IngestSegments & IngestParts are fed into the Blueprints, out comes Segments & Parts.
4. Store result to DB

Not MOS

1. IngestSegments & IngestParts are sent into Sofie (using the Sofie ingest api)
2. IngestSegments & IngestParts are fed into the Blueprints, out comes Segments & Parts.
3. Store result to DB

Suggested new data flow

To allow for different algorithms for grouping stories, we propose modifying the ingest data flow in Sofie Core by introducing a new blueprint method, responsible for reworking and grouping of IngestStories into IngestSegments.

The new blueprint method can be optional in the blueprints, with a default behaviour matching today's.

MOS

1. RunningOrder and Story updates sent over MOS (via Sofie MOS Gateway).
2. CHANGED: Stories are NOT grouped, but wrapped into IngestParts (one in each IngestSegment)
3. NEW: IngestSegments & IngestParts are send into a new blueprint method.
   This blueprint method goes through and groups them into IngestSegments & IngestParts.
4. IngestSegments & IngestParts are fed into the Blueprints, out comes Segments & Parts.
5. Store result to DB

Not MOS

1. IngestSegments & IngestParts are sent into Sofie (using the Sofie ingest api)
2. NEW: IngestSegments & IngestParts are send into a new blueprint method.
   This blueprint method goes through and groups them into IngestSegments & IngestParts.
3. IngestSegments & IngestParts are fed into the Blueprints, out comes Segments & Parts.
4. Store result to DB

Notes from discussion

• Q: Could this be used to modify rundown properties (and possible reject it) based on the stories?
  A: Maybe, it would require the grouping method to also output IngestRundown. More investigations needed.
• Q: Does this mean that in the new blueprint method, information about adjacent stories could be inserted into a story?
  A: Yes, probably.
• getShowStyleId and getRundown both have the IngestRundown (without the segments and parts?) and can ‘reject’ the rundown. Should this be done before or after this processing step? More investigations needed.
• Do we need these methods as separate steps? Probably a separate thing to consider.

[nytamin]

FYI:
While discussing possible implementation solutions in #1121, we discovered that a solution to that might very well be closely related to an implementation of this.

Please let me know before spending time on implementing this and I'll help coordinate efforts.

[PascalViauRC]

We have not started implementation, so yes, if it can be part of #1121, it's fine for us.

[PascalViauRC]

I just saw that #1121 is now closed. Have you had the chance to address this issue as well ? Thanks !

[Julusian]

The PR for #1121 is currently awaiting review and merging #1293

But yes, it puts some pieces in place to allow you to do this. The relevant portion of the change is that the grouping of MOS stories into segments is now handled by the new processIngestData blueprint method.

If not implemented, the default implementation of this method looks like:

// MOS has a special flow to group parts into segments
const groupedResult = context.groupMosPartsInRundownAndChangesWithSeparator(
	nrcsIngestRundown,
	previousNrcsIngestRundown,
	ingestRundownChanges.changes,
	';' // Backwards compatibility
)

context.defaultApplyIngestChanges(
	mutableIngestRundown,
	groupedResult.nrcsIngestRundown,
	groupedResult.ingestChanges
)

which retains backwards compatibility.

For your case, you would need to implement this new processIngestData method, which could be along the lines of:

const groupedResult = context.groupPartsInRundownAndChanges(
	nrcsIngestRundown,
	previousNrcsIngestRundown,
	ingestRundownChanges.changes,
	(segments) => {
		// TODO - perform the grouping here
		return segmentsAfterMyChanges
	}
)

context.defaultApplyIngestChanges(
	mutableIngestRundown,
	groupedResult.nrcsIngestRundown,
	groupedResult.ingestChanges
)

Something would need to be filled in where the TODO is, depending on the workflow you want in inews.
The segments parameter to that function has each mos story as its own segment, to make the types and behaviour when this is not done be sensible.
For inspiration, groupMosPartsInRundownAndChangesWithSeparator is just a wrapper around the groupPartsInRundownAndChanges method, using a function implementation that is only 30 lines of code https://github.com/bbc/sofie-core/blob/aefc95db9e7a8be1f1ecd82e4008f6df4ee44741/packages/job-worker/src/blueprints/ingest/groupPartsInRundownAndChanges.ts#L16-L44. Hopefully what you need for inews will be of a similar level of complexity

[PascalViauRC]

Pretty neat! I will probably wait after the code review to test it in release52. Thanks for the detailed explanations!