idea: kvs: support a native "EOF"

[chu11]

the job-info watcher knows it can stop watching certain keys when the job eventlog reaches its defined end

it could be convenient if there was a native "EOF" that could be added into the KVS, so such special handling wouldn't be necessary and the EOF could be used on other eventlogs (stdio, etc.)

[grondo]

This would imply an "eventlog_close()" or similar function would be required on eventlog writers?

[chu11]

I haven't thought that far yet, but probably something like that.

[grondo]

Job eventlogs could be "closed" when the job becomes inactive and the guest namespace is moved to the main namespace. However, many eventlogs don't really ever "close" (e.g. resource.eventlog). Even stdio eventlogs can have "log" messages interspersed so it may not necessarily be clear when to close them.

[garlick]

We were brainstorming about this yesterday and my first thought was to define an eof event that could be interpreted by the KVS. The downside is that we'd have to teach kvs-watch about eventlogs (ick).

I think where you guys seem to be going is better - something like an end of file token that could be appended to a valref array. Then when a key is being watched wth the APPEND option, the streaming RPC could naturally terminate. Something like flux_kvs_txn_close() to add the eof token?

Seems like not a bad enhancement to the whole "append to a key" design space, which we kind of slapped together at the time.

I think we should discuss whether it's helpful to our current code base before we jump in though.