Allow Lists and Dictionaries to be delimited by OWS

[mnot]

From Magnus's feedback

[mnot]

also @clelland PTAL

[kaduk]

This does not instill much of a panicked reaction in me, which is probably a good sign.
I trust it is intentional to not apply a similar treatment to inner lists (which also has a prose description I had commented on previously) on the grounds of them not resembling existing top-level header field structures? That's certainly defensible, though I have to wonder whether someone will simply assume that it must be the same (not that we have much hope of changing such a person's behavior).

[phluid61]

How does this vibe with #998? I know we spent a bit of time on it, but it's long enough ago that I can't recall detail.

[phluid61]

Ah, right, I think I remember. Doing it this way means inner-list is different:

inner-list = "(" *SP [ sf-item *( 1*SP sf-item ) *SP ] ")"
             *parameter

The original motivation to remove tabs was because the above was going to be

  "(" OWS [ sf-item *( RWS sf-item ) OWS ] ")"

but there was consternation about RWS. Most of the discussion is in #983

[bsdphk]

I dont think I ever saw a TAB in a HTTP header in real life, but I am OK with this.

[mnot]

@phluid61 #988 was where my mind went first too, but that seems to be just about in inner lists (which makes sense). I think this is separate; intermediaries that don't know about the semantics and syntactic limitations of a given field shouldn't be meddling inside it (which inner list delimitation is).

[reschke]

This is so that field line combination as per the Semantics spec can happen, right?

I note that https://greenbytes.de/tech/webdav/draft-ietf-httpbis-semantics-latest.html#field.order currently says:

A recipient MAY combine multiple field lines with the same field name into one field line, without changing the semantics of the message, by appending each subsequent field line value to the initial field line value in order, separated by a comma and optional whitespace. For consistency, use comma SP.

Note that it says "comma an optional whitespace", not "OWS". Depending on how you read it, it might include other whitespace characters. I believe this needs to be fixed in the Semantics spec.

[clelland]

@reschke, I think so -- that's the only reason to allow this. Existing servers may already legitimately be using TAB when combining multiple field lines. And I suppose it makes sense, given the recent discussion about not breaking up individual list or dictionary members over multiple lines, that we don't need to replace *SP with OWS everywhere.

I'm okay with this change -- does it warrant a note in the prose somewhere that this special case only exists to accommodate field concatenation?

Note that it says "comma an optional whitespace", not "OWS". Depending on how you read it, it might include other whitespace characters. I believe this needs to be fixed in the Semantics spec.

Let's hope we don't need to add VTAB or FF to this, too :)

[mnot]

@reschke I don't know that that's necessary; we already define whitespace pretty clearly.

In SF, I looked at changing "whitespace", but in each case the ABNF and algorithms make it clear what we're talking about, so I don't see a strong need to disambiguate.

[reschke]

I'm ok with the change over here; that said, the Semantics spec could be clearer. Opened httpwg/http-core#370.

[mnot]

@bsdphk and I have been talking about this, and have come to the conclusion that this change went too far.

Specifically, steps 1 and 5 in "Parsing Structured Fields" should just be SP, not OWS, because there isn't any risk of an intermediary inserting TAB before the first field line or after the last field line; it's only between List and Dictionary members that we need to be concerned.

Reverting that part of the change also makes writing the test cases considerably simpler.

See 6842793.