[CFF2] uint16 VariationStore.length limitation

[behdad]

Problem statement

The VariationStore.length field in the current CFF2 specification has the type uint16. We have seen complex varfont designs hit this size limitation.

Context

64k ought to be enough, for everybody, right?

Not only it would have been trivial (at the cost of one or two bytes!) to avoid imposing this limitation, as it happens, encoding of the said value itself is completely unnecessary!

See: the processor does not need to know the byte size of the embedded ItemVariationStore. The ItemVariationStore is designed in the style of the GSUB / GPOS / etc tables, where lots of small tables linked together through offsets are used. The CFF / CFF2 tables on the other hand, use either a stack machine (DICT, CharString), or a list of sized data (INDEX), in both cases the length of the payload is encoded explicitly, by necessity. So the people designing how to embed ItemVariationStore in CFF2 decided to encode the byte-length of the structure in there some idea of consistency I suppose.

Background and design notes

It was chosen for CFF2 to encode the actual delta values inline with the CharString as operands to the blend operator. To specify the master configuration however, it was decided to reuse the ItemVariationStore facilities, but not store any actual deltas in there. So, it will include only the master configuration. So somebody thought that cannot need too many bytes and set it to 16 bits instead of eg. 32.

It was initially considered that such ItemVariationStore be added as a new font table, side by side to the CFF2 table itself. It was, however, pointed out that there is no reason to do so, and the variation-store is just yet another operator to add to TopDict as an offset to a ItemVariationStore structure. Did you notice the use of two names: ItemVariationStore and VariationStore. The latter (VariationStore) is a CFF2-specific unnecessary wrapper around the spec-wide ItemVariationStore, just prefixing the actual data with the length of it. Encoding the length is not necessary for any kind of processing whatsoever.

I also want to note how the designers chose to encode such length as a uint16, instead of using a variable-sized operand encoding like those used in TopDICT itself. All numbers in CFF are encoded as variable-sized. If a variable-sized number was used, there would have been no limitation. Alas.

Proposal

Somehow mark this number irrelevant and unused. The way I suggest, and seems like Adobe team agrees as well, is to spec that if the length value is 65,535, it must be ignored and the actual payload might be of any size.

Risks

There is no risk in this change since we are just making more fonts build than that currently possible. Any existing font will continue to behave the same.

It is very likely that no rendering engine has any meaningful check of this number. Even tools like ot-sanitize check for this length field to match the actual payload byte size.

[skef]

I entirely agree this should be fixed. We've also discussed the specific fix you suggest (based on a previous warning about this) and agree that's the way to go.