Witness Set Serialization differs between v11.5 and v12a18

[MarcelKlammer]

We ran into a problem with witness set serialization on v12a18, which the node doesn't accept.

v11 (sizes, no break flags):

v12 (unknown array sizes, break flags)

The arrays don't have a size anymore and break flags are added instead. This leads to this kind of error on node 8.7.3:

The transaction failed because it contains datums not associated with any script or output. This could be because you've left some orphan datum behind, because you've listed the wrong inputs in the transaction or because you've just forgotten to include a datum associated with an input. Either way, the field 'data.extraneousDatums' contains a set of data hashes for these extraneous datums.
Data: {"extraneousDatums":["bf000b171d264ef3be80ad6542fdc4eca6697bd2aa6208159decd50c58dace35"]}

Submitting the same tx with a witness set built by v11.5 works.

[lisicky]

Hi @MarcelKlammer ! Could you provide more information to reproduce the issue ? If you will be able to provide code example (for both cases) it would be perfect. Do you submit both tx's to the same network ? I have checked your cbor (from here ) via https://cardananium.github.io/cquisitor/ and I don't see any undefined length arrays.

[MarcelKlammer]

I mixed it up somehow.

v11 has break flags, v12 has not.

Here are two identical transactions:
Eternl in-wallet swap, trading ADA/AXO (via VyFi, built by DexHunter). We are just adding the witness set to that transaction:

I'm using https://cbor.nemo157.com/ to inspect the cbor.

v12a18: (No break flags):

84a80081825820bb8a8a387b9b2e6e71dd646fbf997ffc203fb614eb39d2082cb6fad57a82863700018383581d71ffff54dd7d1b69e8b388f612c6e7df437dee0aabcf35be641f3a31301a0632ea0058202507e0cfd09556bf06238ba4bc15f4f25b6045adc9f3a57dd0444f9e19fbe3a182583901ffebcc9e31749eb5803e396202d84e3b436ec362463b2fd70fb4c8819086fc9117b2dadb43da1f922c46039a47d51bff09433dcdd18f1cce1a000f424082583901dc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e061a7f69149c021a00034449031a070c022b0758203e2c0d4536bdfce2164ac693186827f89793f640e0cbb3a45b319cc2a2415e75081a070bfea70b58207348ae023f4cfee8407bcbfacb57ee038e705c808f51b525c0fe7c7253774e2f0e82581cdc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c581c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e06a2008282582030776dc4d48c1ee2a744a8f48db8bf3de5b810450c13b7e3e7e4fced626304fe58407bba73ef5ffb478226bbfe8601711e2457aa7f4ebe784d07e8b16676e5f3d5fd67a73a6cf38d8d7e906018f7537dbda1a89caef2e4fd502b63574382ced934018258200dbfcbe1762c2c69b93d13d07f5d4421ea5a88ff9ea73cd877e80aaff2b0f51058407c027a0d2ac7df14e2d81052137ea441be67daf047de195c24e80d5b73b7766205872e99c5d821e484dce16817f44eb5dd2ef7f7018dbb9ed15deb61fed482040481d879825838dc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e06d87c811b0000000211201c78f5a11902a2a1636d7367826f44657868756e7465722054726164656e506172746e657220455445524e4c

v11.5: (break flags)

84a80081825820bb8a8a387b9b2e6e71dd646fbf997ffc203fb614eb39d2082cb6fad57a82863700018383581d71ffff54dd7d1b69e8b388f612c6e7df437dee0aabcf35be641f3a31301a0632ea0058202507e0cfd09556bf06238ba4bc15f4f25b6045adc9f3a57dd0444f9e19fbe3a182583901ffebcc9e31749eb5803e396202d84e3b436ec362463b2fd70fb4c8819086fc9117b2dadb43da1f922c46039a47d51bff09433dcdd18f1cce1a000f424082583901dc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e061a7f69149c021a00034449031a070c02f30758203e2c0d4536bdfce2164ac693186827f89793f640e0cbb3a45b319cc2a2415e75081a070bff6f0b58207348ae023f4cfee8407bcbfacb57ee038e705c808f51b525c0fe7c7253774e2f0e82581cdc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c581c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e06a2008282582030776dc4d48c1ee2a744a8f48db8bf3de5b810450c13b7e3e7e4fced626304fe58402455fccf58d37caab27d8958c59e95a13edf75132ffe523f525ca3cb22b7717895f6229bd146469dfa5a702a1728ee8d4389ca72ca447e89ff97cd4c45c90b008258200dbfcbe1762c2c69b93d13d07f5d4421ea5a88ff9ea73cd877e80aaff2b0f51058407755a1ec620392c8bd0d092a66a23145524a206f17b8378fa296453e6c57c92cba3620f97bb07dd4ebab213a5f82d4c703567ade67718389e214edc6ca1c370e049fd8799f5838dc9598479f80e75a442b6dd84cc47096147011535887ddd8e8e4570c70f25ff6fd9adf330eb318ac48689baa721d52ded5cfda2c36540e06d87c9f1b0000000211201c78fffffff5a11902a2a1636d7367826f44657868756e7465722054726164656e506172746e657220455445524e4c

[MarcelKlammer]

Please disregard this issue for now. I might have found the cause, need to debug a bit more.