Add support for loading UTF16 encoded files #648
Conversation
souleb
force-pushed
the
support-utf16le
branch
from
298f254
to
aa9c983
Compare
| @@ -49,6 +49,14 @@ func TestScanManifests(t *testing.T) { | |||
| base: "./testdata/nokustomization/panic", | |||
| wantErr: true, | |||
| }, | |||
| { | |||
| name: "utf-16LE encoding", | |||
There was a problem hiding this comment.
Can we give this test a more descriptive name, please? What does it actually test? Something like "Files with UTF-16LE encoding should be included".
There was a problem hiding this comment.
Plus: This tests passes just fine without the changes to kustomize_generator.go.
|
I have tested this with an Well it fails with This is not perfect but it is still better than the actual behaviour where it silently drop those files, because it provides an actionable error. Note that it is a breaking change because running workload containing |
If they have a correct BOM (Byte-Order-Mark https://en.wikipedia.org/wiki/Byte_order_mark) at the beginning of the file, support non utf8 files. Falls back on UTF8 encoding, if no understandable BOM is present. Signed-off-by: Soule BA <[email protected]>
souleb
force-pushed
the
support-utf16le
branch
from
aa9c983
to
253e316
Compare
I manually reverted your changes to |
Signed-off-by: Soule BA <[email protected]>
souleb
force-pushed
the
support-utf16le
branch
from
b0f8b93
to
52f6ede
Compare
Yes you are right, but in practice we have seen that it drops silently those files. I have to dig deeper. Let me put this on draft. |
Add support for loading UTF16 encoded files if they have a correct BOM (Byte-Order-Mark https://en.wikipedia.org/wiki/Byte_order_mark) at the beginning of the file. Falls back on UTF8 encoding, if no understandable BOM is present.
This is dependent on go-yaml support, utf8, utf16LE and utf16BE. see: https://github.com/kubernetes-sigs/kustomize/blob/7c36ed21b3587e7124326743a888db5374e86d93/kyaml/internal/forked/github.com/go-yaml/yaml/readerc.go#L181C4-L181C4