configMapGenarator merge behaviour only allows for one override, ignores subsequent overrides

[Gekalash]

What happened?

configMapGenarator merge behaviour changed between Kustomize v4 and Kustomize v5.
In v4, multiple overrides could be applied to the same value in the configMapGenerator, with the last one applied becoming the final value.
In v5, only the first override is applied, and any subsequent ones are ignored.

What did you expect to happen?

Expected compatible manifests to produce the same results when rendered with Kustomize v4 and v5.

How can we reproduce it (as minimally and precisely as possible)?

# base/kustomization.yaml
configMapGenerator:
  - name: literals-configmap
    literals:
      - literal_variable_one=base-value

# component/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

configMapGenerator:
  - name: literals-configmap
    behavior: merge
    literals:
      - literal_variable_one=component-value
      - literal_variable_two=component-value-two

# overlay/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - ../base

components:
  - ../component

configMapGenerator:
  - name: literals-configmap
    behavior: merge
    literals:
      - literal_variable_one=overlay-value
      - literal_variable_two=overlay-value-two

Expected output

apiVersion: v1
data:
  literal_variable_one: overlay-value
  literal_variable_two: overlay-value-two
kind: ConfigMap
metadata:
  name: literals-configmap-6t592hc2f7

Actual output

apiVersion: v1
data:
  literal_variable_one: component-value
  literal_variable_two: component-value-two
kind: ConfigMap
metadata:
  name: literals-configmap-b8d9m4h5k2

Kustomize version

5.1.0

Operating system

MacOS

[valaparthvi]

/assign valaparthvi

[natasha41575]

Thanks @valaparthvi! Please see if you can reproduce the issue by comparing the output of kustomize build on v4 vs v5.

[annasong20]

I suspect this has to do with #5170. merge behavior is probably the same, but components is run after configMapGenerator. If @Gekalash cannot workaround this order change, please leave your feedback at #5170.

@valaparthvi please continue to try to reproduce.

[Gekalash]

A workaround could be – implement the overrides from the conifgMapGenerator as a component, and import it that way, but that's clunky in my particular situation where I have an assortment of overlays that share many things in common, and I only override a thing or two differently for each overlay. So this workaround strategy would imply creating a large number of use-only-once components. That is, undesirable.
I'll see if I can make a PR with a test case that reproduces this issue in an automated way

[valaparthvi]

I can reproduce this.

$ tree                            
.
├── base
│   └── kustomization.yaml
├── component
│   └── kustomization.yaml
└── overlay
    └── kustomization.yaml

3 directories, 3 files

Kustomize 4 output

$ kustomize4 build ./overlay            
apiVersion: v1
data:
  literal_variable_one: overlay-value
  literal_variable_two: overlay-value-two
kind: ConfigMap
metadata:
  name: literals-configmap-6t592hc2f7

Kustomize 5 output

$ kustomize5 build ./overlay  
apiVersion: v1
data:
  literal_variable_one: component-value
  literal_variable_two: component-value-two
kind: ConfigMap
metadata:
  name: literals-configmap-b8d9m4h5k2

[annasong20]

Hi @Gekalash, we have confirmed that the issues is not in the configMapGenerator field.

We tried to run your example with only the configMapGenerator in the overlay, and that merged correctly. Next, as @valaparthvi confirmed, with both the components and configMapGenerator calls in the overlay, the components merge values won in the end because components are now executed after generators and transformers.

This was documented in this Kustomize release. As stated there, #4402 should solve your use case because you'll be able to specify the order that Kustomize fields run in. However, if its implementation is not fast enough for your use case, we invite you to leave feedback on #5141. If there is enough demand there, we'd be interested in coming up with a temporary solution.

Regarding your proposal

A workaround could be – implement the overrides from the conifgMapGenerator as a component, and import it that way

could you provide us with more information? If you want the override values from the overlay configMapGenerator, we're wondering why you can't just leave out the Component. Is it because the Component contains other relevant changes? Some more context could be helpful.

/kind duplicate
/triage needs-information

[k8s-ci-robot]

@annasong20: The label(s) kind/duplicate cannot be applied, because the repository doesn't have them.

In response to this:

Hi @Gekalash, we have confirmed that the issues is not in the configMapGenerator field.

We tried to run your example with only the configMapGenerator in the overlay, and that merged correctly. Next, as @valaparthvi confirmed, with both the components and configMapGenerator calls in the overlay, the components merge values won in the end because components are now executed after generators and transformers.

This was documented in this Kustomize release. As stated there, #4402 should solve your use case because you'll be able to specify the order that Kustomize fields run in. However, if its implementation is not fast enough for your use case, we invite you to leave feedback on #5141. If there is enough demand there, we'd be interested in coming up with a temporary solution.

Regarding your proposal

A workaround could be – implement the overrides from the conifgMapGenerator as a component, and import it that way

could you provide us with more information? If you want the override values from the overlay configMapGenerator, we're wondering why you can't just leave out the Component. Is it because the Component contains other relevant changes? Some more context could be helpful.

/kind duplicate
/triage needs-information

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[annasong20]

/kind support
/triage duplicate

[natasha41575]

Let us know if you have any more information about your use case. Closing this as we are tracking the component/generator ordering in other issues.