Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Observing DynamoDB Table rangeKey throws Unused attributes error #1597

Open
1 task done
chris-believer opened this issue Dec 11, 2024 · 1 comment · May be fixed by crossplane/upjet#461
Open
1 task done

[Bug]: Observing DynamoDB Table rangeKey throws Unused attributes error #1597

chris-believer opened this issue Dec 11, 2024 · 1 comment · May be fixed by crossplane/upjet#461
Labels
bug Something isn't working is:triaged Indicates that an issue has been reviewed.

Comments

@chris-believer
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Affected Resource(s)

  • dynamodb.aws.upbound.io/v1beta2 - Table

Resource MRs required to reproduce the bug

apiVersion: dynamodb.aws.upbound.io/v1beta2
kind: Table
metadata:
  name: example
spec:
  forProvider:
    attribute:
      - name: UserId
        type: S
      - name: GameTitle
        type: S
    billingMode: PAY_PER_REQUEST
    hashKey: UserId
    rangeKey: GameTitle
    region: us-west-2
  deletionPolicy: Orphan

apiVersion: dynamodb.aws.upbound.io/v1beta2
kind: Table
metadata:
  name: example
  annotations:
    crossplane.io/external-name: example
spec:
  forProvider:
    region: us-west-2
  managementPolicies: ["Observe"]

Steps to Reproduce

  1. Create Table with a rangeKey. This can be externally created or an orphaned crossplane created table after deletion from kubernetes.
  2. Add crossplane Table with managementPolicies:["Observe"] and the existing external-name set.

What happened?

observe failed: cannot compute the instance diff: failed to get *terraform.InstanceDiff: all attributes must be indexed. Unused attributes: ["GameTitle"]

Relevant Error Output Snippet

2024-12-11T17:32:19Z    DEBUG    provider-aws    Cannot observe external resource    {"controller": "managed/dynamodb.aws.upbound.io/v1beta1, kind=table", "request": {"name":"example"}, "uid": "58d1d8e2-7f7a-438c-8582-1ebdbe427619", "version": "56239723", "external-name": 
"example", "error": "cannot compute the instance diff: failed to get *terraform.InstanceDiff: all attributes must be indexed. Unused attributes: [\"GameTitle\"]", "errorVerbose": "all attributes must be indexed. Unused attributes: [\"GameTitle\"]\nfailed to get *terraform.
InstanceDiff\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).getResourceDataDiff\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:421\ngithub.com/crossplane/upjet/pkg/controller.(*terraform
PluginSDKExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:495\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Observe\n\tgithub.com/crossplane/[email protected]
184956-3afbb7796d46/pkg/controller/external_async_tfpluginsdk.go:127\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/reconciler/managed/recon
ciler.go:914\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*
Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:
316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s
.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_amd64.s:1700\ncannot compute the instance diff\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).Observe\n\tgithub.com/crossplane/[email protected].
20240911184956-3afbb7796d46/pkg/controller/external_tfpluginsdk.go:497\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_async_tfpluginsdk.go:12
7\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/reconciler/managed/reconciler.go:914\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimi
ter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg
/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Contr
oller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.g
o:227\nruntime.goexit\n\truntime/asm_amd64.s:1700"}

Crossplane Version

1.18.0

Provider Version

1.17.0

Kubernetes Version

v1.30.6-eks-7f9249a

Kubernetes Distribution

EKS

Additional Info

  • I attempted to downgrade the dynamodb provider down to 1.15 with no success.
  • The same error occurs on rangeKey elements in a localSecondaryIndex, but pulled them out during troubleshooting to get a smaller reproduction. The rangeKey attributes are also marked as Unused. I did not test globalSecondaryIndex but suspect it would error as well.
@chris-believer chris-believer added bug Something isn't working needs:triage labels Dec 11, 2024
@turkenf
Copy link
Collaborator

turkenf commented Jan 2, 2025

Hi @chris-believer,

Thank you for your clear and detailed issue report. I was able to reproduce the issue on my side as well:

2025-01-02T19:48:31Z	DEBUG	provider-aws	Cannot observe external resource	{"controller": "managed/dynamodb.aws.upbound.io/v1beta1, kind=table", "request": {"name":"ft-test"}, "uid": "39d4f0a0-5766-4a5d-9960-75a869360951", "version": "2855", "external-name": "ft-test", "error": "cannot compute the instance diff: failed to get *terraform.InstanceDiff: all attributes must be indexed. Unused attributes: [\"GameTitle\" \"TopScore\"]", "errorVerbose": "all attributes must be indexed. Unused attributes: [\"GameTitle\" \"TopScore\"]\nfailed to get *terraform.InstanceDiff\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).getResourceDataDiff\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:421\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:495\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_async_tfpluginsdk.go:127\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/reconciler/managed/reconciler.go:914\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1222\ncannot compute the instance diff\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_tfpluginsdk.go:497\ngithub.com/crossplane/upjet/pkg/controller.(*terraformPluginSDKAsyncExternal).Observe\n\tgithub.com/crossplane/[email protected]/pkg/controller/external_async_tfpluginsdk.go:127\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/reconciler/managed/reconciler.go:914\ngithub.com/crossplane/crossplane-runtime/pkg/ratelimiter.(*Reconciler).Reconcile\n\tgithub.com/crossplane/[email protected]/pkg/ratelimiter/reconciler.go:54\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227\nruntime.goexit\n\truntime/asm_arm64.s:1222"}

This seems like something we need to investigate further. In the meantime, as a potential workaround, could you try importing the resource using the following approach?

apiVersion: dynamodb.aws.upbound.io/v1beta2
kind: Table
metadata:
  name: example
  annotations:
    crossplane.io/external-name: example
spec:
  forProvider:
    region: us-west-2
    globalSecondaryIndex:
    - hashKey: GameTitle
      name: GameTitleIndex
      nonKeyAttributes:
      - UserId
      projectionType: INCLUDE
  managementPolicies: ["Observe"]

@turkenf turkenf added is:triaged Indicates that an issue has been reviewed. and removed needs:triage labels Jan 2, 2025
@mergenci mergenci linked a pull request Jan 16, 2025 that will close this issue
Draft
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working is:triaged Indicates that an issue has been reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove diff calculation in observe-only reconciliation mergenci/upjet
2 participants
@turkenf @chris-believer