New data source: azurerm_storage_blob_content

[edward-milkey]

Community Note

• Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
• Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for PR followers and do not help prioritize for review

Description

This PR proposes adding a new data source to retrieve the content of a blob in a storage account. It would be useful for us to be able to access configuration data stored in our blobs. Our data is small and text-only so it should be OK to be stored in state.

Using a separate data source to access blob content seems to mirror how the GCP provider does it (separate data source for object and object content). It also avoids changing the behavior of the existing azurerm_storage_blob data source for users that do not need the blob content.

The ability to access blob content was requested in #18779. This PR attempts to provide an easy way to retrieve blob content for users that need it.

Thank you for your consideration.

PR Checklist

• I have followed the guidelines in our Contributing Documentation.
• I have checked to ensure there aren't other open Pull Requests for the same update/change.
• I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
• I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
• I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
  For example: “resource_name_here - description of change e.g. adding property new_property_name_here”

Testing

• My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

❯ make acctests SERVICE="storage" TESTARGS="-run=TestAccDataSourceStorageBlobContent_basic" TESTTIMEOUT="10m"
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
TF_ACC=1 go test -v ./internal/services/storage -run=TestAccDataSourceStorageBlobContent_basic -timeout 10m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccDataSourceStorageBlobContent_basic
=== PAUSE TestAccDataSourceStorageBlobContent_basic
=== CONT  TestAccDataSourceStorageBlobContent_basic
--- PASS: TestAccDataSourceStorageBlobContent_basic (123.64s)
PASS
ok  	github.com/hashicorp/terraform-provider-azurerm/internal/services/storage	127.487s

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

• New Data Source: azurerm_storage_blob_content - new data source to retrieve storage blob content [GH-00000]

This is a (please select all that apply):

• Bug Fix
• New Feature (ie adding a service, resource, or data source)
• Enhancement
• Breaking Change

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

[stephybun]

Thanks for this PR @edward-milkey!

As mentioned in the initial request for this, we have concerns around allowing users to store blob contents in their state. After another round of discussion on this we think that with appropriate safe guards most of these concerns can be mitigated, so we should be able to support this.

In order to move this forward could you please take a look at the following points:

• We aren't accepting untyped data sources or resources in the provider anymore, could you re-write this as a typed data source. This contributor guide on how to add a new typed data source can help you get started
• In order to prevent users from breaking their state file or causing the agent/job they're running Terraform in from crashing due to OOM it would be good to ascertain the size of the blob first before attempting to retrieve and store it in state. What this limit should be and whether it should be configurable by the user is still TBD, for the moment though feel free to set a limit that you think is appropriate and we can revisit this in a subsequent review
• The contents should be base64 encoded before storing into state
• We should have appropriate notes/warnings in the documentation that users could inadvertently store sensitive information in their state when using this data source

Let me know what you think and if you have any questions or concerns 🙂

[edward-milkey]

hi @stephybun thank you for considering this PR and for the direction you provided in your comment. I've made some updates that I think address the points you raised. For the blob size limit, I chose 2MiB as that comfortably exceeds my use case; if you have concerns with that, please let me know.