Quick start fails for institutional s3: Repository create

[wachsylon]

What happened?

Getting the storage iss works, but creating the repo fails:

iss = icechunk.s3_storage(
    bucket="bm1344",
    prefix="icechunk-test",
    access_key_id=ACCESS_KEY,
    secret_access_key=SECRETE,
    region="eu-drkz-1",
    endpoint_url="https://s3.eu-dkrz-1.dkrz.cloud"
)
repo = icechunk.Repository.create(iss)

runs into

IcechunkError                             Traceback (most recent call last)
Cell In[10], line 1
----> 1 repo = icechunk.Repository.create(iss)

File /work/bm1344/conda-envs/icezarr/lib/python3.12/site-packages/icechunk/repository.py:51, in Repository.create(cls, storage, config, virtual_chunk_credentials)
     25 @classmethod
     26 def create(
     27     cls,
   (...)     30     virtual_chunk_credentials: dict[str, AnyCredential] | None = None,
     31 ) -> Self:
     32     """
     33     Create a new Icechunk repository.
     34     If one already exists at the given store location, an error will be raised.
   (...)     48         An instance of the Repository class.
     49     """
     50     return cls(
---> 51         PyRepository.create(
     52             storage,
     53             config=config,
     54             virtual_chunk_credentials=virtual_chunk_credentials,
     55         )
     56     )

IcechunkError:   x error listing objects in object store service error
  | 
  | context:
  |    0: icechunk::repository::create
  |              at icechunk/src/repository.rs:140
  | 
  |-> error listing objects in object store service error
  |-> service error
  |-> unhandled error
  `-> error parsing XML: encountered invalid XML root: expected ListBucketResult but got StartEl { name: Name { prefix: "", local: "ListAllMyBucketsResult" }, attributes: [Attr { name: Name
      { prefix: "", local: "xmlns" }, value: "http://s3.amazonaws.com/doc/2006-03-01/" }], closed: false, depth: 0 }. This is likely a bug in the SDK.

fetch the config runs into:

icechunk.Repository.fetch_config(iss)
---------------------------------------------------------------------------
IcechunkError                             Traceback (most recent call last)
Cell In[11], line 1
----> 1 icechunk.Repository.fetch_config(iss)

File /work/bm1344/conda-envs/icezarr/lib/python3.12/site-packages/icechunk/repository.py:168, in Repository.fetch_config(storage)
    153 @staticmethod
    154 def fetch_config(storage: Storage) -> RepositoryConfig | None:
    155     """
    156     Fetch the configuration for the repository saved in storage.
    157 
   (...)    166         The repository configuration if it exists, None otherwise.
    167     """
--> 168     return PyRepository.fetch_config(storage)

IcechunkError:   x error getting object from object store service error
  | 
  | context:
  |    0: icechunk::storage::s3::fetch_config
  |              at icechunk/src/storage/s3.rs:267
  |    1: icechunk::repository::fetch_config
  |              at icechunk/src/repository.rs:346
  | 
  |-> error getting object from object store service error
  |-> service error
  |-> unhandled error (AuthorizationHeaderMalformed)
  `-> Error { code: "AuthorizationHeaderMalformed", message: "The authorization header is malformed; the region is wrong; expecting 'eu-dkrz-1'.", s3_extended_request_id:
      "5bc9c93fe38a0584af783ef1dd1161a23d70a7c62ebdf3dfedc8f8b61d1ab121", aws_request_id: "182A36C0020179A5" }

What did you expect to happen?

That it works

Minimal Complete Verifiable Example

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example runs when copied & pasted into an fresh python environment.
• New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Anything else we need to know?

No response

Environment

internalplatform: Linux-4.18.0-513.24.1.el8_9.x86_64-x86_64-with-glibc2.28
python: 3.12.9
icechunk: 0.2.5
zarr: 3.0.4
numcodecs: 0.15.1
xarray: 2025.1.2
virtualizarr: 1.3.2

[rabernat]

Welcome @wachsylon and sorry for the friction!

Can you share more details about the object storage you're using? (Vendor, platform, etc.)

This issue is similar to #743 in that it involves an apparent incompatibility between the AWS SDK and the object store's behavior.

[rabernat]

Ah ok I read the error message more carefully.

There is a typo in your region argument. Can you spot it? 👀

[wachsylon]

Thanks a lot!

How did you even find that... my bad.

I needed to add a allow_http and then I got no more errors. It also created some objects, so I am proceeding!