Passing abfs as fs parameter to UPath fails.

[mat-ej]

Hi fsspec / universal_pathlib team,

I am trying to implement elegant way of storage switching between local and the azure blob.

I would love to have a unified path of the type UPath that simply knows where the object "lives" and acts accordingly.

I am running into the following issue:

When I pass abfs: AzureBlobFileSystem that is already set up with account_name and can correctly list write files.

I pass the abfs to the UPath as follows:

abfs =  AzureBlobFileSystem(...)
abfs.ls('/') # This succeeds

path = UPath('abfs://{foldername}', fs=abfs)

path
AzurePath('abfs://{foldername}/')

path.glob('*') # This fails

I get

ValueError: unable to connect to account for Must provide either a connection_string or account_name with credentials!!

Q: Why is this happening ? My abfs is already "logged in" and authenticated.

I would think that UPath should have no issue simply using such abfs.

[ap--]

Hi @mat-ej

The UPath constructor currently supports storage options as keyword arguments, and there is no direct way of passing instantiated filesystem classes.

So you have two options to achieve what you want:

# you can provide the same options you use for AzureBlobFileSystem to UPath

pth = UPath('abfs://{foldername}', account_name=..., tenant_id=..., client_id=..., client_secret=..., ...)

# you can pass in the storage_options from the already instantiated fs

abfs =  AzureBlobFileSystem(...)
pth = UPath('abfs://{foldername}', **abfs.storage_options)

Let me know if that helps!

Also, if you don't mind me asking: Was there some text in the documentation, that made you assume you can pass fs=filesystem_instance to UPath as a keyword argument? And/or can you suggest some text for the documentation, that would have made this more intuitive?

Cheers,
Andreas

[mat-ej]

Thanks for quick answer, will try with the storage options. I ve seen filesystem arg in documentation that is where my confusion came from

[bollard]

Just to add I came here about ask the same question, but the suggestion of passing the **abfs.storage_options to the constructor worked perfectly in my S3FileSystem case. In terms of documentation, I think adding a note around this would be very helpful