Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support symlinks and other types in ls #27

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Support symlinks and other types in ls #27

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
958820c
Support symlinks and other types in ls
Erotemic Oct 17, 2023
884147a
Update ipfsspec/core.py
Erotemic Oct 17, 2023
67fcb42
Add more ls types to async code
Erotemic Oct 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion ipfsspec/async_ipfs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,14 @@ async def ls(self, path, session):
res = await self.api_get("ls", session, arg=path)
self._raise_not_found_for_status(res, path)
resdata = await res.json()
types = {1: "directory", 2: "file"}
types = {
0: "raw",
1: "directory",
2: "file",
3: "metadata",
4: "symlink",
5: "shard",
}
return [{
"name": path + "/" + link["Name"],
"CID": link["Hash"],
Expand Down
26 changes: 25 additions & 1 deletion ipfsspec/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@


class IPFSGateway:
"""
Abstrats the IPFS gateway REST API.

Documentation for the REST API can be found here: [Kubo_RPC_API]_.

References:
.. [Kubo_RPC_API] http://docs.ipfs.tech.ipns.localhost:8080/reference/kubo/rpc/
Comment on lines +21 to +24
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess as long as we are talking read only, we might want to stick to the HTTP Gateway API (not the full RPC API). This should always be available on locally running kubo installations, but should work equally well on public gateways as a fallback.

"""
def __init__(self, url):
self.url = url
self.state = "unknown"
Expand Down Expand Up @@ -166,7 +174,23 @@ def ls(self, path, detail=True, **kwargs):
logger.debug("ls on %s", path)
res = self._gw_apipost("ls", arg=path)
links = res["Objects"][0]["Links"]
types = {1: "directory", 2: "file"}

# Documentation for the type codes doesn't seem to have a great location.
# Currently 2023-10-17, the ls API docs specify the ls return
# structure, but not what the type integers mean:
# http://docs.ipfs.tech/reference/kubo/rpc/#api-v0-ls
# Some information about type codes can be found here:
# https://ipfs-search.readthedocs.io/en/latest/ipfs_datatypes.html
# https://github.com/ipfs/go-unixfs/blob/master/pb/unixfs.proto

types = {
0: "raw",
1: "directory",
2: "file",
3: "metadata",
4: "symlink",
5: "shard",
}
Comment on lines +186 to +193
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will also touch what the generic fsspec API expects as a return form ls (and info). I don't know a definitve answer here, but maybe @martindurant could chime in. If I look at e.g. the local filesystem implementation, there are only types file, directory and other. There's additionally the islink flag.
So @martindurant, would raw, metadata, symlink and shard be valid return values for the type? And more specifically, what would be the best return value to communicate the presence of a symlink?

if detail:
return [{"name": path + "/" + link["Name"],
"size": link["Size"],
Expand Down