-
Notifications
You must be signed in to change notification settings - Fork 232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
IPIP-425: Signaling Features on HTTP Gateways #425
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -542,6 +542,46 @@ Optional, present in certain response types: | |||||||||||
non-executable binary response types are not used in `<script>` and `<style>` | ||||||||||||
HTML tags. | ||||||||||||
|
||||||||||||
### `Ipfs-Gateway-Features` (response header) | ||||||||||||
|
||||||||||||
Optional, this header SHOULD be only returned in response to HTTP `OPTIONS` request. | ||||||||||||
|
||||||||||||
The value is a list of key-value pairs, as specified by the 5.6.1. Lists section of :cite[rfc9110]. | ||||||||||||
|
||||||||||||
Each feature is indicated by a key and optional value. When more than one value is supported, `|` is used as a separator. | ||||||||||||
For example: | ||||||||||||
|
||||||||||||
``` | ||||||||||||
Ipfs-Gateway-Features: foo, bar=1, buzz=a|b|c | ||||||||||||
``` | ||||||||||||
|
||||||||||||
A Gateway SHOULD use this header to communicate support for specific Gateway feeatures, enabling clients to make better decisions on how to retrieve data. | ||||||||||||
|
||||||||||||
A Gateway MAY define and return their own features. | ||||||||||||
|
||||||||||||
A Client MUST send HTTP OPTIONS request to inspect this header before performaning any more expensive feature-detection. | ||||||||||||
|
||||||||||||
#### Canonical Ipfs-Features values | ||||||||||||
|
||||||||||||
- `trustless-gateway` for :cite[trustless-gateway] | ||||||||||||
- `path-proof` indicates support for returning parent blocks up to the terminus element | ||||||||||||
Comment on lines
+566
to
+567
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
- `car-version=1|2` indicates CAR support | ||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
we should strictly do v1 for now, v2 doesn't make sense for this (yet), but it might be worth signalling this so we can also add |
||||||||||||
- `dag-scope=block|entity|all` from :cite[ipip-0402] | ||||||||||||
- `entity-bytes` from :cite[ipip-0402], implies support for `dag-scope=entity` as well | ||||||||||||
Comment on lines
+566
to
+570
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @hannahhoward @alanshaw @olizilla I know you want a way for signaling support for partial cars and range requests between Lassie and .storage to avoid expensive feature-sniffing. With this IPIP, signaling support for partial cars WITHOUT range requests woudl look like this: Ipfs-Gateway-Features: dag-scope=block|entity|all and if you add support for Ipfs-Gateway-Features: dag-scope=block|entity|all, entity-bytes |
||||||||||||
- `car-block-order=dfs` from :cite[ipip-0412] | ||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
- `car-block-dupes=y|n` from :cite[ipip-0412] | ||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
- `path-gateway` for deserialized responses defined by :cite[path-gateway] | ||||||||||||
- `subdomain-gateway=example.com` for :cite[subdomain-gateway] support based on `Host` header | ||||||||||||
- `dnslink-gateway` for :cite[dnslink-gateway] support based on `Host` header | ||||||||||||
- `ipns` indicating :cite[ipns-record] support on `/ipns/` content paths | ||||||||||||
- `dnslink` indicating [DNSLink](https://dnslink.dev) support on `/ipns/` content paths | ||||||||||||
|
||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another thing that could be useful is max block size:
Suggested change
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another thing we could signal: if gateway is recursive (fetching content if not available locally) or not (cc @aschmahmann @Jorropo) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. good idea but hold off until we have a real use-case for it reckon - unless you can conjure one now? |
||||||||||||
<!-- TODO do we want these too? | ||||||||||||
- `multibase` list of prefixes, indicates which multibase encoding are supported in CIDs | ||||||||||||
- `multihash` indicates which hash functions are supported in CIDs | ||||||||||||
- `multicodec` indicates which codecs are supported in CIDs | ||||||||||||
--> | ||||||||||||
Comment on lines
+579
to
+583
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ❓ Thoughts on adding these or even more? The need for list of supported IPLD codecs was raised by ipfs-chromium and Capyloon in #402 (comment), having the other two would not hurt too. My only constraint would be to use numeric (decimal or hex) codes, to avoid problems if we ever do this again. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. cc @aschmahmann as you noted in 402:
Would below work? is there a better way of representing this? multihash=0x12|0x13|etc # identity + goodset from https://github.com/ipfs/boxo/blob/cfad09d7156efa2f09822d620cacb2423d884067/verifcid/validate.go#L17
multicodec=0x51|0x55|0x70|0x71|0x72|0x0129|0x0200 # based on boxo/gateway There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. too much information, let's start simple for now |
||||||||||||
|
||||||||||||
### `Server-Timing` (response header) | ||||||||||||
|
||||||||||||
Optional. Implementations MAY use this header to communicate one or more | ||||||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
--- | ||
title: "IPIP-0425: Signaling Features on HTTP Gateways" | ||
date: 2023-07-06 | ||
ipip: proposal | ||
editors: | ||
- name: Marcin Rataj | ||
github: lidel | ||
url: https://lidel.org/ | ||
relatedIssues: | ||
- https://github.com/ipfs/specs/pull/402#pullrequestreview-1396116569 | ||
- https://github.com/ipfs/specs/pull/412#pullrequestreview-1427137365 | ||
order: 425 | ||
tags: ['ipips'] | ||
--- | ||
|
||
## Summary | ||
|
||
Add ability to query HTTP Gateway for an explicit list of supported features. | ||
|
||
## Motivation | ||
|
||
A Gateway always ships with an opinionated set of supported hash functions and | ||
IPLD codecs, and the differences between implementations will grow over time. | ||
|
||
For example, some legacy gateways may not support newly added features like | ||
`dag-scope` and `entity-bytes` from :cite[ipip-0402] or the ability to get some | ||
block ordering guarantees introduced in :cite[ipip-0412]. Future IPIPs may add | ||
more features and response formats. | ||
|
||
We need a light mechanism for clients to detect which gateway supports partial CARs | ||
|
||
## Detailed design | ||
|
||
This IPIP introduces a set of HTTP headers returned in response to `OPTIONS` request: | ||
|
||
The `Ipfs-Gateway-Features` header is used for signalling support for specific Gateway features to the client. | ||
|
||
The lack of the header, or missing key-value pair within the header means support status is unknown. | ||
|
||
Initial list of key-value pairs is documented in `Ipfs-Gateway-Features` section of :cite[path-gateway] | ||
|
||
## Design rationale | ||
|
||
There is a good prior art for this in web browsers where HTTP `OPTIONS` method | ||
is used in [CORS Preflight request](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) | ||
that checks if the CORS protocol is understood and a server is aware using | ||
specific methods and headers. | ||
|
||
The `OPTIONS` request if often sent by web browser anyway, so we would not be | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume this oten a heavily cachable response so we'd be hitting the browser cache. |
||
introducing much overhead. | ||
|
||
### User benefit | ||
|
||
Reduced cost for both client and gateway. Sending `OPTIONS` request will be | ||
very inexpensive, especially when compared with current status quo where a | ||
client has to send at least one request to probe a specific feature like | ||
support forp Blake3 hash function or `dag-scope` or `entity-bytes`. | ||
|
||
This translates to decreased latency and ability to choose the best retrieval | ||
strategy faster. | ||
|
||
### Compatibility | ||
|
||
This IPIP is fully backward-compatible with browsers and existing IPFS | ||
ecosystem. Gateways already return CORS headers with `OPTIONS` responses, we | ||
will simply return additional headers with the same responses. | ||
|
||
:::issue | ||
|
||
For JavaScript running on web pages to be able to read `Ipfs-Gateway-Features` | ||
header it MUST be safelisted via `Access-Control-Expose-Headers`. | ||
|
||
::: | ||
|
||
### Security | ||
|
||
This IPIP does not introduce any new security concerns. Probing gateway for | ||
supported features and hash functions is already possible via regular `GET` | ||
requests. | ||
|
||
### Alternatives | ||
|
||
- Exposing the list of suported features via `GET /.well-known/ipfs/gateway/features-TBD` would also work, but: | ||
- introduces surface for path-related deployment bugs, where Nginx is only exposing `/ipfs` namespace – in such case signaling endpoint would not be exposed to the public internet | ||
|
||
## Test fixtures | ||
|
||
TODO | ||
|
||
### Copyright | ||
|
||
Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(just tuning into this PR now that I care about signalling, sorry!)
Whatever this header is, I think it should also be returned when a server doesn't implement a feature requested, either via
Accept
header or other means and in that case the server should return a415
. That way I can parse the list ofAccept
content types, and be strict about the parameters, and if I don't find one that I fully support, return a415
with this signal that says what I do support.That way, a client can optimistically make a request, but be prepared to downgrade if it doesn't get what it wants. Rather than needing to
OPTIONS
first, thereby saving a round-trip.