identity: add Resolver interface; and SkipHandleVerification flag on BaseDirectory

[bnewbold]

NOTE: see comments below; the PR has been updated to be about a new identity.Resolver interface, instead of expanding identity.Directory

This is a step in the right direction for this package, I think:

• Lookup* for full bi-di verification, and gives a compact/ergonomic struct for atproto use-cases (and good for caching)
• Resolve* does more bare-bones resolution: you get specifically what you ask for. Things like Relay, or pure service auth validation, which don't care about handles, can use ResolveDID

Downsides and open questions:

• ResolveDID doesn't return the raw DID doc verbatim, it is a struct, and might be discarding info (like fields in DID core specification that we aren't using). I think this is good enough, maybe BaseDirectory can expose more "Raw" variants which return json.RawMessage or map[string]any or something.
• the struct returned by ResolveDID isn't the same Identity type used in possible function signatures. They can be converted, but it uses some allocs etc. I think adding a helper to quickly extract the DID signing key and/or PDS endpoint is probably sufficient? But feels a bit redundant with Identity struct
• caching: do we cache one of the Identity struct (compact) or the DIDDocument struct, and convert between? Or both, would be less conversions but more RAM? I think what we should do is cache what is actually requested. Lookup* caches Identity, ResolveDID caches DIDDocument, and these caches aren't shared. Services will be using one or the other anyways. What about an identity caching layer? That could cache the raw DID document and handle resolution, and convert to Identity on demand.

Alternative would be breaking changes to existing structs (which are heavily cached), or maybe a flag or variant like LookupDIDWithoutHandle?

Another alternative is to set a flag to skip handle resolution; I don't like this: #854

An upcoming change here is to add ResolveNSID, which will be used for Lexicon resolution.

(have not pushed implementation of exiting interfaces until design questions are resolved)

[ericvolp12]

I don't mind the idea of two caches here, I think your assumption that a caller will likely be using mostly one type of method (Resolve vs Lookup) is reasonable. The naming is a bit ambiguous imo but I think it makes sense to maintain compatibility with the existing APIs and it's fine.

I do think the "Resolve" method should get something that contains the entire DID doc as it's a "low level" API here compared to the "Lookup" API.

[bnewbold]

The motivation for this has bumped up in a couple places:

• the identity directory services should cache and return full/raw DID documents, as resolved (not just the subset parsed in to identity.DIDDocument struct, or just identity.Identity subset
• services like Discover need fast DID lookup for auth, and efficient identity caching, and don't care about handles
• many firehose consumers, including new relay, start with a DID and need to verify signatures, but don't care about handles
• as network scales, we are doing more and more identity lookups and caching. it is important to not need to re-parse things endlessly; the Identity struct helps with that. but we still sometimes need the original/raw DID documents as well

It felt like this PR wasn't hitting all the use-cases the way I wanted, so I refactored a bit. The changes are:

• create a new identity.Resolver interface for the ResolveHandle() and ResolveDID() methods. this separates concerns: not every identity.Directory needs to implement these
• pull in a SkipHandleVerification flag on BaseDirectory (from identity: hack flag to skip handle resolution #854), for the "directory API, but just DIDs not handles" use-case

Possible additional changes:

• rename BaseDirectory to ResolvingDirectory to clarify what it is doing, and that it fulfills both interfaces
• could include ResolveNSID in the Resolver interface declaration. on the other hand, this might still move back to the atproto/lexicon package? hrm

[brianolson]

this is ResolveDID not a 'variant'? the think-o was that this is the variant of resolveDIDBytes() that does unmarshal? Probably:

// ResolveDID fetches a DIDDocument
// No further resolution (e.g. handle resolution) within the DIDDocument is performed

[bnewbold]

good catch! mangled copy/paste

[brianolson]

one comment oddity, otherwise LGTM; I like the no-handle mode (-: