User-supplied credential callback

[kylebarron]

There's a myriad number of ways to handle credentials for each of these stores, and I don't want to be implementing every last one of these. Luckily, object_store allows for external credential providers, and we can allow users to implement their own totally custom authentication in Python!

This is a proof of concept that is tested as working with both a synchronous or asynchronous credential provider! Here are a couple examples:

def credential_provider() -> S3Credential:
    session = boto3.Session(aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, aws_session_token=aws_session_token)
    credentials = session.get_credentials().get_frozen_credentials()
    return {
        "access_key_id": credentials.access_key,
        "secret_access_key": credentials.secret_key,
        "token": credentials.token,
        "timeout": datetime.now() + timedelta(days=1000)
    }

store = S3Store("ds-wheels", credential_provider=credential_provider)
test = obs.list(store).collect()

async def credential_provider() -> S3Credential:
    session = boto3.Session(aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, aws_session_token=aws_session_token)
    credentials = session.get_credentials().get_frozen_credentials()
    return {
        "access_key_id": credentials.access_key,
        "secret_access_key": credentials.secret_key,
        "token": credentials.token,
        "timeout": datetime.now() + timedelta(days=1000)
    }

store = S3Store("ds-wheels", credential_provider=credential_provider)
await obs.list(store).collect_async()

Notes:

• Only try to refresh credentials (i.e. call the Python callback) when necessary. This includes handling the timeout datetime correctly.
• Passing an async callback will hang if you use sync obstore APIs. You need to use async obstore APIs if you pass in an async callback.
• Example with at least S3, maybe aws sts, and maybe azure (planetary computer?)
• Use something like upstream's TokenCache
• Update pickle "advanced" doc for whether this will work in pickle

Closes #232