Want to use bc-envelope-rust but unsure about seed/key management

[danpape]

Hi - I've got an idea for a nice little app and I want to use bc-envelope-rust so I can use Gordian Envelope tech to secure my data transmission and storage. I've been looking through docs and example code, but I'm finding I'm still confused about how to manage my seeds and keys. For instance, let's say I use seedtool to make me a new seed and output it to a file:

> seedtool -o envelope --name "My Test Seed" --date now --note "Test seed for key generation" | tee my-test-seed
ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte

I could then do the following in my rust code to load the seed and generate private and public keys:

let seed_bytes = read_file_to_string(seed_file_path)?;
let private_key = PrivateKeyBase::from_data(seed_bytes);
let public_key = private_key.schnorr_public_key_base();

and then I can take my data, put it in an Envelope, encrypt, sign, all that good stuff.

But--how do I share the public keys with someone else who will want to verify and decrypt that data? (And how to get their public key to encrypt to them?) I don't see anything obvious in bc-envelope-rust or bc-components-rust for easily saving out the public_key variable. Well, I do see that many of the key types in bc-components-rust have from_data(...) and data() functions, so perhaps I need to try messing around with those to save and load the key data.

Alternately, I could use the envelope cli tool to generate me some keys:

> envelope generate prvkeys | tee my-test-prvkeys 
ur:crypto-prvkeys/hdcxvssgwltohhasjskbskptnlkgtncfosnyhyktwzpfoshkotmuhdgapdverkctbddmbnmonlfn

> envelope generate pubkeys $(cat my-test-prvkeys) | tee my-test-pubkeys
ur:crypto-pubkeys/lftanshfhdcxlsmdiotoimwsnliopdfzheuyenbkbdsngwbboxeckpenjlemgymuhhsrasrpwfmstansgrhdcxwsbygmfhlgrtdnosfhgshsgtpkwzztmoclfyrtnnnsfmpemesgfwiopsfyjtcmgoldfwttcy

but then I'm not sure how to load the ur:crypto-prvkeys/... or ur:crypto-pubkeys/... files into PrivateKeyBase or PublicKeyBase. Maybe 'from_data(...)' will also work?

Finally, I'll note that seedtool -o envelope generates ur:envelope/... data, but if I try to feed that into envelope generate prvkeys --seed ... I get an error:

> envelope generate prvkeys --seed $(cat my-test-seed)                                                          
Error: expected UR type seed, but found envelope

and I'm not sure how to convert a ur:envelope/... into a ur:seed/... even though that's what's in my envelope. I don't see "seed" as a subject type for envelope extract.

thanks!

[wolfmcnally]

@ChristopherA @shannona

Hi, Dan!

Here’s a little walkthrough that will hopefully answer a number of your questions.

OK, let’s start by looking at what’s in your envelope using the command line tool:

~ envelope format ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte 
Bytes(16) [
    'isA': 'Seed'
    'date': 2024-10-18T01:09:59Z
    'hasName': "My Test Seed"
    'note': "Test seed for key generation"
]

This printout is in envelope notation, and it shows that the seed itself is the subject of the envelope (a 16-length byte string), and that subject has four assertions on it: isA, date, hasName, and note. We'll ignore the assertions for now.

Now let’s look at some code to print out the same output from Rust:

#[cfg(test)]
mod tests {
    use bc_components::{PrivateKeyBase, PublicKeyBase};
    use bc_envelope::prelude::*;

    #[test]
    fn format_envelope() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        println!("{}", seed_envelope.format());
    }
}

While seedtool handles seeds directly, the Envelope codebase does not. Seeds are but one application of envelope. The next step is to get the subject of the envelope, the actual seed bytes, and convert them to a form we can use:

    #[test]
    fn extract_seed_bytes() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        println!("{}", hex::encode(seed_bytes)); // prints `11d4239723c0fa84d805d0524c7a2f4c``
    }

So how do we turn the seed into a private key? There’s no one right way to do this, but the simplest is just to use the seed as raw key material. Our stack includes a struct called PrivateKeyBase for this very purpose. PrivateKeyBase is thoughtfully named, because it is explicitly not a private key but a container for key material from which private keys of various sorts are derived, and it uses methods like HKDF to turn key material of any length into private keys as securely as possible. Let’s continue to expand our example:

    #[test]
    fn make_private_key_base() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        println!("{:?}", private_key_base); // prints `PrivateKeyBase``
    }

Notice that the PrivateKeyBase is not so happy just to print out its contents to the terminal, because it definitely contains private key material.

To communicate securely with other parties, you want to sign your messages, and then encrypt them to the recipient(s). When you are a recipient, you want to decrypt it first, then verify the signature came from someone you know. The fact that signing and encryption can be done with various different algorithms, as well as security best practices, dictate that you need different keys for each of these functions:

PrivateSigningKey: sign your message
PublicAgreementKey: encrypt to recipient(s)
PrivateAgreementKey: decrypt received message
PublicSigningKey: verify the sender’s signature

Kind of a hassle, right? Well, we make it a lot easier than that for the most common cases. Your PrivateKeyBase can be used to sign and encrypt, because our stack will automatically derive the right keys to do those functions. But what about the public keys your correspondents need from you? We have another struct called PublicKeyBase that is actually a bundle of both a PublicSigningKey and PublicAgreementKey. So you can treat a PublicKeyBase as everything you need to send to others so they can communicate with you. Our recommended PublicKeyBase uses BIP-340 Schnorr for signing and x25519 for encryption, but those are details you really don’t need to care about to use it. Let’s get our PublicKeyBase and convert it to a UR so we can send it to our recipients:

    #[test]
    fn derive_public_key_base() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        let public_key_base = private_key_base.schnorr_public_key_base();
        let public_key_base_ur = public_key_base.ur_string();
        println!("{}", public_key_base_ur); // prints `ur:crypto-pubkeys/lftanshfhdcxfxpswkatgskkkobsvwuewsgydsdsinbstaaefetkdydszmonrpnlnbfzsgehurjetansgrhdcxtdcnlfpdlfrobswsmycfvdkgpegupkhffxmswlfldngsnbhtkggwtywyrdaaotchloltprjn`
        let public_key_base_2 = PublicKeyBase::from_ur_string(&public_key_base_ur).unwrap();
        assert!(public_key_base == public_key_base_2);
    }

Notice we specify the signing algorithm to derive (schnorr_public_key_base). Notice also that I’m demonstrating how to round-trip the UR back into the PublicKeyBase struct.

OK, so we’ve got our PrivateKeyBase and the PublicKeyBase. Now we need an envelope to sign:

    #[test]
    fn compose_envelope() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        let public_key_base = private_key_base.schnorr_public_key_base();

        let message_envelope = Envelope::new("My First Envelope")
            .add_assertion("body", "Hello, World!");
        println!("{}", message_envelope.format());
    }

When run, this example prints:

"My First Envelope" [
    "body": "Hello, World!"
]

Next we sign it with our PrivateKeyBase:

    #[test]
    fn sign_envelope() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        let public_key_base = private_key_base.schnorr_public_key_base();

        let message_envelope = Envelope::new("My First Envelope")
            .add_assertion("body", "Hello, World!");
        let signed_envelope = message_envelope.sign(&private_key_base);
        println!("{}", signed_envelope.format());
    }

Here’s what the signed envelope looks like. The original message envelope has been wrapped, and the entire wrapped envelope is then signed by adding a verifiedBy assertion using a Schnorr private key derived from your PrivateKeyBase:

{
    "My First Envelope" [
        "body": "Hello, World!"
    ]
} [
    'verifiedBy': Signature
]

Now we want to encrypt this signed envelope to our recipient. So let’s use the crypto-pubkeys you gave in your mesage to me, that you generated using the command line tool:

    #[test]
    fn encrypt_envelope() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        let public_key_base = private_key_base.schnorr_public_key_base();

        let message_envelope = Envelope::new("My First Envelope")
            .add_assertion("body", "Hello, World!");
        let signed_envelope = message_envelope.sign(&private_key_base);

        let recipient_pubkeys_ur = "ur:crypto-pubkeys/lftanshfhdcxlsmdiotoimwsnliopdfzheuyenbkbdsngwbboxeckpenjlemgymuhhsrasrpwfmstansgrhdcxwsbygmfhlgrtdnosfhgshsgtpkwzztmoclfyrtnnnsfmpemesgfwiopsfyjtcmgoldfwttcy";
        let recipient_pubkeys = PublicKeyBase::from_ur_string(recipient_pubkeys_ur).unwrap();
        let encrypted_envelope = signed_envelope.encrypt_to_recipient(&recipient_pubkeys);
        println!("{}", encrypted_envelope.format());
        let encrypted_envelope_ur = encrypted_envelope.ur_string();
        println!("{}", encrypted_envelope_ur);
    }

The initial output is the envelope notation of the encrypted envelope. It’s an EncryptedMessage with a hasRecipient assertion. The SealedMessage is the decryption key for the EncryptedMessage that has been specifically encrypted to the recipient using their public key:

ENCRYPTED [
    'hasRecipient': SealedMessage
]

We also print the UR form of the encrypted envelope. This is the actual message to transmit to the recipient.

ur:envelope/lftansfwlrhdkichhkfrsklyclfspecwjpbktacaytpsoncyzewmeodwhpotledlbadwjthlurtymwdwzcdldwisuepfbzfsmevtrllbbwyloelalgtecekedehlsfndglhssruttofypmoturuygsfstpzobazmlnsoidtpadtyplqdcmmsaxcthsptldnnyaeersbbldjosbcxcstsadjyztqztpwtbyjpdmkibkhpfpjosbfmcmrysswplefrkkzmmnnlgsptoezsbgfhnldkckkpleeylpgdpmsoptkbftwkbsbzolnsplgeaxcksocyhddatansfphdcxfxyaftdibsoldiknkbislnfzfhynrdkpfwoxutplsnhefllghsbztpzohkbayaproyahtpsotansgulftansfwlshddaastktyatkoptkoplfplkmtndoyteayfyaeheotimgyoxpsmtrnaomdnsjodiasbteyvafmgosogscwtbcemtoeswsfkpiogrdsckgdssmwaahkpflscpgartdwylptpmvabgcttansgrhdcxfmimgdoydmdpdmytbdsadihnrkstgsgedllgkphkidkkjzsauyasssbbjsttbefnsnqzgyey

Finally, we add a few lines to reverse the process from the recipient’s point of view, using their private key base (which you also included in your email) to decrypt. Then we use your public key base to verify your signature and unwrap the message. Finally, we print the message:

    #[test]
    fn receive_envelope() {
        let seed_envelope_ur = "ur:envelope/lptpsogdbytycnmscnrtzslrtpahtigmgskndlgsoybetpsosezofptassjnhkvannnloyadcsspoyaatpsoksceghihjkjycxjkihihiecxiyjljpcxjeihkkcxioihjtihjphsjyinjljtoybdtpsojzgtkkcxghihjkjycxguihihievdetswte";
        let seed_envelope = Envelope::from_ur_string(seed_envelope_ur).unwrap();
        let subject = seed_envelope.subject();
        let seed_bytes = ByteString::try_from(subject).unwrap();
        let private_key_base = PrivateKeyBase::from_data(seed_bytes);
        let public_key_base = private_key_base.schnorr_public_key_base();

        let message_envelope = Envelope::new("My First Envelope")
            .add_assertion("body", "Hello, World!");
        let signed_envelope = message_envelope.sign(&private_key_base);

        let recipient_pubkeys_ur = "ur:crypto-pubkeys/lftanshfhdcxlsmdiotoimwsnliopdfzheuyenbkbdsngwbboxeckpenjlemgymuhhsrasrpwfmstansgrhdcxwsbygmfhlgrtdnosfhgshsgtpkwzztmoclfyrtnnnsfmpemesgfwiopsfyjtcmgoldfwttcy";
        let recipient_pubkeys = PublicKeyBase::from_ur_string(recipient_pubkeys_ur).unwrap();
        let encrypted_envelope = signed_envelope.encrypt_to_recipient(&recipient_pubkeys);
        let encrypted_envelope_ur = encrypted_envelope.ur_string();

        let recipient_prvkeys_ur = "ur:crypto-prvkeys/hdcxvssgwltohhasjskbskptnlkgtncfosnyhyktwzpfoshkotmuhdgapdverkctbddmbnmonlfn";
        let recipient_private_key_base = PrivateKeyBase::from_ur_string(recipient_prvkeys_ur).unwrap();
        let received_encrypted_envelope = Envelope::from_ur_string(encrypted_envelope_ur).unwrap();
        let decrypted_envelope = received_encrypted_envelope.decrypt_to_recipient(&recipient_private_key_base).unwrap();
        let verified_envelope = decrypted_envelope.verify(&public_key_base).unwrap();
        println!("{}", verified_envelope.format());
    }

"My First Envelope" [
    "body": "Hello, World!"
]

[danpape]

Wow Wolf, this is an amazing explanation and so very detailed. Thank you so much!

[ChristopherA]

@danpape

Two minor comments in addition to Wolf's answer:

Given your app, if you've not looked at the GSTP (Gordian Sealed Transport Protocol) video (starting at 16:35 https://youtu.be/uFxStP3ATkw?si=4fAP6HRsdoxPIrHg&t=995 ) & documentation (https://developer.blockchaincommons.com/envelope/gstp/ & https://developer.blockchaincommons.com/envelope/gstp/tech/ ), you may also want to look those over. It wraps these types of signed and encrypted objects in requests and responses so that they can be used in protocols.

Also, if you have a older version of the envelope-cli tool v0.92, you should upgrade, as we had a breaking change to fully support BIP340 signatures used by Taproot.

-- Christopher Allen

[danpape]

I have not heard of GSTP before, thanks Christopher.

[ChristopherA]

Also there are more details on what kinds of signing we current support at https://github.com/BlockchainCommons/bc-envelope-cli-rust/blob/master/docs/Signing.md and also how to encrypt then sign at https://github.com/BlockchainCommons/bc-envelope-cli-rust/blob/master/docs/BasicExamples.md

-- Christopher Allen