Can't interop with subkey

[georgiosd]

Guys, I'm driving myself crazy and I hope you can help me out.

If I sign a message with @polkadot, I can't then verify it with subkey, how does that work?

// Output:
//
// pub: 6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
// secret: a8e59c0eaa54b586bafabe3e4c5597d6b96677c61105e60b0f690327f8d4db4fc7f7331ec0a5bd07390edee454c6e60508af900be8b8682da35b94420992f460
// sig: ead0899d04969bea6857a95a2404babf77844d852431383d7adfabbceebe066b87d3888af454ecf1775dd84e7dbfe12266f15d1346c5d8778bad2cdcf2744683

const { sr25519PairFromSeed, sr25519Sign, cryptoWaitReady } = require('@polkadot/util-crypto')

cryptoWaitReady().then(() => {
    const pair = sr25519PairFromSeed(Buffer.from("42f2524bd7800cc244deb8ce5b4c24cb262c1ba317b86880d7ff33f1e93c6348", 'hex'))

    console.log(Buffer.from(pair.publicKey).toString('hex'))
    console.log(Buffer.from(pair.secretKey).toString('hex'))

    console.log(Buffer.from(sr25519Sign("test message", pair)).toString('hex'))
})

From subkey:

$ subkey inspect "time treat merit corn crystal fiscal banner zoo jacket pulse frog long"
Secret phrase:       time treat merit corn crystal fiscal banner zoo jacket pulse frog long

  Network ID:        substrate
  Secret seed:       0x42f2524bd7800cc244deb8ce5b4c24cb262c1ba317b86880d7ff33f1e93c6348
  Public key (hex):  0x6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
  Account ID:        0x6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
  Public key (SS58): 5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2
  SS58 Address:      5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2
  
$ echo "test message" | subkey verify ead0899d04969bea6857a95a2404babf77844d852431383d7adfabbceebe066b87d3888af454ecf1775dd84e7dbfe12266f15d1346c5d8778bad2cdcf2744683 5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2
Error: SignatureInvalid

For context

I got here because I am getting an error 1010 when submitting a balance transfer to Westend. I then found these two but I didn't manage to get anywhere either. This code still fails:

const CTX : &'static [u8] = b"substrate";
let sec = SecretKey::generate();
let pub_bytes = sec.to_public().to_bytes();

let message = b"<Bytes>message to sign</Bytes>";

let sig = match (SecretKey::from_ed25519_bytes(&sec.to_bytes()), PublicKey::from_bytes(&pub_bytes)) {
(Ok(s), Ok(k)) => s
	.sign_simple(CTX, message, &k)
	.to_bytes()
	.to_vec(),
_ => panic!("Invalid secret or pubkey provided.")
};


// ext_sr_verify function
let res =   match (Signature::from_bytes(&sig), PublicKey::from_bytes(&pub_bytes)) {
(Ok(s), Ok(k)) => k
	.verify_simple(CTX, message, &s)
	.is_ok(),
_ => false
};

assert!(res)

[jacogr]

As already commented on Stackoverflow by me, as the first comment on your post - the issue with your gist is that you are sending immortal transactions and the blockHash you specify is not the genesisHash.

You have -

** genesisHash **
0xe143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e
** blockHash **
0x142b31020d705da1ea158a8bc44815e2b4c036944f39460185ccc6d1ad884065

instead of

** genesisHash **
0xe143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e
** blockHash **
0xe143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e

For the echo above, echo adds a newline to the input which is not in your original message, so it is certainly expected that your command will yield invalid results. For echo you need the -n flag. My output with the flag -

$ cargo run --release -p subkey -- inspect "time treat merit corn crystal fiscal banner zoo jacket pulse frog long"
    Finished release [optimized] target(s) in 0.31s
     Running `target/release/subkey inspect 'time treat merit corn crystal fiscal banner zoo jacket pulse frog long'`
Secret phrase:       time treat merit corn crystal fiscal banner zoo jacket pulse frog long
  Network ID:        substrate
  Secret seed:       0x42f2524bd7800cc244deb8ce5b4c24cb262c1ba317b86880d7ff33f1e93c6348
  Public key (hex):  0x6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
  Account ID:        0x6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
  Public key (SS58): 5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2
  SS58 Address:      5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2

$ echo -n "test message" | cargo run --release -p subkey -- verify ead0899d04969bea6857a95a2404babf77844d852431383d7adfabbceebe066b87d3888af454ecf1775dd84e7dbfe12266f15d1346c5d8778bad2cdcf2744683 5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2
    Finished release [optimized] target(s) in 0.31s
     Running `target/release/subkey verify ead0899d04969bea6857a95a2404babf77844d852431383d7adfabbceebe066b87d3888af454ecf1775dd84e7dbfe12266f15d1346c5d8778bad2cdcf2744683 5EUJ3p7ds1436scqdA2n6ph9xVs6chshRP1ADjgK1Qj3Hqs2`
Signature verifies correctly.

[georgiosd]

Wow, that's a real gotcha with the echo, thank you for looking into this @jacogr.

However, I did try with block hash = genesis and still getting the same error. The era is set to [0].

** url **
https://westend-rpc.polkadot.io/
** public **
6a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d
** secret **
a8e59c0eaa54b586bafabe3e4c5597d6b96677c61105e60b0f690327f8d4db4fc7f7331ec0a5bd07390edee454c6e60508af900be8b8682da35b94420992f460
** dest **
5FZDBCeK9FuUvnny3WhXr62Ah6pneeSaxBWL6osFoUXSszxD
** callIndex **
0400
** addressType **
0
** dest **
9A7245201D6C75A4AE54F7BE22F93A7D1DB506790CE9C4F9B1D2B64577B2004D
** compact(amount) **
04
** genesisHash **
0xe143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e
** blockHash **
0xe143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e
** specVersion **
01000000
** transactionVersion **
01000000
** nonce **
46
** method **
0400009a7245201d6c75a4ae54f7be22f93a7d1db506790ce9c4f9b1d2b64577b2004d04
** era **
00
** unsigned **
0400009a7245201d6c75a4ae54f7be22f93a7d1db506790ce9c4f9b1d2b64577b2004d0400b8000100000001000000e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423ee143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e
** verified **
True
** signature **
01182b8ff6c9115fa021dc0264e318442909adda26fd7df4ed54cf8bff2c6e002fea48ea65f78c8201047b2cc140717854af1fd1f0e2372b81b82eb0123fa23680
** signed **
006a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d01182b8ff6c9115fa021dc0264e318442909adda26fd7df4ed54cf8bff2c6e002fea48ea65f78c8201047b2cc140717854af1fd1f0e2372b81b82eb0123fa2368000b800
** payload submitted to author_submitExtrinsic **
0x290284006a7636545fecfa16f2e37679ea8547685cc2149807f94a4ada6ec0a264ead14d01182b8ff6c9115fa021dc0264e318442909adda26fd7df4ed54cf8bff2c6e002fea48ea65f78c8201047b2cc140717854af1fd1f0e2372b81b82eb0123fa2368000b8000400009a7245201d6c75a4ae54f7be22f93a7d1db506790ce9c4f9b1d2b64577b2004d04
Unhandled exception. System.InvalidOperationException: Substrate error 1010; Invalid Transaction: Transaction has a bad signature
   at Engi.Substrate.SubstrateClient.RpcAsync[T](String method, Object[] params) in C:\Users\georgiosd\Documents\engi-blockchain-gql\Engi.Substrate\SubstrateClient.cs:line 47
   at Engi.Substrate.SubstrateClient.RpcAsync(String method, Object[] params) in C:\Users\georgiosd\Documents\engi-blockchain-gql\Engi.Substrate\SubstrateClient.cs:line 63
   at Engi.Substrate.Playground.Program.Main() in C:\Users\georgiosd\Documents\engi-blockchain-gql\Engi.Substrate.Playground\Program.cs:line 173
   at Engi.Substrate.Playground.Program.<Main>()

I did have success with submitting the transaction with a mortal era taken from your code but I would still like to understand why the immortal era is not accepted.

[jacogr]

From a casual glance -

specVersion - it is 9220 on Westend.
transactionVersion - it is 11 on Westend

Both as per the state_getRuntimeVersion RPC

Since these are in the payload and the runtime re-constructs this payload with the on-chain information, they need to be 100% spot-on and filled correctly. Having both the same and encoded as 01000000 as above is certainly not correct, neither has been 1 for quite some time now.

I would suggest checking your signing payload, it can basically be anything - if one field is incorrect, the signature will fail. (And the tx pool cannot say which it is)

EDIT:

** addressType **
0

Also looks suspect, 00 is a zero byte, 0 is not a byte by itself, needs another digit. (This could be a typo). If you wish to debug these, send a signed ed25519 transaction from another client with the same params as yours and then check the signatures. (Which would be deterministic). Only then swap to non-deterministic signatures.

[georgiosd]

Aha, that's probably my downfall. I thought the runtime version has to be fetched from the block hash, which in this case was the generis, hence 1. But what you said makes more sense.

Will check that again.

Address type is ok - just printed as an integer instead of hex, which is why you see it as single digit.

[jacogr]

Perfect, thought the address type was just a typo :)

So spec/tx is applicable to when the tx executes. Basically this means that when either changes, the transaction would become invalid. (In you send a tx just before an upgrade, on the block after the upgrade it cannot be included anymore, it will be rejected)

[georgiosd]

@jacogr, if you're ever in Greece, I'm buying you a beer! My email is my github username @ gmail :)

[polkadot-js-bot]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.