created txid.py and updated solution.md file

SOLUTION.md

@@ -4,14 +4,83 @@
 
 First of all we need to check the structure of the transaction which includes version, locktime, inputs and outputs.
 
+## Transaction Structure Validation Algorithms.md
+
+### Main Validation Function: `check_transaction(tx_data)`
+- Verifies required fields: `"version"`, `"locktime"`, `"vin"`, and `"vout"`.
+- Checks field types: `"version"` and `"locktime"` must be integers.
+- Calls `validate_vin()` and `validate_vout()` for further validation.
+
+### Vin Validation Function: `validate_vin(vin_list)`
+- Checks each `"vin"` element for required fields and types.
+- Verifies `"txid"`, `"vout"`, `"prevout"`, `"scriptsig"`/`"scriptsig_asm"`, `"witness"`, `"is_coinbase"`, and `"sequence"`.
+
+### Vout Validation Function: `validate_vout(vout_list)`
+- Checks each `"vout"` element for required fields and types.
+- Verifies `"scriptpubkey"`, `"scriptpubkey_asm"`, `"scriptpubkey_type"`, `"scriptpubkey_address"`, and `"value"`.
+
+### Signature Script Validation Function: `validate_signature_script(script)`
+- Checks if the script is empty, a valid push data script, a valid opcode, or a valid combination of push data and opcodes.
+
+### Checking Transaction Structure in Mempool: `check_structure_transactions(mempool_folder, valid_folder)`
+- Iterates through JSON files in the `mempool_folder`.
+- Loads transaction data and calls `check_transaction()` for validation.
+- Moves valid transactions to the `valid_folder` and prints messages for valid and invalid transactions.
+- Handles errors during file processing.
+
+These algorithms work together to validate the structure of Bitcoin transactions, checking the presence and types of required fields, validating `"vin"` and `"vout"` lists, and verifying the signature script. The code also provides a function to validate multiple transaction files in the mempool folder and move valid transactions to a [separate folder](./mempool_valid/).
+
+## Creating Transaction ID(TXID)
+
+Step by step implementation for creating TXID and then updating the json file in `mempool_valid` folder.
+
+1. **Initialize necessary modules and functions:**
+   - Import the required modules: `json`, `struct`, `hashlib`, and `os`.
+   - Define the `compact_size` function to serialize the size of data in the transaction.
+   - Define the `create_txid` function to create the transaction ID (txid) based on the JSON data.
+
+2. **Specify the folder path:**
+   - Set the `folder_path` variable to the path of the folder containing the JSON files (e.g., `"./mempool"`).
+
+3. **Iterate over the JSON files in the folder:**
+   - Use `os.listdir(folder_path)` to get a list of files in the specified folder.
+   - Iterate over each file in the folder using a loop.
+   - Check if the file has a `.json` extension using the `endswith()` method.
+
+4. **Read the JSON file:**
+   - Construct the full file path by joining the `folder_path` and the current `filename` using `os.path.join()`.
+   - Open the JSON file in read mode using `open()` and the `"r"` flag.
+   - Read the contents of the file using `file.read()` and store it in the `json_data` variable.
+
+5. **Create the txid:**
+   - Call the `create_txid` function, passing the `json_data` as an argument.
+   - Inside the `create_txid` function:
+     - Parse the JSON data into a dictionary using `json.loads()`.
+     - Serialize the transaction data using `struct.pack()` and `compact_size()` functions.
+     - Concatenate the serialized data to create the `tx_data` bytes.
+   - Calculate the double SHA-256 hash of the `tx_data` using `hashlib.sha256()`.
+   - Reverse the byte order of the hash using `[::-1]` and convert it to a hexadecimal string using `hex()`.
+   - Store the resulting txid in the `txid` variable.
+
+6. **Append the txid to the JSON data:**
+   - Load the JSON data from `json_data` into a dictionary using `json.loads()`.
+   - Create a new dictionary called `updated_data` with `"txid"` as the first property and its value set to the generated `txid`.
+   - Use the `update()` method to merge the original JSON data dictionary into the `updated_data` dictionary, ensuring that `"txid"` remains the first property.
+
+7. **Write the updated JSON data back to the file:**
+   - Open the same JSON file in write mode using `open()` and the `"w"` flag.
+   - Write the `updated_data` dictionary to the file using `json.dump()`.
+   - Specify an indentation of 2 spaces using `indent=2` for better readability.
 
 
 
 
 
 # Implementation Details
 
-Provide pseudo code of your implementation, including sequence of logic, algorithms and variables used etc.
+- Transaction Structure validation: [structural_check.py](./python_files/structural_check.py)
+
+- Creating TXID and Updating the JSoN Structure: [create_txid.py](./python_files/create_txid.py)
 
 
 # Result and Performance

mempool_valid/000cb561188c762c81f76976f816829424e2af9e0e491c617b7bf41038df3d35.json

@@ -0,0 +1,35 @@
+{
+  "txid": "23f6ae5049268f63e473c5314b58a1f2d4a0d4fafd8c8ccf8efe06974ad4e077",
+  "version": 2,
+  "locktime": 0,
+  "vin": [
+    {
+      "txid": "64ca1941edef34b690dd6672c7d395c60882067f7f3fc396e64d88e39c1da5b4",
+      "vout": 0,
+      "prevout": {
+        "scriptpubkey": "0014d5bfb7a6d05d44c1e14443919b30d284c0c0a10a",
+        "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 d5bfb7a6d05d44c1e14443919b30d284c0c0a10a",
+        "scriptpubkey_type": "v0_p2wpkh",
+        "scriptpubkey_address": "bc1q6klm0fkst4zvrc2ygwgekvxjsnqvpgg2jjfurm",
+        "value": 10740
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "3044022100884219ecbb54a6ec4d09597ca6aca49692ded3c2ffb13d1858ca5b70e59fabb4021f2de73021471a01d8f03a71a923b662f00120d181d0f7fa8e06faa1bb750e8f01",
+        "0271d4e7a84804c075017593271c370e8983f704f123d22aa747cd321268981cba"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967293
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "a91450feb99697a4901d3fe082eca341204fb6711b9487",
+      "scriptpubkey_asm": "OP_HASH160 OP_PUSHBYTES_20 50feb99697a4901d3fe082eca341204fb6711b94 OP_EQUAL",
+      "scriptpubkey_type": "p2sh",
+      "scriptpubkey_address": "395H8VPYPtAoZWa2bx5SRyN2VojXrsb7j3",
+      "value": 9520
+    }
+  ]
+}

mempool_valid/001035505afbf143e51bd667099190943a38eee20092bb691e72eaa44992b2f7.json

@@ -0,0 +1,52 @@
+{
+  "txid": "f41c60ea16b21f696611aab3a62552037fbe95012ac4a459a6c19c2c23cd3222",
+  "version": 2,
+  "locktime": 0,
+  "vin": [
+    {
+      "txid": "79593e210f4e7c61be57dfb6c33541ab39513a3bb964737c8e8c988bd395986e",
+      "vout": 1,
+      "prevout": {
+        "scriptpubkey": "5120f6dcb2b44ee298462ffc8dd9d7b969d660ddedd98ff31dd3be803b699aea9416",
+        "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 f6dcb2b44ee298462ffc8dd9d7b969d660ddedd98ff31dd3be803b699aea9416",
+        "scriptpubkey_type": "v1_p2tr",
+        "scriptpubkey_address": "bc1p7mwt9dzwu2vyvtlu3hva0wtf6esdmmwe3le3m5a7sqaknxh2jstq3vztqn",
+        "value": 593313
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "7c6982626a43525e568d328824702b3743b112e40e75293b3162b182c41e7754d481e911839b211cb7547f266814afa911eea85676f69722c1f268410a75f864"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967295
+    },
+    {
+      "txid": "dd006a4a306b45dab04fb249be9449caae04df24269b60afd443238729260d56",
+      "vout": 1,
+      "prevout": {
+        "scriptpubkey": "5120f6dcb2b44ee298462ffc8dd9d7b969d660ddedd98ff31dd3be803b699aea9416",
+        "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 f6dcb2b44ee298462ffc8dd9d7b969d660ddedd98ff31dd3be803b699aea9416",
+        "scriptpubkey_type": "v1_p2tr",
+        "scriptpubkey_address": "bc1p7mwt9dzwu2vyvtlu3hva0wtf6esdmmwe3le3m5a7sqaknxh2jstq3vztqn",
+        "value": 10704
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "c2ed1bd63aa5e6e7d094210615d01cfcd74d87ffeb2232fb909f34dc661300e1ada70e6deda0cf3a05d390820a89b247d2115a59502eb82f1ed619ec929a2acd"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967295
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "a914bf98ba19c45f75bb76b50afce91fb5bb7276bd7787",
+      "scriptpubkey_asm": "OP_HASH160 OP_PUSHBYTES_20 bf98ba19c45f75bb76b50afce91fb5bb7276bd77 OP_EQUAL",
+      "scriptpubkey_type": "p2sh",
+      "scriptpubkey_address": "3KA5wqaDxNSkEWKdmsnx6jxJBKtQXuhM7u",
+      "value": 600107
+    }
+  ]
+}

mempool_valid/0018c221bca3da35128baabe412a14c95b6864b2e6f7f7a8ffdd8eb0923dec49.json

@@ -0,0 +1,42 @@
+{
+  "txid": "6494cdb062c4a1bc3e74760f9fd389c69c9cd0dcba246c0c58ab37688671edbd",
+  "version": 2,
+  "locktime": 834637,
+  "vin": [
+    {
+      "txid": "4bc8a6bbd9f01b7c20fb59adc81352ff087ff7dbc52efb3cc55f188edebf47d8",
+      "vout": 1,
+      "prevout": {
+        "scriptpubkey": "00145017ec1e0f2e6fd64561aa072931bcd261797559",
+        "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 5017ec1e0f2e6fd64561aa072931bcd261797559",
+        "scriptpubkey_type": "v0_p2wpkh",
+        "scriptpubkey_address": "bc1q2qt7c8s09ehav3tp4grjjvdu6fshja2e49rqzq",
+        "value": 259893968
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "304402206cb268614ab72910e5a975893e7cd2cb84aa58ac4f318cf84547dd3a9562761a0220455f61baeb45533265180d07a6dab52a4034ec0ffb5d7cb85757c562936509ea01",
+        "030b512819670cf864aa7605c7bfb32e37002db4f30c9d03a8b8aa1976049105aa"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967294
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "0014b9c1c402d4a435ba726f52e31c71e254af33ee0e",
+      "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 b9c1c402d4a435ba726f52e31c71e254af33ee0e",
+      "scriptpubkey_type": "v0_p2wpkh",
+      "scriptpubkey_address": "bc1qh8qugqk55s6m5un02t33cu0z2jhn8msw8rv8mr",
+      "value": 257963288
+    },
+    {
+      "scriptpubkey": "a914348fed75d9648cad589e072b1098cb674873c69087",
+      "scriptpubkey_asm": "OP_HASH160 OP_PUSHBYTES_20 348fed75d9648cad589e072b1098cb674873c690 OP_EQUAL",
+      "scriptpubkey_type": "p2sh",
+      "scriptpubkey_address": "36UwUnLVs27P4FmsAFvT26tt1zwwa1FRvQ",
+      "value": 1927200
+    }
+  ]
+}

mempool_valid/001e296ba3feddda174e62df57506861823b0831af983364d2c1808779b443f7.json

@@ -0,0 +1,42 @@
+{
+  "txid": "ede7be5d4753a25139d6b6ff033cbdf0e43b729733cb09a6a3b44472ad27bdab",
+  "version": 1,
+  "locktime": 0,
+  "vin": [
+    {
+      "txid": "aa6109f3cea8a458438de069af49cf8bde43db9f746f78fd5d4cac8e94b46e3c",
+      "vout": 1,
+      "prevout": {
+        "scriptpubkey": "0014421b8f0182ae2ab26e9297718f92f82805a48ba2",
+        "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 421b8f0182ae2ab26e9297718f92f82805a48ba2",
+        "scriptpubkey_type": "v0_p2wpkh",
+        "scriptpubkey_address": "bc1qggdc7qvz4c4tym5jjacclyhc9qz6fzazsqje0d",
+        "value": 2993846
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "3045022100da9befe31774321233b2315e9c5077679131274b9900966e45e8ac0c1c0391970220355144dcc40269048bb704c46d9864e47f631792f0b9e28ff380075a52818a5c01",
+        "02ddde52e4b72358ff888c056b2c0d7c3e47361e82e79d9202b1b011141d417192"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967295
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "00148e56a5479728f7786a6a8fc7cc0f3535cea91b52",
+      "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 8e56a5479728f7786a6a8fc7cc0f3535cea91b52",
+      "scriptpubkey_type": "v0_p2wpkh",
+      "scriptpubkey_address": "bc1q3et223uh9rmhs6n23lrucre4xh82jx6jnukpcl",
+      "value": 83000
+    },
+    {
+      "scriptpubkey": "0014421b8f0182ae2ab26e9297718f92f82805a48ba2",
+      "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 421b8f0182ae2ab26e9297718f92f82805a48ba2",
+      "scriptpubkey_type": "v0_p2wpkh",
+      "scriptpubkey_address": "bc1qggdc7qvz4c4tym5jjacclyhc9qz6fzazsqje0d",
+      "value": 2896746
+    }
+  ]
+}

mempool_valid/0022a52ad27796a1a2d9eddd6f4b055c097b51ad7cb8f000fe0d78b26cb71639.json

@@ -0,0 +1,59 @@
+{
+  "txid": "a9c99dc5ec74c6171095a4b156d8127ce003a855c4816f2caf90b8832f563471",
+  "version": 2,
+  "locktime": 0,
+  "vin": [
+    {
+      "txid": "e3acb27a9fb6593822a4d8bbb8be9f9516e33123f196a41fc2801a9cae278ba4",
+      "vout": 0,
+      "prevout": {
+        "scriptpubkey": "5120d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+        "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+        "scriptpubkey_type": "v1_p2tr",
+        "scriptpubkey_address": "bc1p67cpvytqmjxwgm0m74wxqtl7aa0df672m6a7qau33qvmp2smw0nqrgxc8d",
+        "value": 546
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "265f6c3128efad948b8e1d11e1cd165c457e9e5da32f0e4093d81221aa6ecd6ec9bebbda812f4aa8c2caabf50575d22b551cd95d27ec015031950589869b5d26"
+      ],
+      "is_coinbase": false,
+      "sequence": 2147483649
+    },
+    {
+      "txid": "ef575d36c46466f419678345bd0fc17d65cc1a5741a47ac272d9e329dab7c0c9",
+      "vout": 1,
+      "prevout": {
+        "scriptpubkey": "5120d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+        "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+        "scriptpubkey_type": "v1_p2tr",
+        "scriptpubkey_address": "bc1p67cpvytqmjxwgm0m74wxqtl7aa0df672m6a7qau33qvmp2smw0nqrgxc8d",
+        "value": 1811712
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "8b8b8a72ac3d38c85fb59b0ba462d6a4ef39b73bd86a3ac8a300e5ab52033729ecd442e2b0eb5aba0ba9111ad0bccbdb4d24483caf482ebe6d2b819825f27731"
+      ],
+      "is_coinbase": false,
+      "sequence": 2147483649
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "5120d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+      "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+      "scriptpubkey_type": "v1_p2tr",
+      "scriptpubkey_address": "bc1p67cpvytqmjxwgm0m74wxqtl7aa0df672m6a7qau33qvmp2smw0nqrgxc8d",
+      "value": 546
+    },
+    {
+      "scriptpubkey": "5120d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+      "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 d7b0161160dc8ce46dfbf55c602ffeef5ed4ebcadebbe077918819b0aa1b73e6",
+      "scriptpubkey_type": "v1_p2tr",
+      "scriptpubkey_address": "bc1p67cpvytqmjxwgm0m74wxqtl7aa0df672m6a7qau33qvmp2smw0nqrgxc8d",
+      "value": 1809391
+    }
+  ]
+}

mempool_valid/0026c0aa204a6da8916bf5849cff17d3c81b1a2b6f035045b5dc3263d8a448e2.json

@@ -0,0 +1,36 @@
+{
+  "txid": "29ce2496811d9ec9b5c1bac7664a08e63505e29f1e7a1ab72968296117b20957",
+  "version": 2,
+  "locktime": 0,
+  "vin": [
+    {
+      "txid": "a74026e81488bc8ed65781b5904edb3fd401d40e8892fa7c6d91a1cc66f02170",
+      "vout": 7,
+      "prevout": {
+        "scriptpubkey": "51202691567e31e951fc72a28ebace6fd5ab716dd455e7b150eadfcadd03a9e37b0e",
+        "scriptpubkey_asm": "OP_PUSHNUM_1 OP_PUSHBYTES_32 2691567e31e951fc72a28ebace6fd5ab716dd455e7b150eadfcadd03a9e37b0e",
+        "scriptpubkey_type": "v1_p2tr",
+        "scriptpubkey_address": "bc1py6g4vl33a9glcu4z36avum744dckm4z4u7c4p6kletws820r0v8qh8nknh",
+        "value": 1962
+      },
+      "scriptsig": "",
+      "scriptsig_asm": "",
+      "witness": [
+        "937d2bbbde729857003c7c48bec4613141e42b8a12c8e7a9f065ace973b3bc4ed5f7c49beb362cd22c16fd0002ee19422dabfc3027682dccfdb52a8ac44862c1",
+        "20abf657ab0c1aabaa24896a8ab10adde8612397f1eb7067bdcd5d9371d70c2febac0063036f7264010118746578742f706c61696e3b636861727365743d7574662d3800367b2270223a226272632d3230222c226f70223a226d696e74222c227469636b223a2261616161222c22616d74223a223130303030227d68",
+        "c1abf657ab0c1aabaa24896a8ab10adde8612397f1eb7067bdcd5d9371d70c2feb"
+      ],
+      "is_coinbase": false,
+      "sequence": 4294967293
+    }
+  ],
+  "vout": [
+    {
+      "scriptpubkey": "0014606aea6bcae8843bfa542b3231f709d8e8c47366",
+      "scriptpubkey_asm": "OP_0 OP_PUSHBYTES_20 606aea6bcae8843bfa542b3231f709d8e8c47366",
+      "scriptpubkey_type": "v0_p2wpkh",
+      "scriptpubkey_address": "bc1qvp4w5672azzrh7j59verracfmr5vgumxtld8pd",
+      "value": 294
+    }
+  ]
+}