Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

account_tx comes back blank #381

Open
RichardAH opened this issue Oct 22, 2024 · 8 comments
Open

account_tx comes back blank #381

RichardAH opened this issue Oct 22, 2024 · 8 comments

Comments

@RichardAH
Copy link
Contributor

Issue Description

Steps to Reproduce

Expected Result

Actual Result

Environment

Supporting Files
@vasil-nesterov
Copy link

@RichardAH
When I run the same account_tx command 20 times against https://xahau.network, from time to time I get a response with 0 txns and no marker. Is this the same issue as yours, or would it better for me to open a separate issue?

(account: rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C, ledger_index_min: -1, ledger_index_max: -1, no marker)

Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509089
 Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509063
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509096
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509097
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509097
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509098
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509098
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509098
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509099
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509099
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509100
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509100
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509100
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509101
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509101
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509102
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509102
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509103
Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509103
Txns: 0, Marker: , ledger_index_min: 9508450, ledger_index_max: 9509103

Script

require 'http'
require 'json'
require 'securerandom'

account = 'rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C'

20.times do |i|
  query = {
    'jsonrpc': '2.0',
    id: SecureRandom.uuid,
    method: 'account_tx',
    params: [
      {
        ledger_index_min: -1,
        ledger_index_max: -1,
        account: account
      }.compact
    ]
  }

  response =
    HTTP
      .headers('Content-Type' => 'application/json')
      .post('https://xahau.network', body: query.to_json)
      .then { JSON.parse(_1.to_s) }

  if response.dig('result', 'status') != 'success'
    raise "Request failed"
  end

  marker, txns, ledger_index_min, ledger_index_max =
    response['result'].values_at(
      'marker', 'transactions', 'ledger_index_min', 'ledger_index_max'
    )

  puts "Txns: #{txns.count}, Marker: #{marker}, ledger_index_min: #{ledger_index_min}, ledger_index_max: #{ledger_index_max}"
end

@dangell7
Copy link
Collaborator

Are you sure you're not being rate limited?

@vasil-nesterov
Copy link

@dangell7

When I encounter rate limit, the response code is 429 and the body is Rate limited.

This is a different story.

Here's the output with bodies of "0 txs" responses:

0: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
1: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
2: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
3: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
4: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
5: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509533
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509533,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
6: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509534
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509534,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
7: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
8: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
9: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
10: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
11: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
12: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
13: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
14: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
15: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
16: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
17: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509537
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509537,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
18: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509538
19: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509538
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509538,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}

@vasil-nesterov
Copy link

Workaround: retry each request until response.ledger_index_min == 1.
This way I'm able to fetch all txs from a big account in a stable manner.

@dangell7
Copy link
Collaborator

@dangell7

When I encounter rate limit, the response code is 429 and the body is Rate limited.

This is a different story.

Here's the output with bodies of "0 txs" responses:

0: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
1: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509532
2: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
3: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
4: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509533
5: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509533
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509533,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
6: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509534
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509534,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
7: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
8: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509534
9: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
10: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509535
11: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
12: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
13: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
14: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509536
15: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
16: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509537
17: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509537
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509537,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}
18: HTTP code: 200, Txns: 200, Marker: {"ledger"=>8205251, "seq"=>62}, ledger_index_min: 1, ledger_index_max: 9509538
19: HTTP code: 200, Txns: 0, Marker: , ledger_index_min: 9508962, ledger_index_max: 9509538
        {
  "result": {
    "account": "rpEZRGPj3miukn48yfQGePQFjqVhHJLm1C",
    "ledger_index_max": 9509538,
    "ledger_index_min": 9508962,
    "limit": 0,
    "status": "success",
    "transactions": [

    ],
    "validated": true
  }
}

I see. I'm curious what you're trying to accomplish? Why are you increasing the ledger_max by 1? And in your last request you are looking for transactions between 9508962 - 9509538 and there are none... So its correct. If you want all the transactions for an account what you are doing isn't correct.

  1. First you should query the account_tx and leave out ledger min/max with a limit of 1000. Then loop though the markers. (it will auto fill ledger min/max). Store ledger_max response to use later
  2. Later when you want to re-query you use the ledger_max response as the ledger_min so you're not re querying the information.

@vasil-nesterov
Copy link

@dangell7

I'm curious what you're trying to accomplish?

I'm showing that the API is unstable.

In my example, I'm repeating the same request with exactly the same params 20 times.
The request params are: account: pEZRGPj3miukn48yfQGePQFjqVhHJLm1C, ledger_index_min: -1, ledger_index_max: -1, no marker.

Some responses are ok (those that have a marker and txs.count == 200), others are broken. The broken ones have empty txs list, no marker and a very high ledger_index_min value, indicating that the server didn't search the entire ledger as requested.

@tequdev
Copy link
Collaborator

tequdev commented Oct 31, 2024

It appears that some nodes in the Xahaud cluster are holding only the most recent ledger instead of the full history node. (or in backfill).

This is the result of running server_info rpc multiple times and getting complete_ledgers.

complete_ledgers 9548898-9549739
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 9548898-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740
complete_ledgers 1-9549740

@RichardAH
Copy link
Contributor Author

I'll leave this issue open and rename it, but the original issued was fixed by 8b0d427 I believe

@RichardAH RichardAH changed the title [memdb] account_tx comes back blank account_tx comes back blank Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@vasil-nesterov @RichardAH @dangell7 @tequdev