Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"nodetool info" on decomissioned node generate weird error #59

Open
nyh opened this issue Dec 17, 2018 · 0 comments
Open

"nodetool info" on decomissioned node generate weird error #59

nyh opened this issue Dec 17, 2018 · 0 comments
Milestone

Comments

@nyh
Copy link
Contributor

nyh commented Dec 17, 2018

The Scylla dtest materialized_views_test.py:TestMaterializedViews.mv_populating_from_existing_data_during_node_decommission_test used to decomission of the nodes of the cluster with "nodetool decomission", and later used "nodetool info" on it. It runs a command like "nodetool -h localhost -p 7277 info", because ccm deploys several servers on multiple ports.
We see the following output from from nodetool, which then fails:

Nodetool command '/jenkins/workspace/scylla-3.0/scylla-dtest/label/scylla-dtest/mode/release/smp/2/scylla/resources/cassandra/bin/nodetool -h localhost -p 7277 info' failed; exit status: 1; stdout: ID                     : 926bfd57-91b3-4f1b-8650-2f6803dbaebe
Gossip active          : false
Thrift active          : false
Native Transport active: false
Load                   : 8.03 MB
Generation No          : 0
Uptime (seconds)       : 272
Heap Memory (MB)       : 97.28 / 247.50
Off Heap Memory (MB)   : 125.87
Data Center            : dc1
Rack                   : RAC1
Exceptions             : 0
Key Cache              : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, 0.000 recent hit rate, 0 save period in seconds
Row Cache              : entries 133890, size 130.75 KiB, capacity 5.47 GiB, 13769 hits, 13769 requests, 1.000 recent hit rate, 0 save period in seconds
Counter Cache          : entries 0, size 0 bytes, capacity 0 bytes, 0 hits, 0 requests, 0.000 recent hit rate, 0 save period in seconds
Percent Repaired       : 0.0%
nodetool: Scylla API server HTTP GET to URL '/storage_service/tokens/null' failed: Wrong format for IPv4 address null. Please ensure it's in dotted-decimal format
See 'nodetool help' or 'nodetool help <command>'.

I don't know why some of the information is still available, apparently the decomission did not really kill the node or at least its left its JMX server orphan (?). But what bothers me (the problem which caused nodetool to report a failure) is at the end, where we run some malformed REST API request and fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants