Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple tables cannot be migrated simultaneously #96

Open
bharatnailwal opened this issue May 16, 2023 · 3 comments
Open

Multiple tables cannot be migrated simultaneously #96

bharatnailwal opened this issue May 16, 2023 · 3 comments
Labels
question Further information is requested

Comments

@bharatnailwal
Copy link

Hello Team,

Please help me find someone who can assist me with migrating multiple tables from a keyspace at the same time using the Scylla migration tool?
I tried making changes to config.yml, but it did not work.

error:
Exception in thread "main" DecodingFailure(Attempt to decode value on failed cursor, List(DownField(table), DownField(source)))

@bharatnailwal
Copy link
Author

bharatnailwal commented May 19, 2023

config.yaml

source:
  type: cassandra
  host: <great-secret>
  port: 9042
  credentials:
    username: great-secret
    password: 'great-secret'
  keyspace: <testkeyspace>
  table: <testtable1>
  consistencyLevel: ONE
  preserveTimestamps: true
  splitCount: 256
  connections: 8
  fetchSize: 1000
target:
  type: scylla
  host: <great-secret>
  port: 9042
  credentials:
    username: great-secret
    password: 'great-secret'
  keyspace: <testkeyspace>
  table: <testtable1>
  consistencyLevel: ONE
  connections: 16
  stripTrailingZerosForDecimals: false
savepoints:
  path: /app/savepoints
  intervalSeconds: 300
renames: []
skipTokenRanges: []
validation:
  compareTimestamps: true
  ttlToleranceMillis: 60000
  writetimeToleranceMillis: 1000
  failuresToFetch: 100
  floatingPointTolerance: 0.001
  timestampMsTolerance: 0     

@tarzanek
Copy link
Contributor

tarzanek commented May 22, 2023

<testtable1> you likely need to escape or quote the value in source/table

e.g. 
  table: "<testtable1>"
  
but I doubt that is a table name, so you likely need to use proper name there

@tarzanek
Copy link
Contributor

tarzanek commented May 22, 2023

also you can migrate only one by one (I usually script it with bash scripts)
(check https://github.com/scylladb/scylla-code-samples/tree/master/spark-scylla-migrator-demo )

you can even have bash templates and replace the values in them from env
(look at envsubst )

@bharatnailwal ^

@tarzanek tarzanek added the question Further information is requested label May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants