You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, you can manually set scanSegments to control the number of partitions to split the data into.
We could also provide a segmentSize setting which could be an alternative to scanSegments to control the partitioning. The benefits would be that the same configuration (e.g. segmentSize: 100) could be used with different tables and it would automatically compute an appropriate number of segments that fit the desired size. Whereas with scanSegments you have to adapt its value to every table you migrate (because you want to adjust the number of segments according to each table size).
Currently, you can manually set
scanSegments
to control the number of partitions to split the data into.We could also provide a
segmentSize
setting which could be an alternative toscanSegments
to control the partitioning. The benefits would be that the same configuration (e.g.segmentSize: 100
) could be used with different tables and it would automatically compute an appropriate number of segments that fit the desired size. Whereas withscanSegments
you have to adapt its value to every table you migrate (because you want to adjust the number of segments according to each table size).Originally posted by @julienrf in #143 (comment)
The text was updated successfully, but these errors were encountered: