Feature/vector support #201
Conversation
…ercentage or else fail the build
msmygit
requested review from
jeromatron,
pravinbhat,
Ankitp1342 and
faizalrub-datastax
msmygit
force-pushed
the
feature/vector_support
branch
from
9594224 to
2a04ba9
Compare
| @@ -5,17 +5,17 @@ assertCmd="egrep 'JobSession.* Final ' \${OUTPUT_FILE} | sed 's/^.*Final //'" | |||
| _usage() { | |||
| cat <<EOF | |||
|
|
|||
| usage: $0 -f output_file -a assertFile [-d directory] | |||
| usage: $0 -f output_file -a assert_file [-d directory] | |||
There was a problem hiding this comment.
Cosmetic rename to match file name pattern.
| @@ -0,0 +1,4 @@ | |||
| Read Record Count: 2 | |||
There was a problem hiding this comment.
These whole files are being added to smoke_inflight which isn't being tested actively, but will be moved to smoke directory once we've a docker container that supports vector cql data type.
msmygit
force-pushed
the
feature/vector_support
branch
from
2a04ba9 to
ec1c449
Compare
| <artifactId>java-driver-query-builder</artifactId> | ||
| <version>${java-driver.version}</version> | ||
| </dependency> | ||
| <!-- <dependency> |
There was a problem hiding this comment.
This one was pulled earlier via the SCC, but actually not required. Keeping it here just in case commented out.
| @@ -13,9 +13,10 @@ | |||
| <spark.version>3.3.1</spark.version> | |||
| <scalatest.version>3.2.12</scalatest.version> | |||
| <connector.version>3.2.0</connector.version> | |||
| <cassandra.version>3.11.13</cassandra.version> | |||
| <cassandra.version>5.0-alpha1</cassandra.version> | |||
There was a problem hiding this comment.
FYI - There is a datastax/dse-server:7.0.0-a from this blog which I didn't want to use it on here (just yet) for supporting vector data type.
| @@ -50,7 +50,7 @@ public ByteBuffer encode(Double value, @NotNull ProtocolVersion protocolVersion) | |||
| @Override | |||
| public Double decode(ByteBuffer bytes, @NotNull ProtocolVersion protocolVersion) { | |||
| String stringValue = TypeCodecs.TEXT.decode(bytes, protocolVersion); | |||
| return new Double(stringValue); | |||
| return Double.valueOf(stringValue); | |||
There was a problem hiding this comment.
FYI - This addresses the deprecation of new Double(String value) in newer versions with the correct one to use.
| @@ -74,7 +75,48 @@ | |||
| <groupId>com.datastax.spark</groupId> | |||
| <artifactId>spark-cassandra-connector_${scala.main.version}</artifactId> | |||
| <version>${connector.version}</version> | |||
| <exclusions> | |||
There was a problem hiding this comment.
The reason we're excluding the Java Driver from the SCC dependency and adding separatly is that the version that comes with SCC (i.e. 4.13.0) doesn't support Vectors
msmygit
force-pushed
the
feature/vector_support
branch
from
ec1c449 to
056cda0
Compare
What this PR does: Introduces support for migrating tables with
vectorcql data type.Which issue(s) this PR fixes:
Fixes #200
Checklist: