Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement graph binary protocol #217

Open
wants to merge 56 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
153b6cd
Generalized protocol enum to graphson alternatives
criminosis Oct 7, 2024
076949f
Removed redundant serializer content type mapping
criminosis Oct 7, 2024
cbaa2bf
Missed import in generalization
criminosis Oct 8, 2024
946d132
Renamed Protocol to IoProtcol following Tinkerpop naming
criminosis Oct 8, 2024
dce5400
Added rstest
criminosis Oct 18, 2024
e426d0d
Serialized message binary parity to java driver for test traversal
criminosis Oct 18, 2024
6cfc168
Centralized on new serializer based message building & reading
criminosis Oct 19, 2024
97a3de6
Formatting
criminosis Oct 19, 2024
0a39163
Removed defunct methods and restored binary deserialization functions
criminosis Oct 19, 2024
36a9989
Successful graph binary traversal
criminosis Oct 19, 2024
c810c07
Handle mapping JSON Value::Null to GValue::Null
criminosis Oct 20, 2024
6de4156
Fix aio compiling errors
criminosis Oct 20, 2024
5330d8f
Set IT client v2 test's deserializer
criminosis Oct 20, 2024
ee6086f
Handle null in GraphSONV2 g_serializer_2
criminosis Oct 20, 2024
f273b14
Formatting
criminosis Oct 20, 2024
5b5108c
Fix graph binary demo IT
criminosis Oct 20, 2024
678792f
Formatting
criminosis Oct 20, 2024
4ca0eb5
Additional value binary serdes
criminosis Oct 22, 2024
455e329
Created graph binary rw tests
criminosis Oct 23, 2024
4a42319
Adding omni test class
criminosis Oct 24, 2024
32b47ac
Handling more test required serialization types
criminosis Oct 24, 2024
815767a
Formatting
criminosis Oct 25, 2024
753dbf7
Made test_unwrap_map pass
criminosis Oct 31, 2024
0368454
Made test_value_map pass
criminosis Nov 1, 2024
54b2457
Made test_element_map pass
criminosis Nov 1, 2024
35d22ed
test_anonymous_traversal_properties_drop
criminosis Nov 1, 2024
490e925
Saving vertex serde progress
criminosis Nov 5, 2024
69da91e
Express possibly null response request id & made vertex serde tests pass
criminosis Nov 5, 2024
6db1750
Fix passing test_has_with_p_steps
criminosis Nov 7, 2024
e570908
Made test_add_v_with_properties pass
criminosis Nov 8, 2024
f2880a0
Made test_by_columns pass
criminosis Nov 8, 2024
2492d09
Made test_select_pop pass
criminosis Nov 8, 2024
e9a0c6d
Implemented string literal GraphBinaryV1Ser
criminosis Nov 8, 2024
baccdd5
All omni tests passing except merge and test_has_with_text_p_step
criminosis Nov 8, 2024
6626be3
test_has_with_text_p_step passing
criminosis Nov 8, 2024
d07679e
Modifed test to reflect has clause not needing predicate for equality…
criminosis Nov 8, 2024
fe2c0de
Fixed merge tests
criminosis Nov 9, 2024
2083461
Supplanted integration_traversal tests into single parameterized test
criminosis Nov 9, 2024
e2cbd2c
Cleaned up loose ends
criminosis Nov 9, 2024
fb3bf91
Parameterized integration_traversal_async
criminosis Nov 12, 2024
1bd393d
Moved merge_capable_serializers to merge_test mod block so its under …
criminosis Nov 12, 2024
0bd9f9a
Consolidated integration_client tests
criminosis Nov 13, 2024
ae34c00
Centralized integration_traversal_async
criminosis Nov 13, 2024
aacd284
Consolidated to serializers templated in common test module
criminosis Nov 13, 2024
f54ea77
Formatting
criminosis Nov 13, 2024
85a58f1
Use centralized serializer template
criminosis Nov 13, 2024
a1bbe79
Created parameterized integration_client_async
criminosis Nov 13, 2024
b4d9015
Formatting
criminosis Nov 13, 2024
5a4931b
Import cleanup
criminosis Nov 13, 2024
97ca6c1
Added serial annotation for test_partial_content to prevent test stat…
criminosis Nov 13, 2024
2d2b13a
Added derive feature flag to GH Action test
criminosis Nov 13, 2024
081ad4e
Implemented map get & try_get backwards compatability logic to not br…
criminosis Nov 13, 2024
5fb1adb
Parameterized custom vertex id tests, added support for JanusGraph's …
criminosis Nov 14, 2024
03cb90a
Populate edge properties in GraphSONV2/V3 & GraphBinary. Also impleme…
criminosis Nov 14, 2024
49799e2
Removed opaque assert_eq losing Err message
criminosis Nov 14, 2024
766f045
Use universal JG test serial annotation to prevent flakey test failure
criminosis Nov 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix passing test_has_with_p_steps
  • Loading branch information
criminosis committed Nov 8, 2024
commit 6db17500ce7a610923bf0aa563b1562414b16732
68 changes: 61 additions & 7 deletions gremlin-client/src/io/graph_binary_v1.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use std::{collections::HashMap, convert::TryInto, iter};
use std::{collections::HashMap, convert::TryInto};

use chrono::{DateTime, TimeZone, Utc};
use tungstenite::http::request;
use uuid::Uuid;

use crate::{
Expand Down Expand Up @@ -332,10 +331,7 @@ impl GraphBinaryV1Ser for &GValue {
//Type code of 0x1e: P
buf.push(P);
buf.push(VALUE_FLAG);
p.operator().to_be_bytes(buf)?;
//Seems we only support 1 parameter predicates?
GraphBinaryV1Ser::to_be_bytes(1i32, buf)?;
p.value().to_be_bytes(buf)?;
p.to_be_bytes(buf)?;
}
GValue::Scope(scope) => {
//Type code of 0x1f: Scope
Expand All @@ -353,6 +349,11 @@ impl GraphBinaryV1Ser for &GValue {
}
}
}
GValue::T(t) => {
buf.push(T);
buf.push(VALUE_FLAG);
t.to_be_bytes(buf)?;
}
GValue::Bool(bool) => {
buf.push(BOOLEAN);
buf.push(VALUE_FLAG);
Expand All @@ -370,6 +371,43 @@ impl GraphBinaryV1Ser for &GValue {
}
}

impl GraphBinaryV1Ser for &crate::structure::P {
fn to_be_bytes(self, buf: &mut Vec<u8>) -> GremlinResult<()> {
self.operator().to_be_bytes(buf)?;
match self.value() {
//Singular values have a length of 1
//But still need to be written fully qualified
scalar @ GValue::Uuid(_) |
scalar @ GValue::Int32(_) |
scalar @ GValue::Int64(_) |
scalar @ GValue::Float(_) |
scalar @ GValue::Double(_) |
scalar @ GValue::String(_) |
scalar @ GValue::Date(_)
=> {
GraphBinaryV1Ser::to_be_bytes(1i32, buf)?;
scalar.to_be_bytes(buf)?;
}
//"Collections" need to be unfurled, we don't write the collection but
//instead just its lengths and then the fully qualified form of each element
GValue::List(list) => {
write_usize_as_i32_be_bytes(list.len(), buf)?;
for item in list.iter() {
item.to_be_bytes(buf)?;
}
}
GValue::Set(set) => {
write_usize_as_i32_be_bytes(set.len(), buf)?;
for item in set.iter() {
item.to_be_bytes(buf)?;
}
}
other => unimplemented!("P serialization of {other:?} not implemented"),
}
Ok(())
}
}

impl GraphBinaryV1Ser for &GKey {
fn to_be_bytes(self, buf: &mut Vec<u8>) -> GremlinResult<()> {
match self {
Expand Down Expand Up @@ -525,7 +563,10 @@ impl GraphBinaryV1Deser for GValue {
))),
}
}
other => unimplemented!("TODO {other}"),
other => {
let remainder: Vec<u8> = bytes.cloned().collect();
unimplemented!("TODO {other}. Remainder: {remainder:?}");
}
}
}
}
Expand All @@ -545,6 +586,18 @@ impl GraphBinaryV1Deser for T {
}
}

impl GraphBinaryV1Ser for &T {
fn to_be_bytes(self, buf: &mut Vec<u8>) -> GremlinResult<()> {
let literal = match self {
T::Id => "id",
T::Key => "key",
T::Label => "label",
T::Value => "value",
};
GValue::String(literal.to_owned()).to_be_bytes(buf)
}
}

fn consume_expected_null_reference_bytes<'a, S: Iterator<Item = &'a u8>>(
bytes: &mut S,
null_reference_descriptor: &str,
Expand Down Expand Up @@ -905,6 +958,7 @@ impl GraphBinaryV1Deser for Uuid {

#[cfg(test)]
mod tests {
use std::iter;
use chrono::DateTime;
use rstest::rstest;
use uuid::uuid;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ use crate::process::traversal::step::select::SelectStep;
use crate::process::traversal::step::until::UntilStep;
use crate::process::traversal::step::where_step::WhereStep;
use crate::process::traversal::TraversalBuilder;
use crate::structure::{Either2, GIDs, IntoPredicate, Labels, T};
use crate::GValue;
use crate::structure::{Either2, GIDs, Labels, T};
use crate::{GValue, ToGValue};

use super::merge_edge::MergeEdgeStep;
use super::merge_vertex::MergeVertexStep;
Expand Down Expand Up @@ -211,7 +211,7 @@ impl AnonymousTraversalSource {

pub fn is<A>(&self, val: A) -> TraversalBuilder
where
A: IntoPredicate,
A: ToGValue,
{
self.traversal.clone().is(val)
}
Expand Down
6 changes: 3 additions & 3 deletions gremlin-client/src/process/traversal/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use crate::process::traversal::step::until::UntilStep;
use crate::process::traversal::step::where_step::WhereStep;

use crate::process::traversal::{Bytecode, Scope};
use crate::structure::{Cardinality, GIDs, IntoPredicate, Labels};
use crate::structure::{Cardinality, GIDs, Labels};
use crate::GValue;

use super::merge_edge::MergeEdgeStep;
Expand Down Expand Up @@ -505,10 +505,10 @@ impl TraversalBuilder {

pub fn is<A>(mut self, val: A) -> Self
where
A: IntoPredicate,
A: ToGValue,
{
self.bytecode
.add_step(String::from("is"), vec![val.into_predicate().into()]);
.add_step(String::from("is"), vec![val.to_gvalue()]);

self
}
Expand Down
6 changes: 3 additions & 3 deletions gremlin-client/src/process/traversal/graph_traversal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ use crate::process::traversal::strategies::{
use crate::process::traversal::{Bytecode, Scope, TraversalBuilder, WRITE_OPERATORS};
use crate::structure::{Cardinality, Labels, Null};
use crate::{
structure::GIDs, structure::GProperty, structure::IntoPredicate, Edge, GValue, GremlinClient,
List, Map, Path, Vertex,
structure::GIDs, structure::GProperty, Edge, GValue, GremlinClient, List, Map, Path, ToGValue,
Vertex,
};
use std::marker::PhantomData;

Expand Down Expand Up @@ -537,7 +537,7 @@ impl<S, E: FromGValue, T: Terminator<E>> GraphTraversal<S, E, T> {

pub fn is<A>(mut self, val: A) -> Self
where
A: IntoPredicate,
A: ToGValue,
{
self.builder = self.builder.is(val);

Expand Down
14 changes: 7 additions & 7 deletions gremlin-client/src/process/traversal/step/has.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::structure::GValue;
use crate::structure::{Either2, TextP, T};
use crate::structure::{IntoPredicate, P};
use crate::structure::T;
use crate::ToGValue;

pub enum HasStepKey {
Str(String),
Expand Down Expand Up @@ -28,7 +28,7 @@ impl Into<HasStepKey> for &str {
pub struct HasStep {
label: Option<String>,
key: HasStepKey,
predicate: Option<Either2<P, TextP>>,
predicate: Option<GValue>,
}

impl From<HasStep> for Vec<GValue> {
Expand All @@ -55,13 +55,13 @@ impl From<HasStep> for Vec<GValue> {
impl<A, B> From<(A, B)> for HasStep
where
A: Into<HasStepKey>,
B: IntoPredicate,
B: ToGValue,
{
fn from(param: (A, B)) -> Self {
HasStep {
label: None,
key: param.0.into(),
predicate: Some(param.1.into_predicate()),
predicate: Some(param.1.to_gvalue()),
}
}
}
Expand All @@ -70,13 +70,13 @@ impl<A, B, C> From<(A, B, C)> for HasStep
where
A: Into<String>,
B: Into<HasStepKey>,
C: IntoPredicate,
C: ToGValue,
{
fn from(param: (A, B, C)) -> Self {
HasStep {
label: Some(param.0.into()),
key: param.1.into(),
predicate: Some(param.2.into_predicate()),
predicate: Some(param.2.to_gvalue()),
}
}
}
Expand Down
23 changes: 23 additions & 0 deletions gremlin-client/tests/integration_traversal_omni.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1677,6 +1677,29 @@ fn test_has_with_p_steps(client: GremlinClient) {

assert_eq!(&20, results[0].get::<i32>().unwrap());

let results = g
.v(())
.has(("test_has_with_p_steps", "age", 20))
.values("age")
.to_list()
.unwrap();

assert_eq!(1, results.len());

assert_eq!(&20, results[0].get::<i32>().unwrap());

let results = g
.v(())
.has_label("test_has_with_p_steps")
.values("age")
.where_(__.is(P::eq(20)))
.to_list()
.unwrap();

assert_eq!(1, results.len());

assert_eq!(&20, results[0].get::<i32>().unwrap());

let results = g
.v(())
.has_label("test_has_with_p_steps")
Expand Down