Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch from jsonpath_lib to jsonpath-rust #1345

Merged
merged 2 commits into from
Nov 23, 2023
Merged

Switch from jsonpath_lib to jsonpath-rust #1345

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b4f0002
Switch from `jsonpath_lib` to `jsonpath-rust`
ilya-bobyr Nov 16, 2023
4ab6e54
Merge branch 'main' into pr/switch-to-jsonpath-rust
clux Nov 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ assert-json-diff = "2.0.1"
garde = { version = "0.16.1", default-features = false, features = ["derive"] }
anyhow = "1.0.44"
futures = "0.3.17"
jsonpath_lib = "0.3.0"
jsonpath-rust = "0.3.4"
kube = { path = "../kube", version = "^0.87.1", default-features = false, features = ["admission"] }
kube-derive = { path = "../kube-derive", version = "^0.87.1", default-features = false } # only needed to opt out of schema
k8s-openapi = { version = "0.20.0", default-features = false }
Expand Down
24 changes: 17 additions & 7 deletions examples/dynamic_jsonpath.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
use anyhow::{Context, Error};
use jsonpath_rust::JsonPathInst;
use k8s_openapi::api::core::v1::Pod;
use kube::{
api::{Api, ListParams},
Expand All @@ -13,19 +15,27 @@ async fn main() -> anyhow::Result<()> {
// Equivalent to `kubectl get pods --all-namespace \
// -o jsonpath='{.items[*].spec.containers[*].image}'`
let field_selector = std::env::var("FIELD_SELECTOR").unwrap_or_default();
let jsonpath = format!(
"{}{}",
"$",
std::env::var("JSONPATH").unwrap_or_else(|_| ".items[*].spec.containers[*].image".into())
);
let jsonpath = {
let path = std::env::var("JSONPATH").unwrap_or_else(|_| ".items[*].spec.containers[*].image".into());
format!("${path}")
.parse::<JsonPathInst>()
.map_err(Error::msg)
.with_context(|| {
format!(
"Failed to parse 'JSONPATH' value as a JsonPath expression.\n
Got: {path}"
)
})?
};

let pods: Api<Pod> = Api::<Pod>::all(client);
let list_params = ListParams::default().fields(&field_selector);
let list = pods.list(&list_params).await?;

// Use the given JSONPATH to filter the ObjectList
let list_json = serde_json::to_value(&list)?;
let res = jsonpath_lib::select(&list_json, &jsonpath).unwrap();
info!("\t\t {:?}", res);
for res in jsonpath.find_slice(&list_json) {
info!("\t\t {}", *res);
}
Ok(())
}
4 changes: 2 additions & 2 deletions kube-client/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ ws = ["client", "tokio-tungstenite", "rand", "kube-core/ws", "tokio/macros"]
oauth = ["client", "tame-oauth"]
oidc = ["client", "form_urlencoded"]
gzip = ["client", "tower-http/decompression-gzip"]
client = ["config", "__non_core", "hyper", "http-body", "tower", "tower-http", "hyper-timeout", "pin-project", "chrono", "jsonpath_lib", "bytes", "futures", "tokio", "tokio-util", "either"]
client = ["config", "__non_core", "hyper", "http-body", "tower", "tower-http", "hyper-timeout", "pin-project", "chrono", "jsonpath-rust", "bytes", "futures", "tokio", "tokio-util", "either"]
jsonpatch = ["kube-core/jsonpatch"]
admission = ["kube-core/admission"]
config = ["__non_core", "pem", "home"]
Expand Down Expand Up @@ -56,7 +56,7 @@ rustls-pemfile = { version = "1.0.0", optional = true }
bytes = { version = "1.1.0", optional = true }
tokio = { version = "1.14.0", features = ["time", "signal", "sync"], optional = true }
kube-core = { path = "../kube-core", version = "=0.87.1" }
jsonpath_lib = { version = "0.3.0", optional = true }
jsonpath-rust = { version = "0.3.4", optional = true }
tokio-util = { version = "0.7.0", optional = true, features = ["io", "codec"] }
hyper = { version = "0.14.13", optional = true, features = ["client", "http1", "stream", "tcp"] }
hyper-rustls = { version = "0.24.0", optional = true }
Expand Down
46 changes: 28 additions & 18 deletions kube-client/src/client/auth/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
header::{InvalidHeaderValue, AUTHORIZATION},
HeaderValue, Request,
};
use jsonpath_lib::select as jsonpath_select;
use jsonpath_rust::JsonPathInst;
use secrecy::{ExposeSecret, SecretString};
use serde::{Deserialize, Serialize};
use thiserror::Error;
Expand Down Expand Up @@ -440,9 +440,9 @@
if let Some(field) = provider.config.get("token-key") {
let json_output: serde_json::Value =
serde_json::from_slice(&output.stdout).map_err(Error::ParseTokenKey)?;
let token = extract_value(&json_output, field)?;
let token = extract_value(&json_output, "token-key", field)?;
if let Some(field) = provider.config.get("expiry-key") {
let expiry = extract_value(&json_output, field)?;
let expiry = extract_value(&json_output, "expiry-key", field)?;
let expiry = expiry
.parse::<DateTime<Utc>>()
.map_err(Error::MalformedTokenExpirationDate)?;
Expand Down Expand Up @@ -474,22 +474,32 @@
}
}

fn extract_value(json: &serde_json::Value, path: &str) -> Result<String, Error> {
let pure_path = path.trim_matches(|c| c == '"' || c == '{' || c == '}');
match jsonpath_select(json, &format!("${pure_path}")) {
Ok(v) if !v.is_empty() => {
if let serde_json::Value::String(res) = v[0] {
Ok(res.clone())
} else {
Err(Error::AuthExec(format!(
"Target value at {pure_path:} is not a string"
)))
}
}

Err(e) => Err(Error::AuthExec(format!("Could not extract JSON value: {e:}"))),
fn extract_value(json: &serde_json::Value, context: &str, path: &str) -> Result<String, Error> {
let parsed_path = path
.trim_matches(|c| c == '"' || c == '{' || c == '}')
.parse::<JsonPathInst>()
.map_err(|err| {
Error::AuthExec(format!(

Check warning on line 482 in kube-client/src/client/auth/mod.rs

View check run for this annotation

Codecov / codecov/patch

kube-client/src/client/auth/mod.rs#L482 

Added line #L482 was not covered by tests
"Failed to parse {context:?} as a JsonPath: {path}\n
Error: {err}"
))
})?;

let res = parsed_path.find_slice(json);

let Some(res) = res.into_iter().next() else {
return Err(Error::AuthExec(format!(

Check warning on line 491 in kube-client/src/client/auth/mod.rs

View check run for this annotation

Codecov / codecov/patch

kube-client/src/client/auth/mod.rs#L491 

Added line #L491 was not covered by tests
"Target {context:?} value {path:?} not found"
)));
};
Comment on lines +488 to +494
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will it always work to use find_slice here? to me that sounds like we expect the query to return a vector of things, but curious if this will work if we do a query returning a single item like '{.items[3].metadata.name}'

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ya so find_slice will return an empty vector if the query doesn't match and will return anything else as elements in an array, including if it results in just a single item. The item will just be returned in the vector. See: https://github.com/besok/jsonpath-rust/blob/main/src/lib.rs#L450-L460.

Test here shows find_slice() returning a single item in a vector: https://github.com/besok/jsonpath-rust/blob/main/src/lib.rs#L1194-L1198

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Existing logic here is a bit suspicious.
If any elements are selected by the JSONPath, it would always use the first element, even if there is more than one result.

I wonder if it would be better to fail with an error, if there are multiple results.
If someone really wants to use the first result, they can always clarify their selector expression.
On the other hand, choosing the first result could hide an error and complicate debugging.

I can send a separate PR to change this behavior.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is ultimately only for a small amount of niche behaviour; local auth where the usage from exec based providers needs to grab one key - so it's not as exposed as something like kubectl jsonpath querying. If there's a clear way to improve it feel free to raise a PR, but I don't see this as a big problem atm.


_ => Err(Error::AuthExec(format!("Target value {pure_path:} not found"))),
if let Some(val) = res.as_str() {
Ok(val.to_owned())
} else {
Err(Error::AuthExec(format!(

Check warning on line 499 in kube-client/src/client/auth/mod.rs

View check run for this annotation

Codecov / codecov/patch

kube-client/src/client/auth/mod.rs#L499 

Added line #L499 was not covered by tests
"Target {:?} value {:?} is not a string: {:?}",
Dav1dde marked this conversation as resolved.
Show resolved Hide resolved
context, path, *res

Check warning on line 501 in kube-client/src/client/auth/mod.rs

View check run for this annotation

Codecov / codecov/patch

kube-client/src/client/auth/mod.rs#L501 

Added line #L501 was not covered by tests
)))
}
}

Expand Down
Loading