Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(graphql): reimplement GraphQL API #294

Merged
merged 25 commits into from
Apr 18, 2024
Merged

feat(graphql): reimplement GraphQL API #294

merged 25 commits into from
Apr 18, 2024

Conversation

andrewazores
Copy link
Member

@andrewazores andrewazores commented Feb 21, 2024

Welcome to Cryostat3! 👋

Before contributing, make sure you have:

  • Read the contributing guidelines
  • Linked a relevant issue which this PR resolves
  • Linked any other relevant issues, PR's, or documentation, if any
  • Resolved all conflicts, if any
  • Rebased your branch PR on top of the latest upstream main branch
  • Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
  • Signed all commits using a GPG signature

To recreate commits with GPG signature git fetch upstream && git rebase --force --gpg-sign upstream/main


Fixes: #11

Includes:

Description of the change:

This is a roll-up PR which will include other PRs re-implementing various GraphQL features present in Cryostat 2.4. This first PR includes adding the GraphQL extension, plus implementing the first rootNode query and label selection matching.

Motivation for the change:

Feature parity between the major release versions.

How to manually test:

  1. Run CRYOSTAT_IMAGE=quay.io... bash smoketest.bash -O ...
  2. Try GraphQL queries, ex:
$ http --follow -v --auth=user:pass :8080/api/v2.2/graphql query='query { rootNode { id name nodeType labels { key } descendantTargets(filter: { labels: ["io.openshift.tags in (go, rust, java)"] }) { id name nodeType labels { key value } } } }'

You can also retrieve the generated GraphQL schema to compare against 2.4's queries.graphqls and types.graphqls.

$ http --follow -v --auth=user:pass :8080/api/v3/graphql/schema.graphql

@andrewazores andrewazores added feat New feature or request safe-to-test labels Feb 21, 2024
@github-actions github-actions bot added the needs-triage Needs thorough attention from code reviewers label Feb 21, 2024
@andrewazores andrewazores removed the needs-triage Needs thorough attention from code reviewers label Feb 21, 2024
@andrewazores
Copy link
Member Author

/build_test

Copy link

Workflow started at 2/22/2024, 11:54:15 AM. View Actions Run.

Copy link

CI build and push: All tests pass ✅ (JDK21)
https://github.com/cryostatio/cryostat3/actions/runs/8008111232

@andrewazores
Copy link
Member Author

@aali309 @mwangggg please give this an early "review" (more like look-over and manual test) since this includes the first piece of getting GraphQL back in. This is working off of an upstream feature branch, so the remaining GraphQL queries and mutations will get added to that branch (and this PR) by further PRs which will have their own reviews. Once it's all done this will merge into main so all of GraphQL gets in as a whole.

Copy link

CI build and push: All tests pass ✅ (JDK17)
https://github.com/cryostatio/cryostat3/actions/runs/8008111232

@andrewazores
Copy link
Member Author

One notable difference is in how the Quarkus GraphQL extension handles Map return types: https://quarkus.io/guides/smallrye-graphql#built-in-support-for-maps

2.4:

$ https --auth=user:pass -v :8181/api/v2.2/graphql query='query { rootNode { name labels descendantTargets { name labels target { labels annotations { platform cryostat } } } } }'
POST /api/v2.2/graphql HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: Basic dXNlcjpwYXNz
Connection: keep-alive
Content-Length: 133
Content-Type: application/json
Host: localhost:8181
User-Agent: HTTPie/3.2.2

{
    "query": "query { rootNode { name labels descendantTargets { name labels target { labels annotations { platform cryostat } } } } }"
}


HTTP/1.1 200 OK
content-encoding: gzip
content-length: 841
content-type: application/json

{
    "data": {
        "rootNode": {
            "descendantTargets": [
                {
                    "labels": {},
                    "name": "service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi",
                    "target": {
                        "annotations": {
                            "cryostat": {
                                "HOST": "cryostat",
                                "JAVA_MAIN": "io.cryostat.Cryostat",
                                "PORT": "9091",
                                "REALM": "JDP"
                            },
                            "platform": {}
                        },
                        "labels": {}
                    }
                },
                {
                    "labels": {},
                    "name": "service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi",
                    "target": {
                        "annotations": {
                            "cryostat": {
                                "HOST": "localhost",
                                "PORT": "0",
                                "REALM": "Podman"
                            },
                            "platform": {}
                        },
                        "labels": {
                            "architecture": "x86_64",
                            "build-date": "2024-01-18T10:36:52",
                            "com.redhat.component": "openjdk-17-runtime-ubi8-container",
                            "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI",
                            "description": "Image for Red Hat OpenShift providing OpenJDK 17 runtime",
                            "distribution-scope": "public",
                            "io.buildah.version": "1.33.2",
                            "io.cekit.version": "4.9.1",
                            "io.cryostat.discovery": "true",
                            "io.cryostat.jmxHost": "localhost",
                            "io.cryostat.jmxPort": "0",
                            "io.cryostat.jmxUrl": "service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi",
                            "io.k8s.description": "Platform for running plain Java applications (fat-jar and flat classpath)",
                            "io.k8s.display-name": "Java Applications",
                            "io.openshift.expose-services": "",
                            "io.openshift.tags": "java",
                            "maintainer": "Red Hat OpenJDK <[email protected]>",
                            "name": "ubi8/openjdk-17-runtime",
                            "org.jboss.product": "openjdk",
                            "org.jboss.product.openjdk.version": "17",
                            "org.jboss.product.version": "17",
                            "org.opencontainers.image.documentation": "https://jboss-container-images.github.io/openjdk/",
                            "release": "2.1705573231",
                            "summary": "Image for Red Hat OpenShift providing OpenJDK 17 runtime",
                            "url": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/openjdk-17-runtime/images/1.18-2.1705573231",
                            "usage": "https://jboss-container-images.github.io/openjdk/",
                            "vcs-ref": "12aac04fffa038f171574a2c3f057a2c253f5c27",
                            "vcs-type": "git",
                            "vendor": "Red Hat, Inc.",
                            "version": "1.18"
                        }
                    }
                }
            ],
            "labels": {},
            "name": "Universe"
        }
    }
}

3.0:

$ http --follow --auth=user:pass -v :8080/api/v2.2/graphql query='query { rootNode { name labels { key value } descendantTargets { name labels { key value } target { labels { key value } annotations { platform { key value } cryostat { key value } } } } } }'
POST /api/v2.2/graphql HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: Basic dXNlcjpwYXNz
Connection: keep-alive
Content-Length: 203
Content-Type: application/json
Host: localhost:8080
User-Agent: HTTPie/3.2.2

{
    "query": "query { rootNode { name labels { key value } descendantTargets { name labels { key value } target { labels { key value } annotations { platform { key value } cryostat { key value } } } } } }"
}


HTTP/1.1 308 Permanent Redirect
Content-Encoding: identity
Content-Length: 0
Date: Thu, 22 Feb 2024 21:34:48 GMT
Gap-Auth: user
Location: http://localhost:8080/api/v3/graphql



POST /api/v3/graphql HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate, br
Authorization: Basic dXNlcjpwYXNz
Connection: keep-alive
Content-Length: 203
Content-Type: application/json
Host: localhost:8080
User-Agent: HTTPie/3.2.2

{
    "query": "query { rootNode { name labels { key value } descendantTargets { name labels { key value } target { labels { key value } annotations { platform { key value } cryostat { key value } } } } } }"
}


HTTP/1.1 200 OK
Content-Length: 6650
Content-Type: application/json
Date: Thu, 22 Feb 2024 21:34:48 GMT
Gap-Auth: user

{
    "data": {
        "rootNode": {
            "descendantTargets": [
                {
                    "labels": [],
                    "name": "service:jmx:rmi:///jndi/rmi://cryostat3:9091/jmxrmi",
                    "target": {
                        "annotations": {
                            "cryostat": [
                                {
                                    "key": "JAVA_MAIN",
                                    "value": "/deployments/quarkus-run.jar"
                                },
                                {
                                    "key": "REALM",
                                    "value": "JDP"
                                },
                                {
                                    "key": "PORT",
                                    "value": "9091"
                                },
                                {
                                    "key": "HOST",
                                    "value": "cryostat3"
                                }
                            ],
                            "platform": []
                        },
                        "labels": []
                    }
                },
                {
                    "labels": [
                        {
                            "key": "summary",
                            "value": "Image for Red Hat OpenShift providing OpenJDK 17 runtime"
                        },
                        {
                            "key": "usage",
                            "value": "https://jboss-container-images.github.io/openjdk/"
                        },
                        {
                            "key": "io.buildah.version",
                            "value": "1.33.2"
                        },
                        {
                            "key": "io.cryostat.component",
                            "value": "cryostat3"
                        },
                        {
                            "key": "vcs-ref",
                            "value": "12aac04fffa038f171574a2c3f057a2c253f5c27"
                        },
                        {
                            "key": "com.docker.compose.config-hash",
                            "value": "61679cc13487e43e9f0cde71c9cd5bf1fc1fc6f98164a0e71d49dbc1abaf69f5"
                        },
                        {
                            "key": "io.cekit.version",
                            "value": "4.9.1"
                        },
                        {
                            "key": "com.docker.compose.project.working_dir",
                            "value": "/home/work/workspace/cryostat3/compose"
                        },
                        {
                            "key": "build-date",
                            "value": "2024-01-18T10:36:52"
                        },
                        {
                            "key": "io.cryostat.discovery",
                            "value": "true"
                        },
                        {
                            "key": "com.redhat.license_terms",
                            "value": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI"
                        },
                        {
                            "key": "io.cryostat.jmxPort",
                            "value": "0"
                        },
                        {
                            "key": "org.opencontainers.image.documentation",
                            "value": "https://jboss-container-images.github.io/openjdk/"
                        },
                        {
                            "key": "url",
                            "value": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/openjdk-17-runtime/images/1.18-2.1705573231"
                        },
                        {
                            "key": "com.docker.compose.oneoff",
                            "value": "False"
                        },
                        {
                            "key": "architecture",
                            "value": "x86_64"
                        },
                        {
                            "key": "org.jboss.product.version",
                            "value": "17"
                        },
                        {
                            "key": "com.redhat.component",
                            "value": "openjdk-17-runtime-ubi8-container"
                        },
                        {
                            "key": "vcs-type",
                            "value": "git"
                        },
                        {
                            "key": "name",
                            "value": "ubi8/openjdk-17-runtime"
                        },
                        {
                            "key": "maintainer",
                            "value": "Red Hat OpenJDK <[email protected]>"
                        },
                        {
                            "key": "vendor",
                            "value": "Red Hat, Inc."
                        },
                        {
                            "key": "io.k8s.display-name",
                            "value": "Java Applications"
                        },
                        {
                            "key": "com.docker.compose.container-number",
                            "value": "1"
                        },
                        {
                            "key": "com.docker.compose.project.config_files",
                            "value": "/home/work/workspace/cryostat3/compose/cryostat.yml,/home/work/workspace/cryostat3/compose/db.yml,/home/work/workspace/cryostat3/compose/cryostat-grafana.yml,/home/work/workspace/cryostat3/compose/jfr-datasource.yml,/home/work/workspace/cryostat3/compose/auth_proxy.yml,/home/work/workspace/cryostat3/compose/s3-seaweed.yml"
                        },
                        {
                            "key": "io.cryostat.jmxHost",
                            "value": "localhost"
                        },
                        {
                            "key": "com.docker.compose.service",
                            "value": "cryostat"
                        },
                        {
                            "key": "distribution-scope",
                            "value": "public"
                        },
                        {
                            "key": "org.jboss.product.openjdk.version",
                            "value": "17"
                        },
                        {
                            "key": "release",
                            "value": "2.1705573231"
                        },
                        {
                            "key": "kompose.service.expose",
                            "value": "cryostat3"
                        },
                        {
                            "key": "description",
                            "value": "Image for Red Hat OpenShift providing OpenJDK 17 runtime"
                        },
                        {
                            "key": "io.cryostat.jmxUrl",
                            "value": "service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi"
                        },
                        {
                            "key": "version",
                            "value": "1.18"
                        },
                        {
                            "key": "io.k8s.description",
                            "value": "Platform for running plain Java applications (fat-jar and flat classpath)"
                        },
                        {
                            "key": "io.openshift.tags",
                            "value": "java"
                        },
                        {
                            "key": "com.docker.compose.project",
                            "value": "compose"
                        },
                        {
                            "key": "com.docker.compose.version",
                            "value": "1.29.2"
                        },
                        {
                            "key": "io.openshift.expose-services",
                            "value": ""
                        },
                        {
                            "key": "org.jboss.product",
                            "value": "openjdk"
                        }
                    ],
                    "name": "service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi",
                    "target": {
                        "annotations": {
                            "cryostat": [
                                {
                                    "key": "PORT",
                                    "value": "0"
                                },
                                {
                                    "key": "REALM",
                                    "value": "Podman"
                                },
                                {
                                    "key": "HOST",
                                    "value": "localhost"
                                }
                            ],
                            "platform": []
                        },
                        "labels": [
                            {
                                "key": "org.jboss.product",
                                "value": "openjdk"
                            },
                            {
                                "key": "org.opencontainers.image.documentation",
                                "value": "https://jboss-container-images.github.io/openjdk/"
                            },
                            {
                                "key": "io.cryostat.jmxPort",
                                "value": "0"
                            },
                            {
                                "key": "io.buildah.version",
                                "value": "1.33.2"
                            },
                            {
                                "key": "url",
                                "value": "https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/openjdk-17-runtime/images/1.18-2.1705573231"
                            },
                            {
                                "key": "com.redhat.license_terms",
                                "value": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI"
                            },
                            {
                                "key": "architecture",
                                "value": "x86_64"
                            },
                            {
                                "key": "io.cryostat.jmxHost",
                                "value": "localhost"
                            },
                            {
                                "key": "io.k8s.description",
                                "value": "Platform for running plain Java applications (fat-jar and flat classpath)"
                            },
                            {
                                "key": "io.openshift.expose-services",
                                "value": ""
                            },
                            {
                                "key": "vcs-type",
                                "value": "git"
                            },
                            {
                                "key": "vcs-ref",
                                "value": "12aac04fffa038f171574a2c3f057a2c253f5c27"
                            },
                            {
                                "key": "org.jboss.product.version",
                                "value": "17"
                            },
                            {
                                "key": "distribution-scope",
                                "value": "public"
                            },
                            {
                                "key": "com.docker.compose.service",
                                "value": "cryostat"
                            },
                            {
                                "key": "io.k8s.display-name",
                                "value": "Java Applications"
                            },
                            {
                                "key": "description",
                                "value": "Image for Red Hat OpenShift providing OpenJDK 17 runtime"
                            },
                            {
                                "key": "com.docker.compose.project.working_dir",
                                "value": "/home/work/workspace/cryostat3/compose"
                            },
                            {
                                "key": "vendor",
                                "value": "Red Hat, Inc."
                            },
                            {
                                "key": "release",
                                "value": "2.1705573231"
                            },
                            {
                                "key": "com.docker.compose.version",
                                "value": "1.29.2"
                            },
                            {
                                "key": "build-date",
                                "value": "2024-01-18T10:36:52"
                            },
                            {
                                "key": "io.cryostat.component",
                                "value": "cryostat3"
                            },
                            {
                                "key": "com.docker.compose.project.config_files",
                                "value": "/home/work/workspace/cryostat3/compose/cryostat.yml,/home/work/workspace/cryostat3/compose/db.yml,/home/work/workspace/cryostat3/compose/cryostat-grafana.yml,/home/work/workspace/cryostat3/compose/jfr-datasource.yml,/home/work/workspace/cryostat3/compose/auth_proxy.yml,/home/work/workspace/cryostat3/compose/s3-seaweed.yml"
                            },
                            {
                                "key": "io.cryostat.jmxUrl",
                                "value": "service:jmx:rmi:///jndi/rmi://localhost:0/jmxrmi"
                            },
                            {
                                "key": "com.redhat.component",
                                "value": "openjdk-17-runtime-ubi8-container"
                            },
                            {
                                "key": "maintainer",
                                "value": "Red Hat OpenJDK <[email protected]>"
                            },
                            {
                                "key": "summary",
                                "value": "Image for Red Hat OpenShift providing OpenJDK 17 runtime"
                            },
                            {
                                "key": "version",
                                "value": "1.18"
                            },
                            {
                                "key": "io.cryostat.discovery",
                                "value": "true"
                            },
                            {
                                "key": "io.openshift.tags",
                                "value": "java"
                            },
                            {
                                "key": "com.docker.compose.project",
                                "value": "compose"
                            },
                            {
                                "key": "name",
                                "value": "ubi8/openjdk-17-runtime"
                            },
                            {
                                "key": "kompose.service.expose",
                                "value": "cryostat3"
                            },
                            {
                                "key": "usage",
                                "value": "https://jboss-container-images.github.io/openjdk/"
                            },
                            {
                                "key": "com.docker.compose.config-hash",
                                "value": "61679cc13487e43e9f0cde71c9cd5bf1fc1fc6f98164a0e71d49dbc1abaf69f5"
                            },
                            {
                                "key": "com.docker.compose.container-number",
                                "value": "1"
                            },
                            {
                                "key": "org.jboss.product.openjdk.version",
                                "value": "17"
                            },
                            {
                                "key": "io.cekit.version",
                                "value": "4.9.1"
                            },
                            {
                                "key": "com.docker.compose.oneoff",
                                "value": "False"
                            }
                        ]
                    }
                },
                {
                    "labels": [],
                    "name": "service:jmx:rmi:///jndi/rmi://jfr-datasource:11223/jmxrmi",
                    "target": {
                        "annotations": {
                            "cryostat": [
                                {
                                    "key": "REALM",
                                    "value": "JDP"
                                },
                                {
                                    "key": "HOST",
                                    "value": "jfr-datasource"
                                },
                                {
                                    "key": "PORT",
                                    "value": "11223"
                                },
                                {
                                    "key": "JAVA_MAIN",
                                    "value": "/deployments/quarkus-run.jar"
                                }
                            ],
                            "platform": []
                        },
                        "labels": []
                    }
                }
            ],
            "labels": [],
            "name": "Universe"
        }
    }
}

In short, rather than encoding a Java Map directly into a JSON object where map entries become object properties, map entries instead become JSON objects with the key-value form inside of a list. This is a reasonable design choice on the API side and should not be too much work to adapt on the web-client side.

@aali309
Copy link
Contributor

aali309 commented Feb 27, 2024

Working on environmentNodes.

@andrewazores
Copy link
Member Author

/build_test

Copy link

Workflow started at 2/29/2024, 12:06:43 AM. View Actions Run.

Copy link

CI build and push: All tests pass ✅ (JDK21)
https://github.com/cryostatio/cryostat3/actions/runs/8091291975

Copy link

CI build and push: All tests pass ✅ (JDK17)
https://github.com/cryostatio/cryostat3/actions/runs/8091291975

@andrewazores
Copy link
Member Author

I'm having to rethink the [{ key: 'foo', value: 'bar' }] default map encoding that the quarkus-graphql extension uses. This leads to the situation where a Target serialized in a response to ex. GET /api/v3/targets would have a structure like { "labels": { "foo": "bar" } }, but the same Target embedded in a GraphQL response would have { "labels": [ { "key": "foo", "value": "bar" ] }. It is very much not ideal to have two different serialization formats for the same model type from different parts of the API, because this is annoying and inconsistent for clients to deal with, especially if they want to take data from one API response and use it as part of a request for another.

@andrewazores
Copy link
Member Author

I'm having to rethink the [{ key: 'foo', value: 'bar' }] default map encoding that the quarkus-graphql extension uses. This leads to the situation where a Target serialized in a response to ex. GET /api/v3/targets would have a structure like { "labels": { "foo": "bar" } }, but the same Target embedded in a GraphQL response would have { "labels": [ { "key": "foo", "value": "bar" ] }. It is very much not ideal to have two different serialization formats for the same model type from different parts of the API, because this is annoying and inconsistent for clients to deal with, especially if they want to take data from one API response and use it as part of a request for another.

In #307 , I have updated the Jackson serialization so that any Map<String, String> being written out by the standard HTTP (not GraphQL) API, or the WebSocket, should serialize with the key-value form, the same way as GraphQL. This way it's symmetric, at least for the <String, String> case, across the whole API. Any other kinds of Map will continue to serialize the typical way and directly become JSON objects, which I think is fine. We generally don't return plain Maps anywhere anyway, except for discovery nodes and recordings labels and annotations, where this works and makes sense.

@andrewazores
Copy link
Member Author

/build_test

Copy link

Workflow started at 2/29/2024, 5:06:49 PM. View Actions Run.

Copy link

CI build and push: All tests pass ✅ (JDK17)
https://github.com/cryostatio/cryostat3/actions/runs/8103235759

Copy link

CI build and push: All tests pass ✅ (JDK21)
https://github.com/cryostatio/cryostat3/actions/runs/8103235759

@aali309
Copy link
Contributor

aali309 commented Mar 1, 2024

working on src/test/java/itest/GraphQLTest.java

andrewazores and others added 10 commits April 12, 2024 23:36
* feat(graphql): add top-level query for targetNodes

* return targets with distinct JVM IDs, implement archived recordings query

* implement active recordings query

* refactor enum registration

* implement subqueries (actually nested mutations) for doStartRecording and doSnapshot

* add mbeanMetrics query

* messaging server uses shared ObjectMapper

* add Jackson serialization customizer so Map<String, String> is encoded the same way GraphQL extension does
…ysis card (#312)

* feat(graphql): implement features to support dashboard automated analysis card

* disable JVM ID node filtering, populate archived recordings query with data

* populate aggregate size

* refactor

* refactor to split out class
@andrewazores andrewazores marked this pull request as ready for review April 18, 2024 22:04
@andrewazores andrewazores requested a review from a team as a code owner April 18, 2024 22:04
@andrewazores
Copy link
Member Author

/build_test

Copy link

Workflow started at 4/18/2024, 6:04:43 PM. View Actions Run.

Copy link

OpenAPI schema change detected:

diff --git a/schema/openapi.yaml b/schema/openapi.yaml
index 3290d94..4ce4608 100644
--- a/schema/openapi.yaml
+++ b/schema/openapi.yaml
@@ -539,20 +539,44 @@ paths:
         "200":
           description: OK
         "401":
           description: Not Authorized
         "403":
           description: Not Allowed
       security:
         - SecurityScheme: []
       tags:
         - Recordings
+  /api/beta/recordings/{connectUrl}/{filename}/upload:
+    post:
+      parameters:
+        - in: path
+          name: connectUrl
+          required: true
+          schema:
+            type: string
+        - in: path
+          name: filename
+          required: true
+          schema:
+            type: string
+      responses:
+        "200":
+          description: OK
+        "401":
+          description: Not Authorized
+        "403":
+          description: Not Allowed
+      security:
+        - SecurityScheme: []
+      tags:
+        - Recordings
   /api/beta/recordings/{jvmId}:
     get:
       parameters:
         - in: path
           name: jvmId
           required: true
           schema:
             type: string
       responses:
         "200":

Copy link

CI build and push: All tests pass ✅ (JDK17)
https://github.com/cryostatio/cryostat3/actions/runs/8745266353

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat New feature or request safe-to-test
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Story] GraphQL
2 participants