From 9346498a90e4be7d98c540ccb1b6ae40464a3a54 Mon Sep 17 00:00:00 2001 From: agriffith50 Date: Tue, 17 Jan 2023 13:30:15 -0500 Subject: [PATCH] resolving pr comments Signed-off-by: agriffith50 --- docs/modelserving/data_plane/v1_protocol.md | 1 - docs/modelserving/data_plane/v2_protocol.md | 13 +++---------- docs/modelserving/v1beta1/serving_runtime.md | 8 ++++---- 3 files changed, 7 insertions(+), 15 deletions(-) diff --git a/docs/modelserving/data_plane/v1_protocol.md b/docs/modelserving/data_plane/v1_protocol.md index 382921584..a94f7a718 100644 --- a/docs/modelserving/data_plane/v1_protocol.md +++ b/docs/modelserving/data_plane/v1_protocol.md @@ -12,7 +12,6 @@ KServe's V1 protocol offers a standardized prediction workflow across all model Note: The response payload in V1 protocol is not strictly enforced. A custom server define and return its own response payload. We encourage using the KServe defined response payload for consistency. -TODO: make sure list models/model ready is correct. ## API Definitions diff --git a/docs/modelserving/data_plane/v2_protocol.md b/docs/modelserving/data_plane/v2_protocol.md index 190732f19..61c55bd79 100644 --- a/docs/modelserving/data_plane/v2_protocol.md +++ b/docs/modelserving/data_plane/v2_protocol.md @@ -48,7 +48,7 @@ For example, if a model does not implement a version, the Model Metadata request | Server Ready | The “server ready” health API indicates if all the models are ready for inferencing. The “server ready” health API can be used directly to implement the Kubernetes readinessProbe | | Server Live | The “server live” health API indicates if the inference server is able to receive and respond to metadata and inference requests. The “server live” API can be used directly to implement the Kubernetes livenessProbe. | | Server Metadata | The "server metadata" API returns details describing the server. | - +| Model Ready | The “model ready” health API indicates if a specific model is ready for inferencing. The model name and (optionally) version must be available in the URL. | ### Health/Readiness/Liveness Probes @@ -140,15 +140,8 @@ status (typically 400). The HTTP body must contain the * “error” : The descriptive message for the error. -The per-model metadata endpoint provides information about a model. A -model metadata request is made with an HTTP GET to a model metadata -endpoint. In the corresponding response the HTTP body contains the -[Model Metadata Response JSON Object](#model-metadata-response-json-object) -or the -[Model Metadata Response JSON Error Object](#model-metadata-response-json-error-object). -The model name and (optionally) version must be available in the -URL. If a version is not provided the server may choose a version -based on its own policies or return an error. +The per-model metadata endpoint provides information about a model. A model metadata request is made with an HTTP GET to a model metadata endpoint. In the corresponding response the HTTP body contains the [Model Metadata Response JSON Object](#model-metadata-response-json-object) or the [Model Metadata Response JSON Error Object](#model-metadata-response-json-error-object). +The model name and (optionally) version must be available in the URL. If a version is not provided the server may choose a version based on its own policies or return an error. --- diff --git a/docs/modelserving/v1beta1/serving_runtime.md b/docs/modelserving/v1beta1/serving_runtime.md index 220cba3b8..638072cd8 100644 --- a/docs/modelserving/v1beta1/serving_runtime.md +++ b/docs/modelserving/v1beta1/serving_runtime.md @@ -30,16 +30,16 @@ The table below identifies each of the model serving runtimes supported by KServ | ------------- | ------------- | ------------- | ------------- | ------------- | ------------- |--------------------------------------| | [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model) | | [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.0.0 (MLServer) | 3 | [LightGBM Iris V2](./lightgbm) | -| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1 | -- | v0.10.0 (KServe) | 3 | [LightGBM Iris](./lightgbm) | +| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1 | -- | v0.10 (KServe) | 3 | [LightGBM Iris](./lightgbm) | | [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.0.0 (MLServer) | 1 | [MLFLow wine-classifier](./mlflow) | -| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1 | -- | v0.10.0 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml) | +| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1 | -- | v0.10 (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml) | | [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2| v1.0.0 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2) | -| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1 | -- | v0.10.0 (KServe) | 1 | [SKLearn Iris](./sklearn/v2) | +| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1 | -- | v0.10 (KServe) | 1 | [SKLearn Iris](./sklearn/v2) | | [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow) | | [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.7.0 (TorchServe) | 1 | [TorchServe mnist](./torchserve) | | [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md)| v2 | v2 | 21.09-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 1 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html)| [Torchscript cifar](triton/torchscript) | | [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.0.0 (MLServer) | 1 | [XGBoost Iris V2](./xgboost) | -| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1 | -- | v0.10.0 (KServe) | 1 | [XGBoost Iris](./xgboost) | +| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1 | -- | v0.10 (KServe) | 1 | [XGBoost Iris](./xgboost) |