Skip to content

Commit 1765e90

Browse files
nvautonvauto
authored
[auto-merge] branch-25.02 to branch-25.04 [skip ci] [bot] (#491)
auto-merge triggered by github actions on `branch-25.02` to create a PR keeping `branch-25.04` up-to-date. If this PR is unable to be merged due to conflicts, it will remain open until manually fix.
2 parents ff6ae3f + 59b4920 commit 1765e90

12 files changed

+12
-12
lines changed

‎examples/ML+DL-Examples/Spark-DL/dl_inference/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Deep Learning Inference on Spark
22

3-
Example notebooks demonstrating **distributed deep learning inference** using the [predict_batch_udf](https://developer.nvidia.com/blog/distributed-deep-learning-made-easy-with-spark-3-4/) introduced in Spark 3.4.0.
3+
Example notebooks demonstrating **distributed deep learning inference** using the [predict_batch_udf](https://developer.nvidia.com/blog/distributed-deep-learning-made-easy-with-spark-3-4/#distributed_inference) introduced in Spark 3.4.0.
44
These notebooks also demonstrate integration with [Triton Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html), an open-source, GPU-accelerated serving solution for DL.
55

66
## Contents:

‎examples/ML+DL-Examples/Spark-DL/dl_inference/huggingface/conditional_generation_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -561,7 +561,7 @@
561561
"source": [
562562
"## Inference using Spark DL API\n",
563563
"\n",
564-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
564+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
565565
"\n",
566566
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
567567
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/huggingface/conditional_generation_torch.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -622,7 +622,7 @@
622622
"source": [
623623
"## Inference using Spark DL API\n",
624624
"\n",
625-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
625+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
626626
"\n",
627627
"- predict_batch_fn uses PyTorch APIs to load the model and return a predict function which operates on numpy arrays \n",
628628
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/huggingface/pipelines_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -564,7 +564,7 @@
564564
"source": [
565565
"## Inference using Spark DL API\n",
566566
"\n",
567-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
567+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
568568
"\n",
569569
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
570570
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/huggingface/pipelines_torch.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -461,7 +461,7 @@
461461
"source": [
462462
"## Inference using Spark DL API\n",
463463
"\n",
464-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
464+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
465465
"\n",
466466
"- predict_batch_fn uses PyTorch APIs to load the model and return a predict function which operates on numpy arrays \n",
467467
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/huggingface/sentence_transformers_torch.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -395,7 +395,7 @@
395395
"source": [
396396
"## Inference using Spark DL API\n",
397397
"\n",
398-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
398+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
399399
"\n",
400400
"- predict_batch_fn uses PyTorch APIs to load the model and return a predict function which operates on numpy arrays \n",
401401
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/pytorch/housing_regression_torch.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1151,7 +1151,7 @@
11511151
"source": [
11521152
"## Inference using Spark DL API\n",
11531153
"\n",
1154-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
1154+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
11551155
"\n",
11561156
"- predict_batch_fn uses PyTorch APIs to load the model and return a predict function which operates on numpy arrays \n",
11571157
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/pytorch/image_classification_torch.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1743,7 +1743,7 @@
17431743
"source": [
17441744
"## Inference using Spark DL API\n",
17451745
"\n",
1746-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
1746+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
17471747
"\n",
17481748
"- predict_batch_fn uses PyTorch APIs to load the model and return a predict function which operates on numpy arrays \n",
17491749
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/tensorflow/image_classification_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1000,7 +1000,7 @@
10001000
"source": [
10011001
"## Inference using Spark DL API\n",
10021002
"\n",
1003-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
1003+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
10041004
"\n",
10051005
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
10061006
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/tensorflow/keras_preprocessing_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1160,7 +1160,7 @@
11601160
"source": [
11611161
"## Inference using Spark DL API\n",
11621162
"\n",
1163-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
1163+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
11641164
"\n",
11651165
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
11661166
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/tensorflow/keras_resnet50_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -613,7 +613,7 @@
613613
"source": [
614614
"## Inference using Spark DL API\n",
615615
"\n",
616-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
616+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
617617
"\n",
618618
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
619619
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

‎examples/ML+DL-Examples/Spark-DL/dl_inference/tensorflow/text_classification_tf.ipynb

+1-1
Original file line numberDiff line numberDiff line change
@@ -1374,7 +1374,7 @@
13741374
"source": [
13751375
"## Inference using Spark DL API\n",
13761376
"\n",
1377-
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/3.4.0/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
1377+
"Distributed inference using the PySpark [predict_batch_udf](https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.functions.predict_batch_udf.html#pyspark.ml.functions.predict_batch_udf):\n",
13781378
"\n",
13791379
"- predict_batch_fn uses Tensorflow APIs to load the model and return a predict function which operates on numpy arrays \n",
13801380
"- predict_batch_udf will convert the Spark DataFrame columns into numpy input batches for the predict function"

0 commit comments

Comments
 (0)