Merge pull request #1657 from vespa-engine/kkraune/move-semantic

Demote stale sample app

Author: kkraune

README.md

@@ -7,7 +7,7 @@
 </picture>
 
 
-# Vespa sample applications
+# Vespa Sample Applications
 The [Vespa](https://vespa.ai/) sample applications are created to run both self-hosted and on Vespa Cloud.
 You can easily deploy the sample applications to Vespa Cloud without changing the files -
 just follow the same steps as for
@@ -21,7 +21,7 @@ and see [operations](examples/operations) for operational examples.
 
 
 
-## Getting started
+## Getting Started
 [![logo](/assets/vespa-logomark-tiny.png) Album Recommendations](album-recommendation/) is the intro application to Vespa.
 Learn how to configure the schema for simple recommendation and search use cases.
 
@@ -66,14 +66,10 @@ and searching efficiently over these representations.
 [![logo](/assets/vespa-logomark-tiny.png) Managed Vector Search using Vespa Cloud](https://github.com/vespa-cloud/vector-search)
 describes how to unlock the full potential of multimodal AI-powered vector representations using Vespa Cloud.
 
-[![logo](/assets/vespa-logomark-tiny.png) Simple Semantic Search](simple-semantic-search/)
-demonstrates indexed vector search using [HNSW](https://docs.vespa.ai/en/reference/schema-reference.html#index-hnsw),
-creating embedding vectors from a transformer language model inside Vespa, and hybrid text and semantic ranking.
-This app also demonstrates using native Vespa embedders.
-
 [![logo](/assets/vespa-logomark-tiny.png) Vespa Multi-Vector Indexing with HNSW](multi-vector-indexing/) and
 [![logo](/assets/vespa-logomark-tiny.png) Pyvespa: Multi-vector indexing with HNSW](https://pyvespa.readthedocs.io/en/latest/examples/multi-vector-indexing.html)
 demonstrate how to index multiple vectors per document field for semantic search for longer documents.
+These are more advanced than the Hybrid Search examples in the Getting Started section.
 
 [![logo](/assets/vespa-logomark-tiny.png) Vector Streaming Search](vector-streaming-search)
 uses vector streaming search for naturally partitioned data, see the

custom-embeddings/README.md

@@ -67,10 +67,10 @@ Download a frozen embedding model file, see
 <pre data-test="exec">
 $ mkdir -p models
 $ curl -L -o models/tokenizer.json \
-  https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json
+  https://raw.githubusercontent.com/vespa-engine/sample-apps/master/examples/model-exporting/model/tokenizer.json
 
 $ curl -L -o models/frozen.onnx \
-  https://github.com/vespa-engine/sample-apps/raw/master/simple-semantic-search/model/e5-small-v2-int8.onnx
+  https://github.com/vespa-engine/sample-apps/raw/master/examples/model-exporting/model/e5-small-v2-int8.onnx
 
 $ cp models/frozen.onnx models/tuned.onnx
 </pre>

examples/README.md

@@ -109,11 +109,16 @@ It is built using [FastHTML](https://www.fastht.ml/) and written in pure Python.
 
 
 ### ONNX Model export and deployment example
-Use ![logo](/assets/vespa-logomark-tiny.png) [model-deployment](model-deployment) to generate a model in ONNX format in the models directory,
+Use [![logo](/assets/vespa-logomark-tiny.png) model-deployment](model-deployment) to generate a model in ONNX format in the models directory,
 by running the ONNXModelExport notebook.
 <!-- ToDo: improve this -->
 
 
+### Model exporting
+[![logo](/assets/vespa-logomark-tiny.png) Model exporting](model-exporting)
+demonstrates how to export a Huggingface sentence-transformer model to ONNX format.
+
+
 ### Reranker sample application
 [![logo](/assets/vespa-logomark-tiny.png) reranker](reranker) is a stateless application which re-ranks results obtained from another Vespa application.
 While this does not result in good performance and is not recommended for production,

examples/fasthtml-demo/deploy_app.ipynb

@@ -191,13 +191,13 @@
     "                Parameter(\n",
     "                    \"transformer-model\",\n",
     "                    {\n",
-    "                        \"url\": \"https://github.com/vespa-engine/sample-apps/raw/master/simple-semantic-search/model/e5-small-v2-int8.onnx\"\n",
+    "                        \"url\": \"https://github.com/vespa-engine/sample-apps/raw/master/examples/model-exporting/model/e5-small-v2-int8.onnx\"\n",
     "                    },\n",
     "                ),\n",
     "                Parameter(\n",
     "                    \"tokenizer-model\",\n",
     "                    {\n",
-    "                        \"url\": \"https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json\"\n",
+    "                        \"url\": \"https://raw.githubusercontent.com/vespa-engine/sample-apps/master/examples/model-exporting/model/tokenizer.json\"\n",
     "                    },\n",
     "                ),\n",
     "            ],\n",

simple-semantic-search/.gitignore examples/model-exporting/.gitignore

@@ -7,21 +7,17 @@
   <img alt="#Vespa" width="200" src="https://assets.vespa.ai/logos/Vespa-logo-dark-RGB.svg" style="margin-bottom: 25px;">
 </picture>
 
-# Vespa sample applications - Simple semantic search
-
-A minimal semantic search application: 
-- Query and document text is converted to embeddings by the application using Vespa's [embedder functionality](https://docs.vespa.ai/en/embedding.html#huggingface-embedder).
-- Search by embedding or text match and use [reciprocal rank fusion](https://docs.vespa.ai/en/phased-ranking.html#cross-hit-normalization-including-reciprocal-rank-fusion) to fuse 
-different rankings.
-
-<p data-test="run-macro init-deploy simple-semantic-search">
-minimum-required-vespa-version="8.311.28"
-</p>
+# Vespa sample applications - Model Exporting
+This example demonstrates how to export a Huggingface sentence-transformer model to ONNX format.
 
 ## To try this application
 
 Follow [Vespa getting started](https://cloud.vespa.ai/en/getting-started)
-through the <code>vespa deploy</code> step, cloning `simple-semantic-search` instead of `album-recommendation`.
+through the <code>vespa deploy</code> step, using this example instead of `album-recommendation`.
+
+<p data-test="run-macro init-deploy examples/model-exporting">
+minimum-required-vespa-version="8.311.28"
+</p>
 
 Feed documents (this includes embed inference in Vespa):
 
@@ -55,11 +51,6 @@ Remove the container after use:
 $ docker rm -f vespa
 </pre>
 
-## Ready for production
-
-The E5-small-v2 [embedding model](https://huggingface.co/intfloat/e5-small-v2) used in this sample application
-is suitable for production use and will produce good results in many domains without fine-tuning,
-especially when combined with text match features.
 
 ## Model exporting
 Transformer-based embedding models have named inputs and outputs that must  

simple-semantic-search/.vespaignore examples/model-exporting/.vespaignore

@@ -7,8 +7,8 @@
 
         <!-- See https://docs.vespa.ai/en/embedding.html#huggingface-embedder -->
         <component id="e5" type="hugging-face-embedder">
-            <transformer-model url="https://github.com/vespa-engine/sample-apps/raw/master/simple-semantic-search/model/e5-small-v2-int8.onnx"/>
-            <tokenizer-model url="https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json"/>
+            <transformer-model url="https://github.com/vespa-engine/sample-apps/raw/master/examples/model-exporting/model/e5-small-v2-int8.onnx"/>
+            <tokenizer-model url="https://raw.githubusercontent.com/vespa-engine/sample-apps/master/examples/model-exporting/model/tokenizer.json"/>
             <prepend> <!-- E5 prompt instructions -->
                 <query>query:</query>
                 <document>passage:</document>

simple-semantic-search/README.md examples/model-exporting/README.md

@@ -64,7 +64,7 @@ $ vespa clone multilingual-search my-app && cd my-app
 This sample app embedder configuration in [services.xml](services.xml) points to a quantized model.
 
 Alternatively, [export your own model](https://docs.vespa.ai/en/onnx.html#onnx-export), see also the
-export script in [simple-semantic-search](../simple-semantic-search/README.md).
+export script in [simple-semantic-search](/examples/model-exporting/README.md).
 
 Deploy the application :
 <pre data-test="exec" data-test-assert-contains="Success">

simple-semantic-search/export_hf_model_from_hf.py examples/model-exporting/export_hf_model_from_hf.py

@@ -42,10 +42,9 @@
 {"update": "id:vespaapps-p:paragraph::vespaapps/multi-vector-indexing/-hybrid-search-and-ranking", "fields": {"questions": {"assign": ["What is hybrid rank-profile?", "What additional features does hybrid rank-profile calculate?", "What is the purpose of second-phase ranking?"]}}}
 {"update": "id:vespaapps-p:paragraph::vespaapps/multi-vector-indexing/-hybrid-search-and-filter", "fields": {"questions": {"assign": ["What is hybrid ranking?", "What is filtering?"]}}}
 {"update": "id:vespaapps-p:paragraph::vespaapps/multi-vector-indexing/-cleanup", "fields": {"questions": {"assign": ["How to tear down a Docker container?"]}}}
-{"update": "id:vespaapps-p:paragraph::vespaapps/simple-semantic-search/-vespa-sample-applications---simple-semantic-search", "fields": {"questions": {"assign": ["What is a semantic search?", "How does Vespa enable semantic search?", "What is the minimum Vespa version needed for semantic search?", "What are the requirements to use Vespa semantic search?", "What is the benefit of using Vespa semantic search over regular text match search?"]}}}
-{"update": "id:vespaapps-p:paragraph::vespaapps/simple-semantic-search/-to-try-this", "fields": {"questions": {"assign": ["What is nearestNeighbor in Vespa?", "What is the 'vespa deploy' step?", "What is simple-semantic-search?", "How do you deploy simple-semantic-search?"]}}}
-{"update": "id:vespaapps-p:paragraph::vespaapps/simple-semantic-search/-ready-for-production", "fields": {"questions": {"assign": ["What are the domains suited for the model?", "Does the model require fine-tuning?", "What features can improve the model results?"]}}}
-{"update": "id:vespaapps-p:paragraph::vespaapps/simple-semantic-search/-model-exporting", "fields": {"questions": {"assign": ["How to export Huggingface model?", "What is onnx format?"]}}}
+{"update": "id:vespaapps-p:paragraph::vespaapps/examples/model-exporting/-to-try-this", "fields": {"questions": {"assign": ["What is nearestNeighbor in Vespa?", "What is the 'vespa deploy' step?", "What is simple-semantic-search?", "How do you deploy simple-semantic-search?"]}}}
+{"update": "id:vespaapps-p:paragraph::vespaapps/examples/model-exporting/-ready-for-production", "fields": {"questions": {"assign": ["What are the domains suited for the model?", "Does the model require fine-tuning?", "What features can improve the model results?"]}}}
+{"update": "id:vespaapps-p:paragraph::vespaapps/examples/model-exporting/-model-exporting", "fields": {"questions": {"assign": ["How to export Huggingface model?", "What is onnx format?"]}}}
 {"update": "id:vespaapps-p:paragraph::vespaapps/billion-scale-image-search/-billion-scale-image-search", "fields": {"questions": {"assign": ["What is the image search application?", "What is multimodal AI?", "What is the billion-scale-vector-search application?"]}}}
 {"update": "id:vespaapps-p:paragraph::vespaapps/billion-scale-image-search/-the-vector-dataset", "fields": {"questions": {"assign": ["What is LAION 5B?", "What is CLIP?", "What is StableDiffusion?", "What is the use of LAION 5B?", "What does LAION 5B consist of?"]}}}
 {"update": "id:vespaapps-p:paragraph::vespaapps/billion-scale-image-search/-use-cases", "fields": {"questions": {"assign": ["What is LAION dataset?", "What is CLIP retrieval?", "What is Vespa's query language?"]}}}

simple-semantic-search/export_model_from_hf.py examples/model-exporting/export_model_from_hf.py

@@ -1,13 +1,13 @@
 # Copyright Vespa.ai. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
 
 urls:
+    - examples/model-exporting/README.md
     - splade/README.md
     - colbert-long/README.md
     - multi-vector-indexing/README.md
     - vector-streaming-search/README.md
     - custom-embeddings/README.md
     - commerce-product-ranking/README.md
-    - simple-semantic-search/README.md
     - colbert/README.md
     - text-image-search/README.md
     - text-search/README.md

simple-semantic-search/ext/1.json examples/model-exporting/ext/1.json

@@ -8,8 +8,8 @@ mkdir -p $DIR
 
 echo "Downloading encoder model" 
 curl -L -o $DIR/e5-small-v2-int8.onnx \
-https://github.com/vespa-engine/sample-apps/blob/master/simple-semantic-search/model/e5-small-v2-int8.onnx?raw=true
+https://github.com/vespa-engine/sample-apps/blob/master/examples/model-exporting/model/e5-small-v2-int8.onnx?raw=true
 
 echo "Downloading vocab" 
 curl -L -o $DIR/tokenizer.json \
-https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json
+https://raw.githubusercontent.com/vespa-engine/sample-apps/master/examples/model-exporting/model/tokenizer.json

simple-semantic-search/ext/2.json examples/model-exporting/ext/2.json

@@ -7,8 +7,8 @@
 
         <!-- See https://docs.vespa.ai/en/embedding.html#huggingface-embedder -->
         <component id="e5-small-q" type="hugging-face-embedder">
-            <transformer-model url="https://github.com/vespa-engine/sample-apps/raw/master/simple-semantic-search/model/e5-small-v2-int8.onnx"/>
-            <tokenizer-model url="https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json"/>
+            <transformer-model url="https://github.com/vespa-engine/sample-apps/raw/master/examples/model-exporting/model/e5-small-v2-int8.onnx"/>
+            <tokenizer-model url="https://raw.githubusercontent.com/vespa-engine/sample-apps/master/examples/model-exporting/model/tokenizer.json"/>
             <!-- E5 prompt instructions -->
             <prepend>
                 <query>query:</query>