Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test and documentation updates for image and audio data ingestion #56

Open
wants to merge 3 commits into
base: mmqna-phase3
Choose a base branch
from
Open

Test and documentation updates for image and audio data ingestion #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
064288a
Add test for image and audio data ingestion
dmsuehir Feb 14, 2025
5255518
README updates
dmsuehir Feb 14, 2025
77f934f
Add Gaudi tests
dmsuehir Feb 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion MultimodalQnA/docker_compose/intel/cpu/xeon/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ curl --silent --write-out "HTTPSTATUS:%{http_code}" \
-X POST -F "files=@./${image_fn}"
```

Now, test the microservice with posting a custom caption along with an image and a PDF containing images and text.
Now, test the microservice with posting a custom caption along with an image and a PDF containing images and text. The image caption can be provided as a text (`.txt`) or as spoken audio (`.wav` or `.mp3`).

```bash
curl --silent --write-out "HTTPSTATUS:%{http_code}" \
Expand Down
2 changes: 1 addition & 1 deletion MultimodalQnA/docker_compose/intel/hpu/gaudi/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,7 @@ curl --silent --write-out "HTTPSTATUS:%{http_code}" \
-X POST -F "files=@./${image_fn}"
```

Now, test the microservice with posting a custom caption along with an image and a PDF containing images and text.
Now, test the microservice with posting a custom caption along with an image and a PDF containing images and text. The image caption can be provided as a text (`.txt`) or as spoken audio (`.wav` or `.mp3`).

```bash
curl --silent --write-out "HTTPSTATUS:%{http_code}" \
Expand Down
22 changes: 16 additions & 6 deletions MultimodalQnA/tests/test_compose_on_gaudi.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ WORKPATH=$(dirname "$PWD")
LOG_PATH="$WORKPATH/tests"
ip_address=$(hostname -I | awk '{print $1}')

export image_fn="apple.png"
export image_fn="sample.png"
export video_fn="WeAreGoingOnBullrun.mp4"
export audio_fn="sample_audio.mp3"
export caption_fn="apple.txt"
export audio_fn="sample.mp3"
export caption_fn="sample.txt"
export pdf_fn="nke-10k-2023.pdf"

function check_service_ready() {
Expand Down Expand Up @@ -139,16 +139,19 @@ function validate_service() {
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-caption"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-ingest-image-audio"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./${audio_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-ingest"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./apple.txt" -H 'Content-Type: multipart/form-data' "$URL")
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./${caption_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-pdf"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${pdf_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep_get"* ]]; then
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -H 'Content-Type: application/json' "$URL")
elif [[ $SERVICE_NAME == *"dataprep_del"* ]]; then
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d '{"file_path": "apple.txt"}' -H 'Content-Type: application/json' "$URL")
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d '{"file_path": "${caption_fn}"}' -H 'Content-Type: application/json' "$URL")
else
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d "$INPUT_DATA" -H 'Content-Type: application/json' "$URL")
fi
Expand Down Expand Up @@ -220,12 +223,19 @@ function validate_microservices() {
"dataprep-multimodal-redis-transcript" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with Image & Caption Ingestion"
echo "Validating Data Prep with Image & Text Caption Ingestion"
validate_service \
"${DATAPREP_INGEST_SERVICE_ENDPOINT}" \
"Data preparation succeeded" \
"dataprep-multimodal-redis-ingest" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with Image & Audio Caption Ingestion"
validate_service \
"${DATAPREP_INGEST_SERVICE_ENDPOINT}" \
"Data preparation succeeded" \
"dataprep-multimodal-redis-ingest-image-audio" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with PDF"
validate_service \
Expand Down
22 changes: 16 additions & 6 deletions MultimodalQnA/tests/test_compose_on_xeon.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ WORKPATH=$(dirname "$PWD")
LOG_PATH="$WORKPATH/tests"
ip_address=$(hostname -I | awk '{print $1}')

export image_fn="apple.png"
export image_fn="sample.png"
export video_fn="WeAreGoingOnBullrun.mp4"
export audio_fn="sample_audio.mp3"
export caption_fn="apple.txt"
export audio_fn="sample.mp3"
export caption_fn="sample.txt"
export pdf_fn="nke-10k-2023.pdf"

function check_service_ready() {
Expand Down Expand Up @@ -136,16 +136,19 @@ function validate_service() {
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-caption"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-ingest-image-audio"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./${audio_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-ingest"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./apple.txt" -H 'Content-Type: multipart/form-data' "$URL")
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${image_fn}" -F "files=@./${caption_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep-multimodal-redis-pdf"* ]]; then
cd $LOG_PATH
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -F "files=@./${pdf_fn}" -H 'Content-Type: multipart/form-data' "$URL")
elif [[ $SERVICE_NAME == *"dataprep_get"* ]]; then
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -H 'Content-Type: application/json' "$URL")
elif [[ $SERVICE_NAME == *"dataprep_del"* ]]; then
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d '{"file_path": "apple.txt"}' -H 'Content-Type: application/json' "$URL")
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d '{"file_path": "${caption_fn}"}' -H 'Content-Type: application/json' "$URL")
else
HTTP_RESPONSE=$(curl --silent --write-out "HTTPSTATUS:%{http_code}" -X POST -d "$INPUT_DATA" -H 'Content-Type: application/json' "$URL")
fi
Expand Down Expand Up @@ -217,13 +220,20 @@ function validate_microservices() {
"dataprep-multimodal-redis-transcript" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with Image & Caption Ingestion"
echo "Validating Data Prep with Image & Text Caption Ingestion"
validate_service \
"${DATAPREP_INGEST_SERVICE_ENDPOINT}" \
"Data preparation succeeded" \
"dataprep-multimodal-redis-ingest" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with Image & Audio Caption Ingestion"
validate_service \
"${DATAPREP_INGEST_SERVICE_ENDPOINT}" \
"Data preparation succeeded" \
"dataprep-multimodal-redis-ingest-image-audio" \
"dataprep-multimodal-redis"

echo "Validating Data Prep with PDF"
validate_service \
"${DATAPREP_INGEST_SERVICE_ENDPOINT}" \
Expand Down
Loading