Skip to content

Commit

Permalink
Updated script and documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
rimi-itk committed Nov 21, 2024
1 parent 87843f5 commit 4c054ea
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 21 deletions.
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
# OS2Loop AI stuff

Clone the [`os2loop-ai-stuff` branch](https://github.com/itk-dev/os2loop/tree/ai-stuff):

``` shell
git clone --branch ai-stuff https://github.com/itk-dev/os2loop os2loop-ai-stuff
```

[The scripts](#scripts) must be run in a running [OS2Loop setup](https://github.com/itk-dev/os2loop) by adding the
`os2loop-ai-stuff` directory (with the Git clone from above) as a volume to the `phpfpm` container, e.g.

``` shell
docker compose run --rm --volumes $PWD/os2loop-ai-stuff:/os2loop-ai-stuff phpfpm /os2loop-ai-stuff/data-export/export.sh
```

> [!NOTE]
> Change `$PWD/os2loop-ai-stuff` to match your actual setup.
## Scripts

``` shell
./os2loop-ai-stuff/data-export/export.sh
data-export/export.sh
```

## Development
Expand Down
31 changes: 11 additions & 20 deletions data-export/export.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,52 +13,42 @@ EOF
fi

>&2 cat <<EOF
Usage: ${BASH_SOURCE[0]} project-dir site-uri
Usage: ${BASH_SOURCE[0]} site-uri
EOF
exit 1
}

if (( $# < 2 )); then
if (( $# < 1 )); then
usage "Too few arguments"
fi

project_dir="$1"
uri="$2"
site_uri="$1"

if [ -z "$project_dir" ]; then
usage "Invalid project directory"
fi

if [ ! -d "$project_dir" ] ; then
(>&2 echo 'Project directory "'"$project_dir"'" does not exist')
exit 1
fi

if [ -z "$uri" ]; then
if [ -z "$site_uri" ]; then
usage "Invalid site-uri"
fi

cd "$project_dir"
# cd "$project_dir"

filenames=("$script_dir"/export_*.sql)

for filename in "${filenames[@]}"; do
echo "$filename"
echo "Running $filename"

# JSON

# https://tldp.org/LDP/abs/html/string-manipulation.html
output_filename=${filename/%.sql/.json}
# https://github.com/drush-ops/drush/issues/3071#issuecomment-347929777
vendor/bin/drush --uri="$uri" php:eval "return \Drupal::database()->query(file_get_contents('$filename'))->fetchAll()" --format=json >| "$output_filename" || true
echo "$output_filename"
vendor/bin/drush --uri="$site_uri" php:eval "return \Drupal::database()->query(file_get_contents('$filename'))->fetchAll()" --format=json >| "$output_filename" || true
ls -lh "$output_filename"

# CSV

output_filename=${filename/%.sql/.csv}
# https://stackoverflow.com/a/22421445/2502647
vendor/bin/drush --uri="$uri" sql:cli < "$filename" | awk 'BEGIN { FS="\t"; OFS="," } {
vendor/bin/drush --uri="$site_uri" sql:cli < "$filename" | awk 'BEGIN { FS="\t"; OFS="," } {
rebuilt=0
for(i=1; i<=NF; ++i) {
if ($i ~ /,/ && $i !~ /^".*"$/) {
Expand All @@ -70,5 +60,6 @@ for filename in "${filenames[@]}"; do
if (!rebuilt) { $1=$1 }
print
}' >| "$output_filename" || true
echo "$output_filename"
ls -lh "$output_filename"

done

0 comments on commit 4c054ea

Please sign in to comment.