Skip to content

Commit

Permalink
added example for zipping keys and values arrays
Browse files Browse the repository at this point in the history
  • Loading branch information
dzlab committed Sep 29, 2024
1 parent 8b54f58 commit 80a6153
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions _posts/2024-06-26-postgres-cdc.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ $ curl -X PUT $ES_URL
```shell
cat stream/regresstion-slot.jsonl | \
jq -c '. | select(.change | length > 0) | .' | \
jq '.change | flatten'
curl -X POST $ES_URL/_doc -H "Content-Type: application/json" --data-binary @-
```

Expand All @@ -261,6 +262,20 @@ Replace your_index with the name of your index and your_document_id with the ID
Handle authentication:
If your Elasticsearch instance requires authentication, add the appropriate credentials to the curl command.

```shell
sudo cat stream/regresstion-slot.jsonl | \
jq -c '. | select(.change | length > 0) | .' | \
jq -r '.change[] | @json' | \
jq -c '[.columnnames, .columnvalues] | transpose | map({(.[0]): .[1]}) | add'
```

does the following `jq -c '[.columnnames, .columnvalues] | transpose | map({(.[0]): .[1]}) | add'`
Here's a breakdown of how this jq command works:
[.a, .b] creates an array containing the values of keys "a" and "b".
transpose converts the array of arrays into an array of paired elements.
map({(.): .[1]}) transforms each pair into an object with the first element as the key and the second as the value.
add combines all the individual objects into a single object.

```shell
curl 'http://localhost:9200/customers/_search?pretty'
```
Expand All @@ -272,6 +287,12 @@ End the application:

```shell
$ docker-compose down

Stopping db ... done
Stopping es ... done
Removing db ... done
Removing es ... done
Removing network pg-wal2json_default
```

Using filebeat
Expand Down

0 comments on commit 80a6153

Please sign in to comment.