Algolia complains about the json

[Magneticdud]

hi, in february 2023 i started to use this theme and i setup algolia to be updated via algolia objects import [...] during the webhook and all was working ok.

But now i noticed in the error log:

Importing records ⣾missing objectID in object []interface {}{map[string]interface {}{"iconClass":"fa-user", "objectID":"[...CONTENTS OF THE WHOLE JSON OMITTED...]

Did i do something wrong? I added the theme as a submodule so it should be the latest version every single time (the webhook does a git submodule update --recursive --remote on the theme directory when a new commit is pushed)

Edit: and I opened the json and it doesn't look like wrong. It seems valid, and manually uploading it on the algolia website does not give errors

[igor-baiborodine]

@Magneticdud, could you please point me to the GitHub repository in question, specifically the webhook that uploads the indexes to Algolia?

[Magneticdud]

hi, because i was lazy i hardcoded the ftp passwords so it's on a private repo, so i copied the relevant part (the script that is run by the webhooks app on ubuntu)

https://pastebin.com/GepbKJDu

[igor-baiborodine]

To validate the JSON index file generated by the hugo command, could you please follow the steps for the Manual Upload of indexes to Algolia? If you are able to manually upload your indexes, the issue here might be with the Algolia CLI, which I'm not familiar with.

As an alternative, you may use the npm Algolia package provided by the theme, which is bundled with the example site: https://github.com/Lednerb/bilberry-hugo-theme/tree/master/exampleSite/algolia. I use it to automate index upload for my website, kiroule.com. For more details, read this article.

[Magneticdud]

Manual upload works both by following your guide (copy paste) or by uploading the file whole (it's a new feature?)

Maybe they changed something with the cli tool then

I didn't use the npm route because I have almost zero experience with node and I'm a bit scared

[igor-baiborodine]

You may try using this script in your webhook: https://github.com/igor-baiborodine/kiroule.com/blob/master/algolia/run-data-upload-js.sh. Basically, copy/paste the algolia folder from my site to your site and plug the above-mentioned scripts into your webhook.

[Magneticdud]

I solved the problem. A few months ago, they changed the CLI tool to accept only "new line delimited json" (pretty json) and it didn't understand normal minimized json. This isn't mentioned at all in the documentation, i accidentally noticed it from the output it was making when it exported existing objects

so i just need to add newlines and prettify the json before uploading:

cat index.json | jq -c '.[]' > index.ndjson
algolia objects import [NAME] --file index.ndjson --profile [PROFILENAME]
✓ Successfully imported 21 objects to [PROFILENAME] in 183.783508ms