Skip to content

Commit

Permalink
Use create operation in order to not update a document with the same …
Browse files Browse the repository at this point in the history
…_id (same data) and resolve issues with concurrent operation
  • Loading branch information
nicolas-f committed Jun 25, 2024
1 parent 896178f commit feccbc0
Showing 1 changed file with 2 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@ def fetch_data(args):
json_dict["_id"] = base64.b64encode(
hashlib.sha256(line).digest()).decode(
sys.getdefaultencoding())
if "_op_type" not in json_dict:
json_dict["_op_type"] = "create"
yield json_dict
processed_documents += 1
except json.decoder.JSONDecodeError:
Expand Down

0 comments on commit feccbc0

Please sign in to comment.