Use bulk actions #28
Use bulk actions #28
Conversation
dustinblack
force-pushed
the
use-bulk-actions
branch
2 times, most recently
from
25d5dc5 to
70b5963
Compare
dustinblack
force-pushed
the
use-bulk-actions
branch
from
4f567b9 to
b9857e4
Compare
Automatic upate of README.md by arcaflow-docsgen arcabot fix http authentication
add tests for conversion functions linting pass through dicts in list conversion function
enable serialization tests schema tweaks implement generator for ingesting the bulk list correct error capture output cleanup update integration tests changing bulkuploadobject operation dict key temporarily to a string linting Automatic upate of README.md by arcaflow-docsgen arcabot only append metadata if it exists enable multi-arch build add plugin step to pre-process a list of data into a bulk_upload_list Automatic upate of README.md by arcaflow-docsgen arcabot add step definition to docker-compose action for build
dustinblack
force-pushed
the
use-bulk-actions
branch
from
b9857e4 to
703b9b0
Compare
| typing.Optional[typing.Dict[str, typing.Any]], | ||
| schema.name("metadata"), | ||
| schema.description( | ||
| "Optional global metadata object that will be added " "to every document." |
There was a problem hiding this comment.
Looks like the linter removed the code wrap.
There was a problem hiding this comment.
Should there be a way to name the metadata first-level key instead of just hardcoding "metadata"? For example, {"@metadata" {...}}? You could do this easily either by merging your value here directly into the document with doc.update(params.metadata) or with a separate metadata_key parameter which could even default to "metadata" and doc[params.metadata_key] = params.metadata.
There was a problem hiding this comment.
@dbutenhof I guess I'm not following. The schema.name is part of the self-documenting feature of the plugin. Often the name simply matches the key, but not always.
| typing.Optional[typing.Dict[str, typing.Any]], | ||
| schema.name("metadata"), | ||
| schema.description( | ||
| "Optional global metadata object that will be added " "to every document." |
There was a problem hiding this comment.
Should there be a way to name the metadata first-level key instead of just hardcoding "metadata"? For example, {"@metadata" {...}}? You could do this easily either by merging your value here directly into the document with doc.update(params.metadata) or with a separate metadata_key parameter which could even default to "metadata" and doc[params.metadata_key] = params.metadata.
| bulk_upload_list = [] | ||
| for item in params.data_list: | ||
| bulk_upload_list.append(BulkUploadObject(params.operation, item)) |
There was a problem hiding this comment.
Alternatively,
bulk_upload_list = [BulkUploadObject(params.operation, i) for i in params.data_list]There was a problem hiding this comment.
For me the longer way is a bit more readable. Is there an advantage here beyond tidiness?
There was a problem hiding this comment.
There are two:
- doing it all in one line (provided that the line does not wrap) means that more lines fit on a screen which makes it easier for the reader to comprehend the code (i.e., the reader should be able to move his/her eyes and see the whole functional unit without having to move the scroll bar); and,
- the "list comprehension" is scoped and bounded, such that none of the variables escape the expression and none of the operations have side-effects (unless they are a call to a function whose purpose is to have side-effects...), which means that the reader can safely consider the comprehension to be a "black box" if that suits his/her purposes.
| ids = [] | ||
| for i in resp["items"]: | ||
| ids.append(list(i.values())[0]["_id"]) |
There was a problem hiding this comment.
Alternately,
ids = [next(iter(i.values()))["_id"] for i in resp["items"]]
Changes introduced with this PR
Redesigning the main operational construct to use bulk actions instead of discrete indexing operations.
By contributing to this repository, I agree to the contribution guidelines.