Design the REST-full API

[benedeki]

Background

The current REST API of the server is not complex nor very REST-full. And very rudimentary.

Feature

Design the REST API endpoints.

[benedeki]

Operations to (eventually support)

Partitioning

• Read partitioning context (partitioning) - this might include creation of the record in DB, in DB pattern might be used for the creation
• Read checkpoints
• Combined context and checkpoints data (partitioning) ?
• List
• Flows partitioning belongs to (partitioning)

Checkpoint

• Write (partitioning, checkpoint id)
• Open (partitioning, checkpoint id)
• Write partial (checkpoint id)
• Close (checkpoint id)

Additional data

• Write (partitioning)
• Get partitioning based on AD (additional data key, additional data value)
• Flows
• List
• Set relationship of partitionings - think through
• Partitionings of flow (flow id)
• Additional data of flow (flow id)
• Checkpoints of flow (flow id)
• Combined additional data & checkpoints of flow (flow id)?

Patterns
CRUD

[benedeki]

GET /flows
GET /flows/{flowId}/partitionings (just an alias of flow's partitionings field)

GET /flows/{flowId}/checkpoints?type=flat|structured (aggregate up all checkpoints from respective partitionning's CPs)
GET /flows/{flowId}/additionalData?type=flat-union-all|flat-union-newest|structured (dtto)

GET /partitionings
GET /partitionings/{partId} (field parent=partId)
PATCH /partitionings/{partId} to set parent=partId

GET /partitionings/{partId}/flows (just an alias of partitioning's flows field)

GET /partitionings/{partId}/checkpoints
GET /partitionings/{partId}/checkpoints/{checkpointId}

for batched data:
POST /partitionings/{partId}/checkpoints
-> {checkpointId communicated in the body, opened, written, closed}

for streamed data:
POST /partitionings/{partId}/checkpoints to open
POST /partitionings/{partId}/checkpoints/{cpid}/parts
PATCH /partitionings/{partId}/checkpoints/{cpid} to close (just field e.g."isClosed" being updated)
or
POST /partitionings/{partId}/checkpoints/{cpid} to a custom close-operation to be carried out (payload: {action: close, xxx:yyy})

GET /partitionings/{partId}/additionalData
PUT/PATCH /partitionings/{partId}/additionalData
POST /partitionings/{partId}/additionalData

GET /partitionings?searchByAdditionalData=k,v

GET /patterns
GET /patterns/{patternId}

---

PUT change all-at-once / PATCH partial update (field or multiple)

[benedeki]

It's suggested to have 3 controllers for now:

• flows controller
• partitioning controller
• patterns controller (in future, not imminent)

[benedeki]

For reference:
https://stackoverflow.com/questions/18470588/in-rest-is-post-or-put-best-suited-for-upsert-operation

[salamonpavel]

// new layer in Atum agent

• GET /partitionings?partitions=serializedPartitions (returns seq of Partitioning including ID) PAGINATION(limit,offset)

• GET /partitionings/{partId} (vraci konkretni Partitioning vcetne ID)

• POST /partitionings (body Partitioning) vraci 'Created' a v headeru 'location' s URI vytvorene entity

• GET /partitionings/{partId}/additional-data (vraci konkretni AdditionalData pro dany partitioning)
  POST /partitionings/{partId}/additional-data (body AdditionalData) vraci 'Created' a v headeru 'location' s URI vytvorene entity
  PUT /partitionings/{partId}/additional-data (body AdditionalData) vraci 'Ok'

• PATCH /partitionings/{partId}/additional-data (body AdditionalData) vraci 'Ok'

• GET /partitionings/{partId}/measures (vraci konkretni Measures pro dany partitioning)

• GET /partitionings/{partId}/checkpoints returns sequence of Checkpoints PAGINATION

• GET /partitionings/{partId}/checkpoints/{checkpId} vraci konkretni Checkpoint

• POST /partitionings/{partId}/checkpoints vraci 'Created' a v headeru 'location' s URI vytvorene entity

• GET /partitionings/{partId}/main-flow returns main flow

• GET /flows/{flowId}/checkpoints?name= returns sequence Checkpoints PAGINATION

• GET /flows/{flowId}/partitonings/ PAGINATION

• GET /flows/?partitions=serializedPartitions,onlyMain=true/false PAGINATION(limit,offset)

-- TODO these two:
PUT /partitionings/{partId}/parent (body parent partitioning) vraci 'Ok'

• PATCH /partitionings/{partId}/parents (body parent partitioning) vraci 'Ok'

Eventually (!!!!):

• GET /checkpoints?name=, and what not other filters
• GET /checkpoints/{chekcpointId}
• GET /partitionings/{partId}/parents ... returns all ancestors, not just direct ones
• GET /partitionings/{partId}/flows vraci sekvenci flows

// optional query params limit and offset
case class Pagination(
limit: int,
offset: Long
hasMore: Boolean
)