Delete after insert #174
Draft
Delete after insert #174
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ePaul
added
spring-boot-2
ePaul
force-pushed
the
delete-after-insert
branch
from
0ec1b31 to
a7e30b3
Compare
ePaul
force-pushed
the
delete-after-insert
branch
2 times, most recently
from
7b6d8fc to
87e711c
Compare
ePaul
added
auto-configuration
ePaul
commented
Aug 10, 2023
Comment on lines
+68
to
+82
| /** | ||
| * Persists and immediately deletes some event log entries. | ||
| * This is meant to be used together with infrastructure listening to a logical DB replication stream. | ||
| * <p><b>Implementer's note:</b> The default implementation just calls {@link #persist(Collection)} | ||
| * and {@link #delete(Collection)} and will only work if {@code persist} stores back the IDs into the objects | ||
| * (which is not part of the contract). If that's not the case, this methods needs to be overridden to | ||
| * implement it in a different way. | ||
| * </p> | ||
| * (The implementation in Nakadi-Producer Spring Boot starter reimplements this method.) | ||
| * @param eventLogs the event log entries to be persisted and deleted. | ||
| */ | ||
| default void persistAndDelete(Collection<EventLog> eventLogs) { | ||
| persist(eventLogs); | ||
| delete(eventLogs); | ||
| }; |
There was a problem hiding this comment.
I've added this default implementation so we can stay compatible with applications which don't use the spring boot starter, but implement this manually. But it doesn't actually work, unless the persist method is implemented in a way that it fills the id field. So I wonder whether we should just skip the default implementation here?
Alternatively, we could have it throw an exception.
This is only relevant in the case that someone doesn't use the spring-boot-starter, but uses their own repository implementation. I know of exactly one app which did this (because it didn't use spring-boot), and that was decommissioned two years ago (and only ever got to version 4.3.0). Not sure we even need to worry about this.
ePaul
force-pushed
the
delete-after-insert
branch
2 times, most recently
from
26fe4a3 to
41affd2
Compare
- auto configuration - test for non-existence of certain beans - documentation
We create a new method for this instead of changing the existing one, as this is significantly slower this way, so it should only be used when needed.
ePaul
force-pushed
the
delete-after-insert
branch
from
b057e3e to
f750c20
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(This includes the changes from #171, #173, #176, #175 and builds on top. These should likely be merged first, then this one gets smaller. Implements parts of #170.)
This introduces an optional "delete-after-insert" mode. In this mode, events stored into the database will be immediately deleted.
The purpose of this is to connect the eventlog table to Fabric Event Stream's Debezium-connector (this is a Zalando-internal project), which will read the events from the logical replication log and takes care of sending them out to Nakadi.
Relevant changes:
EventLogRepositoryImpl.persistWithIds()methods will now update theidfield in the EventLog objects (implemented using the QueryStatementBatcher from ↑).EventLogRepository.persistAndDelete(), which, as the name implies, calls persist() and then delete(). In EventLogRepositoryImpl, it instead calls persistWithIds, so it actually works.nakadi-producer.delete-after-writefor the spring configuration which, when true, will make EventLogWriter usepersistAndDelete()instead ofpersist().TODO: