Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

journal the world! #2441

Closed
7 of 10 tasks
raulk opened this issue Jul 16, 2020 · 1 comment
Closed
7 of 10 tasks

journal the world! #2441

raulk opened this issue Jul 16, 2020 · 1 comment
Assignees

Comments

@raulk
Copy link
Member

raulk commented Jul 16, 2020

As part of filecoin-project/oni#146, we need to make windowed PoSt and sector sealing write out journal entries.

Another component that would benefit from (selective) journalling is the mpool, so we can observe the effect of spamming attacks on the mpools of nodes. See discussion here: filecoin-project/oni#145 (comment).


This issue includes:

  • injecting the journal as a constructor upon Node construction, so we can instantiate different kinds of journals.
  • implement an observable, in-mem journal, which we'll use for testing.
    • if we migrate to zap as a backend, we might get this for free, although that implementation is not reactive :-(
    • we'll need to subscribe to journal entries in a streaming fashion, applying optional filter predicates.
    • we'll need to be able to clear the store (on every epoch, likely), so that we don't leak memory.
  • add a "event" field to journal entries, so we'll have "system" and "event" fields in the journal entry "header", and then a custom "payload/data" field.
  • consider typed events; right now journal entries are map[string]interface{}, which we could map through https://github.com/mitchellh/mapstructure when consuming. I wonder if the journal should take typed events, though, from the start. Not a priority right now, but food for thought.
  • make window PoSt write journal entries (atop Consume miner actor refactor #2413).
  • make the mpool write journal entries; this can be very noisy in production, so make sure that we can select the level of verbosity (local mpool pushes, mpool add, mpool drop).
  • make deals write journal entries.
  • make sealing write journal entries.
@raulk
Copy link
Member Author

raulk commented Jan 18, 2021

We can consider this done, since #2455. The part that's left is related to testing (in-memory journal), but will be addressed separately, if still needed.

@raulk raulk closed this as completed Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant