Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

234 refactor dia implementation #387

Merged
merged 41 commits into from
Oct 13, 2023
Merged

234 refactor dia implementation #387

merged 41 commits into from
Oct 13, 2023

Conversation

ebma
Copy link
Member

@ebma ebma commented Aug 23, 2023

This PR creates a re-usable definition for the DIA oracle mock, which is moving the implementation from pallets/oracle/src/mock.rs to a new module pallets/oracle/src/testing_utils and adding some other trait definitions that make it reusable.
The duplicated definitions are then removed from the other pallets and testchain.

Notes

  • The implementation contained in the oracle::testing_utils module is now 'no_std'. This is important to prevent compilation issues when re-using this module in the other crates.
  • The spin crate is used for no_std Mutex support
  • The once_cell crate is used for thread-safe and synchronized static variables. Using lazy_static is not sufficient because it does not prevent the multi-threaded tests initialize the static variable at the same time, leading to race conditions. We have to use a struct of once_cells 'race' module because only these are no_std, see the FAQ here.
  • The static LOCK contained in the oracle::testing_utils is supposed to be used in tests that need to access the oracle mock data. It effectively makes all tests (that use this lock) execute sequentially instead of in parallel, similar to the serial_test::serial macro. But I did not want us to put the #[serial] macro to every test because it's ugly and a little more error-prone IMO.

Closes #234.

@ebma ebma linked an issue Aug 23, 2023 that may be closed by this pull request
2 tasks
@ebma ebma marked this pull request as draft August 24, 2023 14:03
@ebma ebma unassigned b-yap and adelarja Aug 24, 2023
@ebma
Copy link
Member Author

ebma commented Aug 24, 2023

Turning this into a draft again because apparently there are some more severe conflicts I need to solve.
Also sorry for actually putting you guys as 'assignees' and not as 'reviewers' haha @b-yap @adelarja.

@ebma ebma self-assigned this Aug 24, 2023
@ebma ebma force-pushed the 234-refactor-dia-implementation branch from cf6a7bf to 47e4896 Compare August 31, 2023 16:48
@ebma ebma force-pushed the 234-refactor-dia-implementation branch from 70317ba to 13fd08b Compare September 1, 2023 17:43
@ebma
Copy link
Member Author

ebma commented Sep 5, 2023

I now figured out why the vault registry tests were suddenly failing. The reason is that I replaced the thread_local() implementation with a lazy_static() initialization. I didn't realize though that this introduces a problem because this lazy_static reference of the COINS is now shared across all tests, which then introduces race conditions because the exchange rates are cleared before each test run. Since the tests are running in parallel by default, I had to introduce the #[serial] macro to make sure that the clearing of before one test does not clear the values that were prepared for the execution of another test, etc.

I find this #[serial] annotation but I don't know of a better way to fix this right now. I had to make the code in the oracle_mock free of any std references so that it can be shared among all pallets without introducing compilation errors. Thus thread_local is not available.

Does anyone have some ideas how to approach this without the #[serial] macro, @b-yap @adelarja @gianfra-t ?

@ebma ebma force-pushed the 234-refactor-dia-implementation branch from e508da1 to f95ece5 Compare September 7, 2023 15:41
@ebma ebma marked this pull request as ready for review September 7, 2023 15:42
@ebma ebma requested a review from a team September 7, 2023 15:42
@ebma
Copy link
Member Author

ebma commented Sep 7, 2023

Thanks for your comment @gianfra-t. I decided not to use condvar but your suggestion led me to implement a locking mechanism that works similarly to the #[serial] macro.

Copy link
Contributor

@b-yap b-yap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside from the conflicts, I have no complaints.

}
}
<OracleKeys<T>>::put(oracle_keys.clone());
Ok(())
}

// public only for testing purposes
#[cfg(feature = "testing-utils")]
pub fn _clear_values() -> DispatchResult {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a rule about when to use underscore prefix function names?
Currently I had this impression of using such function/method only internally... and I mean only on the same lib/mod. However, the _clear_values() is liberally called in other pallets like issue and vault-registry.

examples:

  • the fn _offchain_worker() in pallet/vault-registry
  • the fn _withdraw_collateral() in pallet/nomination
  • the fn _request_issue() in pallet/issue

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. We did not discuss this yet and I just used the underscore prefix because it is used by the other 'testing-utils' functions. But you are right that it's used in other pallets as well.

Soooo... @pendulum-chain/devs what's your opinion on _function() names? I would use them for functions that are supposed to be used 'module-only' or helper functions. Thus, I would need to change the naming to clear_values() in this case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I now renamed all three functions because all of them are used in other pallets as well.

@ebma
Copy link
Member Author

ebma commented Oct 13, 2023

I ran the tests and clippy locally and it worked fine so I'll merge this before waiting for CI to finish.

@ebma ebma merged commit 91d6ed9 into main Oct 13, 2023
0 of 2 checks passed
@ebma ebma deleted the 234-refactor-dia-implementation branch October 13, 2023 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor DIA implementation
4 participants