-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate to Prosopo Procaptcha #372
base: main
Are you sure you want to change the base?
Conversation
* working frontend temp * temp * package json fix * npm version bump * Drip request type changing to procaptcha * Drip request type changing to procaptcha * swapping all occurrences of recaptcha with procaptcha and adding captcha logic * Renaming recaptcha files and updating imports * adding Prosopo outputs to tests * bugtesting * Removing bundle and adding back regular tests * changing frontend to use remote bundle * Add CAPTCHA_PROVIDER config item * add feature flag for captcha provider and update tests for procaptcha * Add react deps * Add test configs for both captcha providers * lint:fix and yarn install * add Paseo testnet config * remove wococo testnet config * Adding forgotten paseo deployment * Address PR comments * Fix dummy body * Add debug * Bump the npm_and_yarn group across 1 directories with 1 update (paritytech#368) * Create e2e test environment for procaptcha * remove rogue file * Update zombienet config in workflow * yarn.lock * yarn.lock * downgrade jest * try fix for buildcheck * Pin polkadot versions * yarn.lock with 1.22.21 * try new yarn.lock * Update faucet lockfile * Procaptcha dev (#2) * dev setup wip * update docker setup for procaptcha * Remove extra unecessary stuff * Undo formatting * Update mock provider URL to use prosopo subdomain * bump prosopo versions * add missing dep * Update to working package versions * Upgrade prosopo deps * Add missing body parameter --------- Co-authored-by: Hugh <[email protected]> Co-authored-by: hugh <[email protected]> Co-authored-by: Pierre Besson <[email protected]> Co-authored-by: Yuri Volkov <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
User @HughParry, please sign the CLA here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @forgetso! Thank you for the PR.
First of all, thank you a lot for adding tests! That's amazing!
More detail to follow...
Does this mean that the PR is still draft?
@mutantcornholio could you please look into the faucet (not client) code? I would appreciate a second set of eyes
|
||
- `PUBLIC_CAPTCHA_KEY`: The [reCaptcha v2 site key](https://www.google.com/u/0/recaptcha/admin). | ||
- `PUBLIC_CAPTCHA_PROVIDER`: The captcha provider. Currently `procaptcha` and `recaptcha` are supported. You will then need one of the following site keys: | ||
- `PUBLIC_PROSOPO_SITE_KEY`: The [Prosopo site key](https://prosopo.io/) which is `5HUBceb4Du6dvMA9BiwN5VzUrzUsX9Zp7z7nSR2cC1TCv5jg`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this key generated by you? If we want to change it we need to create an account and update it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is generated by us but you can generate your own by registering on our website. I think it would be best if we hop on a call to go through everything next week. I'll reach out to you on Element to arrange.
No, problem. It wouldn't be a proper integration without them although it was a mission to build them!
It's 99% of the way there, I plan to write a proper overview of everything that's changed and why because there are a lot of changes. The only reason I'm not saying 100% is because there is an occasionally failing test and I can't work out why it's happening. I'll try to get the summary done now so that you guys can review with additional context. |
I've added a summary of the changes @Bullrich, hope it helps 🙂 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for updating the body of the PR. Everything is very detailed.
I haven't find any problems with the code in the client
directory, but, I have a couple of questions:
- I tried to create an account in https://prosopo.io/#signup but I never received the email. Could you please look into this? I would like to be able to manage the key and store it as a secret.
- I'm unable to test the client in my computer.
- When loading the project (
yarn dev
) and clicking on the captcha, I'm getting the errorProviderUrlUsed
. I haven't modified any file, any idea how/what could be causing this?
- When loading the project (
Hey @Bullrich, I think the email might have been delayed. Have you now received it?
The contract that's on rococo is slightly out of date with the one in our sources. We plan to update this very soon. The error you should have received is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a whole bunch of questions, but there is an independent big one: I don't think we need two different captcha systems. Let's remove recaptcha completely. If Procapthca won't work for some reason, we can revert the changes, but I can't see a reason in supporting two.
Hey @mutantcornholio, I'd be happy to remove the ad tracking reCAPTCHA out of the faucet app. The option to have 2 captcha systems was implemented due to previous request in the issue. Maybe you guys could discuss and let me know which way you want to go. Cheers |
I see. I actually missed that. We had an internal discussion now and our opinions have diverged (and looks like I'm in minority). |
* lockfile update * Upgrade deps * Package-lock.json * Updating e2e tests readme (paritytech#375) * Fixing e2e readme link * Adding runtimeRestarter (paritytech#376) Fixes paritytech#223 + Adding prom-client dependency, because it was removed from @eng-automation/js (rightfully so) * Updating zombinenet verion in e2e docs (paritytech#378) @josepot has reported an error with `@zombienet/[email protected]`, while the latest version worked * Using PAPI for e2e tests (paritytech#379) * Currently, e2e types are generated on postinstall and prebuild steps, which requires metadata to be available in Docker image, even though it's not technically used in the application code. I think separating things would make things more complicated, but it's possible. * Couldn't undestand the reason for having two different COPY instructions in Dockerfile, but that doesn't live well with generating types in postinstall, so merging those two * .scale metadata files from Zombienet hosts saved in the codebase, in order to be able to compile the code separately from running Zombienet * Update dependencies * yarn.lock * Bump the npm_and_yarn group across 2 directories with 2 updates (paritytech#381) * removed base url (paritytech#382) This removes the base url, making the website work on the root of the domain. Also, by doing this change, and having set up the dns, this resolves paritytech#348 * Update deps * Update API calls * Silence polkadot API type errors * Update captcha type to image and set width of captcha container * remove grid class * switch back to PJS. change port number * remove prettier deps * update port * set procaptcha details * fix dropdown z-index * Fix z-indexes * Change mock provider version * remove console.errors --------- Co-authored-by: Yuri Volkov <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Javier Bullrich <[email protected]>
Hi @forgetso! How's the PR coming along? Has all the requested changes been addressed? I know it's troublesome, but could you also resolves the conflicts please? Once this is done we'll jump to give it another review 🙂 Thanks! |
Hey @Bullrich, apologies for the radio silence. I actually have a second PR in progress which will merge into our forked copy. This addresses all of the issues and merges all upstream changes. The most recent upstream changes have caused a bit of difficulty due to an upstream polkadot-sdk bug. However, I'm hoping to have an update to the PR for you to review this week. |
* Updating e2e tests readme (paritytech#375) * Fixing e2e readme link * Adding runtimeRestarter (paritytech#376) Fixes paritytech#223 + Adding prom-client dependency, because it was removed from @eng-automation/js (rightfully so) * Updating zombinenet verion in e2e docs (paritytech#378) @josepot has reported an error with `@zombienet/[email protected]`, while the latest version worked * Using PAPI for e2e tests (paritytech#379) * Currently, e2e types are generated on postinstall and prebuild steps, which requires metadata to be available in Docker image, even though it's not technically used in the application code. I think separating things would make things more complicated, but it's possible. * Couldn't undestand the reason for having two different COPY instructions in Dockerfile, but that doesn't live well with generating types in postinstall, so merging those two * .scale metadata files from Zombienet hosts saved in the codebase, in order to be able to compile the code separately from running Zombienet * Bump the npm_and_yarn group across 2 directories with 2 updates (paritytech#381) * removed base url (paritytech#382) This removes the base url, making the website work on the root of the domain. Also, by doing this change, and having set up the dns, this resolves paritytech#348 * Added markup code generation (paritytech#383) It generates a JSON object inside the header following the instructions from: https://developers.google.com/search/docs/appearance/structured-data/faqpage It parses the information of the markdown text to generate this json object. This resolves paritytech#277 * Updated actions to latest and added retention period (paritytech#384) We currently don't have an artifact retention period, so it defaults to 90 days. While the artifact for the site is not too big, having one build per push and PR can quickly accumulate the amount and make us hit our hard limit. By adding a limit to the retention day of 5 days we ensure that we get rid of the old artifacts faster than waiting for two whole months. I also updated all the actions version to latest release * Bump the npm_and_yarn group across 1 directory with 1 update (paritytech#385) * Bump the npm_and_yarn group across 1 directory with 1 update (paritytech#387) * Update Frequency Faucet URL (paritytech#393) As the Frequency Faucet now also supports Frequency Paseo testnet in addition to the Rococo one, the url is being set to a generic one instead of specific for the network. https://faucet.testnet.frequency.xyz/ * moved tag-client to work in pull_request_target (paritytech#394) It fails when an external contributors make a PR. This will fix that issue. * Merge latest upstream and use a transaction queue to send txs to nodes * Add MIT LICENSE (paritytech#395) * bump to polkadot 10.13.1 * lockfile * yarn.lock * yarn.lock * yarn.lock * yarn.lock * yarn.lock client * Fix test mock * Add @polkadot/util to dev dependencies * Address comments from @mutantcornholio --------- Co-authored-by: Yuri Volkov <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Javier Bullrich <[email protected]> Co-authored-by: Wil Wade <[email protected]> Co-authored-by: Maksym H <[email protected]>
* link to docs * Add missing setter
Hey @Bullrich, I think the PR should now pass if you run the tests. I'll keep checking for updates. Cheers |
@forgetso the unit tests run has failed with
and
The error in e2e tests looks papi-related, let's ignore it for now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a couple of notes, but overall it looks good from my side! Good job!
* Update paseo RPC wss://sys.dotters.network/paseo (paritytech#398) * Prettier write * Make properties public. Remove try catches. Lint * remove console.log * Increase test timeout * try installing playwright with yarn --------- Co-authored-by: Maksym H <[email protected]>
* Update paseo RPC wss://sys.dotters.network/paseo (paritytech#398) * Prettier write * Make properties public. Remove try catches. Lint * remove console.log * Increase test timeout * try installing playwright with yarn * Moved GitLab checks to GitHub's action (paritytech#402) Resolves paritytech#401 * Bump yaml from 2.4.1 to 2.4.2 in /client in the npm_and_yarn group across 1 directory (paritytech#403) * remove empty spaces * Address more PR comments * Address comments * Add type check function to test file * Fix import so that it works from tests dir2 * Update eng-automation * fix import * try after linting * Suppress ESLint to get CICD to run * yarn:fix * pass block number to mock provider * fix tests * yarn format:fix --------- Co-authored-by: Maksym H <[email protected]> Co-authored-by: Javier Bullrich <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks good from my side!
All my concerns have been addressed (with the exception of those two small lint ones)
@Bullrich any idea when this will actually merge into main? I'd like to ensure we are around to support when it's released. |
@mutantcornholio any blockers from your side? Or can we merge? |
@forgetso FYI the faucet tests are failing: https://github.com/paritytech/polkadot-testnet-faucet/actions/runs/8986761444/job/24684989457?pr=372 I re-run it 3 times and it is still failing 😢 |
Hey, sorry! I had it passing in my PR... strange. Will look into this today. |
@forgetso fyi there are some conflicts to resolve :( |
Fixes #337
PR to migrate rococo faucet from reCAPTCHA to Prosopo Procaptcha, which runs on the rococo test network.
Overview
When using Procaptcha, the flow of data in the faucet client app is as follows :
Frontend Flow
user_account
is generated using fingerprinting as random entropyuser_account
and the faucet'ssite_key
are passed to the Procaptcha Contract in rococo, and details of a random Captcha Provider are returned.site_key
,user_account
,block_number
, and other data related to the Captcha Provider.queryAt(block_number)
)Additional Info
Page Reloads
We noticed that the reCAPTCHA implementation would ask the user for a new captcha on every page load. We have recreated this functionality using procaptcha by clearing any local storage on load of the
CaptchaV2.svelte
component. This prevents a locally stored Captcha Provider from being contacted on page reload, and passing theuser_account
due to their historical records stored on the Captcha Provider.Max Verified Time
The maximum amount of time a captcha solution is valid for is 60s. This can be changed using the env variable
PROCAPTCHA_MAX_VERIFIED_TIME
, which takes a millisecond amount.Backend Flow
We'll begin the backend flow from where the frontend flow left off.
@prosopo/server
.a. Check if the provider was actually chosen at
block_number
b. If a
provider_url
is passed with the payload, the Captcha Provider is contacted to verify the user. Otherwise, read theuser_account
score from the contract directly after checking the time since the last correct captcha is less than themaxVerifiedTime
(60s
, mentioned previously)Testing
Client
Our client bundle continues to reach out to the rococo network for a random provider and the provider sends a real captcha. Solving this is clearly out of scope for UI tests so the provider is patched by injecting our mock provider (https://mockprovider.prosopo.io) into local storage. The mock provider always returns
{ "success": true }
for the Alice account and the testsite_key=5C4hrfjw9DjXZTzV3MwzrrAr9P1MJhSrvWGWqi1eSuyUpnhM
, which is the all zero address.E2E
As we are using a contract to return a provider, we implemented an additional contracts parachain in the E2E tests. This requires us to do the following:
site_key
) in the contractThen when a random provider is selected, it is always the mock provider. Again, passing a
user_account
of Alice's address and the zero address forsite_key
returns{ "success": true }
. Obviously, the downside here is the time taken for transactions in 12s parachains in zombienet. We have tried to use the substrate-contracts-node, which has instant seal, alongside the zombienet configuration but didn't succeed.We would appreciate any feedback on the approach so far!