Tor and Cloudflare #783
Comments
|
I don't see how this is related to the ungoogled chromium project but I personally appreciate your effort on this issue.
Oof! |
|
wow! seemingly they cannot understand that people just wanna use curl and wget with sites on cloudflare. Instead of clearly stating that that is impossible due to their anti-malicious-actor policy they produce all that verbiage. I have set up my cloudflare site to max. friendliness to tor users and it works in torbrowser-bundle, but curl always gets the challenge. Meaning I whitelisted TOR / T1 in the cloudflare-firewall as well as enbled the other onion-related setting: enable Onion Routing in the Crypto app. To achieve curl compatibility I would have to switch off cloudflare entirely or at least take out those subdomains which I want to have curl-friendly. Having said that, it already pains me to write here on an M$ site which also introduced pretty gross policies since the takeover. why ungoogle a package then publish via another Big Data actor? seems contradictory... |
|
Isn't the issue with curl and ungoogled-chromium in that they do not support .onion alt-svc headers described in the article mentioned by CF employee (https://blog.cloudflare.com/cloudflare-onion-service/) ? Tor Browser does support them, according to the article. If other tools don't, then that's probably why they are handled differently by CF. |
|
indeed cloudflare improved things for users of TBB but not for curl-users. I also used https://blog.cloudflare.com/privacy-pass-the-math/ privacy pass but havent figured out how I get a benefit from it. have solved tons of bleeding capchas but to no avail. |
the onion chain in TBB looks quite normal with an guard and exit node like always. |
That's probably due to this issue in TBB, which is also mentioned in that article: https://trac.torproject.org/projects/tor/ticket/27590 |
|
Isn't the issue with curl and ungoogled-chromium in that they do not support .onion alt-svc headers described in the article mentioned by CF employee (https://blog.cloudflare.com/cloudflare-onion-service/) ?
It seems chromium does support that header:
https://bugs.chromium.org/p/chromium/issues/detail_ezt?id=392575
But the issue has nothing to do with that. Opening clouldflare.com doesn't show any alt-svc response headers.
|
After changing circuits in TBB and refreshing a page a few times, it seems like they are available only for some exit nodes. |
|
After changing circuits in TBB and refreshing a page a few times, it seems like they are available only for some exit nodes.
I don't think this has anything to do with particular exit nodes at all. In my tests (in which I have changed circuits at least 50 times) TBB could always open cloudflare.com and ungoogled-chromium (or curl) could never do it.
I suspect it may be something in the crypto layer that allows TBB to open CF sites. I wonder where we can read about it (or ask for info).
|
|
they want to make a curl with support for Alt-Svc curl/curl#3498 e.g. try with i.xerx.xyz/222 (https too) curl --http2 http://example.com/ curl --http2-prior-knowledge http://i.xerx.xyz/222 If you by some chance already know that your server speaks HTTP/2 you can shortcut the HTTP/2 "negotiation" with --http2-prior-knowledge in TBB hit F12 , network tab ,on https://perfectoid.space/test.php alt-svc | h2="cflarejlah424meosswvaeqzb5…ion:443"; ma=86400; persist=1 so they delivered a clearnet website via tor thus gaining a crepload of anonymity... |
status quo is betterI prefer the status quo. It's useful for Ungoogled Chromium to show the CAPTCHA when entering a CF site because I do not want to be an enabler for those sites by giving them my traffic. The "Attention Required" banner serves as an instant flag to avoid the site. That's actually a service to the users. Circumventing the CAPTCHA would actually help Tor users unwittingly support a privacy adversary. make it a user-controlled optionIf someone makes U/C behave like TB w.r.t CloudFlare, then I would appreciate it if the functionality could be switchable. And if that functionality is enabled, I would hope to still see some kind of indicator that I'm on a site that's normally CF-CAPTCHA encumbered. Note that Firefox has a useful extension that automatically redirects CloudFlared sites to an archived version of the page. We a bit crippled on Chromium-based browsers where that extension does not exist. fingerprintingIt's also worth noting that Torproject is pushing hard to get everyone on TB because their fingerprinting countermeasure relies on many people using the same TB fingerprint. If U/C fixes #783, it will poach from TB users. At the same time, U/C does nothing to mitigate fingerprinting. So it will hinder Torproject's anti-fingerprinting strategy. |
|
## status quo is better
I prefer the status quo.
"is better" != "I prefer"
Others may not prefer it.
It's *useful* for Ungoogled Chromium to show the CAPTCHA when entering a CF site because I do not want to be an enabler for those sites by giving them my traffic.
Then the question is: do you want to give Google information about the sites you visit?
The instant "Attention Required" banner serves as an instant "avoid this site" flag.
But when you have JS disabled (as a privacy aware user would, especially through Tor) it is not "avoid" but really *block*. So it becomes an enforcement for reducing privacy. The "the status quo" (what TBB offers) does not enforce that.
Also it may be more important to the user to be able to access the site rather than avoid it. Through Tor it is not such a big privacy issue to visit CF enabled site.
Circumventing the CAPTCHA would actually help Tor users unwittingly support an adversary.
Not circumventing it would help Google correlate your visits across websites because Google receives the IP address of your visit through the captcha (without your consent). So if you visit multiple sites, even through Tor, the 3rd party HTTP requests to Google still help fingerprinting. So there are actually 3 issues here:
- Cloudflare (and its centralization)
- captcha (and its relation to Google)
- the combination of these two when using Tor
The first two are to be solved regardless of Tor as they are privacy (data correlation) issues with or without Tor.
## make it a user-controlled option
If someone makes U/C behave like TB w.r.t CloudFlare, then I would appreciate it if it could be switchable. And if that functionality is enabled, I would hope to still see some indicator that I'm on a site that's normally CF-CAPTCHA encumbered. Note that Firefox has a useful extension that automatically redirects CloudFlared sites to an archived version of the page. We a bit crippled on Chromium-based browsers where that extension does not exist.
Having an optional functionality is good. However having it through an extension is questionable because one has to trust yet another extension too. Having to base privacy on trust is not really the best thing.
## fingerprinting
It's also worth noting that Torproject is pushing hard to get everyone on TB because their fingerprinting countermeasure relies on many people using the same TB fingerprint. If U/C fixes #783, it will poach from TB users.
How do you "poach from users" and what exactly are you poaching?
At the same time, U/C does nothing to mitigate fingerprinting.
You are wrong. U/C adds specific flags about it. Additionally considering that the user-agent string of Chrome is statistically more popular than that of TBB, I think one may be better with U/C.
As a whole fingerprinting is all about statistics - you stand out less if you are closer to the most popular. So the question is - how do you know if you are close = how do you get actual statistics in order to minimize the fingerprint.
So it will hinder Torproject's anti-fingerprinting strategy.
How exactly?
And what is that strategy which cannot be implemented in U/C? Personally I think that using Tor with U/C could provide better privacy because unlike TBB which still keeps some of the background connections to third parties U/C comes out of the box with no such connections. Not having those connections at all is much better than having them through a Tor IP address.
|
The status quo is better w.r.t privacy, but worse w.r.t convenience. Users who want convenience choose Google Chromium, not U/C.
You've misunderstood. I'm not suggesting going beyond the CAPTCHA. Google does not know what CloudFlare site you're visiting if you do not execute Google's j/s. You see the CAPTCHA frame (for which the puzzle therein is withheld due to not running Google j/s), and you hit control-w. End of story. What you're proposing ruins that and compromises privacy. Users unwittingly feed traffic to CloudFlare.
It's the other way around. It's accessing the website that compromises privacy (either by running j/s and solving the CAPTCHA, or circumventing the CAPTCHA in order to feed traffic to CloudFlare and feed a privacy abuser).
The status quo is for TB to circumvent the CAPTCHA, and for U/C to display the CAPTCHA.
That's less secure and it harms everyone. When you feed a privacy abuser (a CloudFlare site in this case), it enables the privacy abuser to thrive and continue to damage everyone. If a user insists on feeding a privacy abuser, they can use Tor Browser. It's not good, and worse to expand on it by creating more tools to help privacy abusers.
Nonsense. Sharing all traffic including unhashed passwords is a huge privacy issue, most particularly when users don't even know it's going on. It happens without users knowledge or consent. U/C displays an SSL padlock. Most users are mislead by this, and believe they have a secure tunnel all the way to the web ownership (when really the tunnel terminates at CloudFlare). You're advocating to expose more users to that unwitting compromise in a blind way.
That's only if you execute Google j/s, which you should not do in the first place.
You poach users by giving them convenience. ATM, Tor Browser is the only way to bypass CF CAPTCHAs. So Tor users who traffic CF sites are limited to TB, but once U/C becomes an option for that use-case, some of them will switch. They will shrink the number of users emitting the shared fingerprint of TB thus increasing uniqueness.
No, you do not know how fingerprinting works. It's a collection of metadata and it goes well beyond user agent strings. I cannot give you all the knowledge you need in this post, but very briefly, you cannot simply "add a flag" and make fingerprinting go away. Every browser has a fingerprint and because it's a collection of metadata inherent in a browser's behavior, you cannot eliminate it. What you can do is investigate all the possible pieces of metadata that are usually taken for browser fingerprinting and hardcode the browser to project the same static metadata for all instances of that browser (privacy in numbers). Controlling the user-agent string doesn't even begin to control the problem. It's a huge effort that only TB developers have undertaken. Most importantly, U/C does not protect you from browser fingerprinting in the slightest. By moving users from TB to U/C, you are shrinking the number of users who share the single TB fingerprint, and you're moving them on to a platform that does nothing to control all the fingerprinting variables.
It could be implemented, but it's not. You should implement a static shared fingerprint first before implementing #783. |
Sounds like you know about this. I'm curious to know what the Tor devs have done; could you describe what changes they've done? For your reference, ungoogled-chromium currently has these three flags to directly mitigate fingerprinting (from docs/flags.md):
|
|
The status quo is *better* w.r.t *privacy*, but worse w.r.t *convenience*.
Using Tor proxy and U/C - how exactly does the captcha page improve your privacy? The IP address is torified but how does sharing it with Google (through the captcha) improve my privacy? Private means "not shared". So how does sharing more info with more parties give better privacy?
You've misunderstood. I'm not suggesting going beyond the CAPTCHA. Google does not know what CloudFlare site you're visiting if you do not execute Google's j/s. You see the CAPTCHA (for which the puzzle is withheld due to not running Google j/s), and you hit control-w. End of story.
Google's captcha is an iframe and even with JS completely disabled, loading the page which shows you "Attention Required" still sends an HTTP request to Google = Google knows your IP address, can set a cookie (if 3rd party cookies are enabled) etc. So neither the fact that it is "Ungoogled" chromium nor the fact that you may be using Tor protects you from that. What can protect you is uMatrix or uBO with blocking of 3rd party HTTP requests. (but perhaps that may be impossible too considering #662)
It's the other way around. It's accessing the website that compromises privacy (either by running j/s and solving the CAPTCHA, or circumventing the CAPTCHA in order to feed traffic to CloudFlare and feed a privacy abuser).
Accessing the site does not compromise privacy when you use Tor proxy. All that Cloudflare will know is the torified IP address and which site it contacted - that is the case with TBB.
When you are blocked from accessing it you have only 2 options:
- to disable Tor (and expose your real IP address)
- to enable JS and 3rd party requests to Google
Both reduce your privacy (and anonymity) much more than the case in which you would simply access the Cloudflare'd website through a torified IP address.
Nonsense. Sharing *all traffic* including unhashed passwords is a huge privacy issue, most particularly when users don't even know it's going on. It happens without users knowledge or consent. U/C displays an SSL padlock. Most users are mislead by this, and believe they have a secure tunnel all the way to the web ownership (when really the tunnel terminates at CloudFlare). You're advocating to expose more users to that unwitting compromise.
You are simply becoming aggressive without understanding what I am actually talking about. Just because you don't want to visit CF sites and want to have an option to be warned that a site uses CF before visiting it does *not* mean the workaround you use improves anything.
I am in no way claiming that CF is very good. And you are not "sharing all traffic" with them. They cannot possibly see your traffic to other (non-CF) sites. But if you have to pass the Google captcha and you also visit other sites using the same Tor circuit (as unlike TBB U/C has no way to request a new circuit) then you surely inform Google that someone is using the same circuit to visit sites A, B, C, D... which creates a much bigger possibility for data correlation and fingerprinting.
You poach users by giving them convenience. ATM, Tor Browser is the only way to bypass CF CAPTCHAs. So Tor users who traffic CF sites are limited to TB, but once U/C becomes an option for that use-case, some of them will switch. They will shrink the number of users emitting the shared fingerprint of TB thus increasing uniqueness.
This is a statistical fallacy. U/C has far less users than TBB. Those who use U/C + Tor are even less.
Even if we suppose people start switching from TBB to U/C+Tor there will be a positive result because with U/C their browsers won't make background connections to other hosts (what Firefox->TBB does).
Also why are you assuming that switching to U/C would be "increasing uniqueness"? Assuming a disabled JS and the fact that Chrome is a more popular browser (and thus statistically less fingerprintable) - how would switching to U/C "increase uniqueness"? Makes no sense. Instead of suggesting to keep U/C crippled, I would rather hope to have a more controllable fingerprint (protocol wise) along with making it possible to open CF sites through Tor. Then you would be able to appear identical to TBB, without all the Mozilla garbage.
No, you do not know how fingerprinting works. It's a collection of metadata and it goes well beyond user agent strings.
Did you even read the explanations I sent to CF and my tests? Or are you simply assuming that you know everything just because you like to keep a workaround which currently gives you a false sense of added privacy?
It *could* be implemented, but it's not. You should implement that first before implementing #783.
I am not against reducing the fingerprinting and I am not saying what should be first. This issue is about something else. You can open a separate one to discuss fingerprinting.
Here is also a reference (JS required):
https://bugs.chromium.org/p/chromium/issues/detail_ezt?id=807614
|
It discourages users from accessing a privacy-abusing website. CloudFlare gets less information about users under the status quo than with the convenience feature you're proposing.
This is a begging the question fallacy. The question itself relies on an bogus assumption.
Nonsense.
You've fallen for the "Tor will keep me safe" blanket misconception. Tor network coupled with various tools will prevent some disclosures of some information in some situations, but when you are on a CloudFlare reverse proxied site CloudFlare sees every single packet, including any unhashed passwords or HTTP PUT form data that you may transmit, and everything you click on, even when you have an HTTPS connection.
Actually you have omitted the most important option: to walk, which is actually the only privacy conscious option. A CAPTCHA barrier facilitates that choice especially when the CAPTCHA is broken. Suggesting removal of that barrier to send more users traffic to CloudFlare is obviously a lousy idea w.r.t. privacy. Perhaps your convenience feature would make more sense to implement on the upstream Chromium code or some other Chromium fork where users put convenience above privacy.
We're talking about user privacy, which is incidentally what I "want". I could just as well say this about what you "want" (convenience). The status quo is better for privacy. You're proposing hiding CloudFlare's very loud presence and at the same time making privacy-abusing websites more functionally convenient, which ultimately leads to more unwitting users feeding more information to a privacy abuser.
It's statistically correct to realize that TB (btw, stop calling it TBB that's an obsolete term) that TB uses a single fingerprint across all instances. Two users of stock TB have the same fingerprint, whereas two users of U/C have distinct fingerprints. Because you cannot guarantee two U/C users have the same fingerprint, you also cannot guarantee that hundred of thousands of U/C users have the same fingerprint. It makes no sense to look at population size of a platform that does not ensure fingerprint uniformity.
Uniqueness is a measure of the number of users sharing the same fingerprint. Obviously if fewer users share a common fingerprint, uniqueness of that common fingerprint increases. Uniqueness is indirectly proportional to user count.
Actually your cURL example proves my point. The list of headers you've attempted to mimic are insufficient. Your fingerprint differs from that of TB enough for CF to treat your traffic differently. FYI, plugins/extensions are enough to change a browser fingerprint. BTW, your second question again makes use of begging the question fallacy. |
|
@Eloston There are a lot of variables to fingerprinting. I've not worked close enough to anti-fingerprinting development efforts to have the whole map of it in my head. It is nice to see that U/C has taken some steps to mitigate fingerprinting. I would look through TB docs and code to review what all they've done. Note as well that TB may be the best for fingerprint masking but it may not be absolute either. Certainly plugins are known to disturb that and create a unique fingerprint. We also know that Tor users are getting deanonymized and we couldn't rule out fingerprinting of variables not controlled. |
|
> Using Tor proxy and U/C - how exactly does the captcha page improve your privacy?
It discourages users from accessing a privacy-abusing website. CloudFlare gets less information about users under the status quo than with the convenience feature you're proposing.
Similarly you could simply stop using GitHub because it is owned by another big tech - Microsoft. You could as well block the whole IP address ranges owned by big tech companies which are part of PRISM and not communicate with anyone who uses any services by these companies - just because you don't want your privacy to be abused (which would be pretty much equivalent to not using Internet at all). However this issue here is about making Cloudflare sites *work* with U/C over Tor, not about your personal preferences. So please kindly stop flooding it with contentious off-topic comments. If you would like U/C to have the functionality which you want (and which currently you *think* you have although you don't) as an additional option - please open a separate issue where it would be fine to discuss it. Personally I would welcome such *option*. But again - that is a separate issue.
> Google's captcha is an iframe and even with JS completely disabled, loading the page which shows you "Attention Required" still sends an HTTP request to Google = Google knows your IP address
Nonsense.
What I said is a fact which you cannot negate with arrogance. Here is a screenshot that proves it (clean U/C with JS and cookies disabled and no extensions):
https://www.picpasteplus.com/view.php?i=2p4pe45424l4ujp4zjtjjpti.jpg
It's statistically correct to realize that TB (btw, stop calling it TBB that's an obsolete term) that TB uses a single fingerprint across all instances. Two users of stock TB have the same fingerprint, whereas two users of U/C have distinct fingerprints. Because you cannot guarantee two U/C users have the same fingerprint, you also cannot guarantee that hundred of thousands of U/C users have the same fingerprint. It makes no sense to look at population size of a platform that does not ensure fingerprint uniformity.
This is also wrong but I won't explain why because I don't want to encourage you to post more off-topic comments.
As I said: Please open a separate issue if you want to discuss fingerprint reduction.
Actually your cURL example proves my point. The list of headers you've attempted to mimic are insufficient. Your fingerprint differs from that of TB enough for CF to treat your traffic differently. FYI, plugins/extensions are enough to change a browser fingerprint. BTW, your second question again makes use of *begging the question* fallacy.
It is not "your point". It is my question to CF which they didn't answer.
Considering the HTTP headers are the same and there is no JavaScript in curl, AFAIK the only thing left (which may introduce a difference) is the crypto layer (i.e. below HTTP but closer to TCP). You can call that "fingerprint" if you want as practically any HTTP client's dimension can be used for fingerprinting but again - this issue is not about fingerprinting per se.
I may still be wrong about the crypto though. Unfortunately I don't know where we could ask (Some Tor community? Suggestions are welcome).
|
Take your own advice. Github is off-topic here but I've opened #795 so you can discuss it in a more appropriate place.
It is not the duty of UC to facilitate CloudFlare Inc, work around CF bugs, or ultimately to expand on CF privacy abuses. You have filed your ticket in the wrong place. #783 should not have been filed as an UC issue, and you're the only one to pollute the thread with off-topic content. You're asking for a convenience feature that compromises the security of all U/C users. The request is inappropriate and to the extent that there is any discussion at all it's quite perverse of you to suggest that discussion of the privacy compromises entailed by your feature request should be silenced as "off-topic". How dare you.
Your screenshot only shows that you've failed to prevent google code from executing in your browser (evidenced by the status 200). My capture shows what it looks like when a U/C user properly mitigates google code from executing, and it proves that google contact is avoidable without implementing #783.
Do your own homework. It is not my responsibility to put in place preconditions to implementing #783. Why should I? I will however expose missing preconditions that would otherwise compromise everyone's security as a consequence of implementing your reckless and self-entitled feature request. If you don't have the motivation to do what it takes to support your own request, that's your problem.
Citation needed. Unless you can prove that CloudFlare does not use fingerprinting to accommodate TB sessions you can't rule it out. It's a very likely possibility. Otherwise you've misunderstood CF's motivation behind the CAPTCHA in the first place. If the client-side could be coded to circumvent the CAPTCHA it would defeat CF's anti-bot purpose, as any bot could simply be coded as needed. ATM the only known way to systematically circumvent CF's reCAPTCHA is using Selenium code (or the like) to control a TB instance. |
|
It is not the duty of UC to facilitate CloudFlare Inc, work around CF bugs, or ultimately to expand on CF privacy abuses.
Similarly you could assert that the whole web should be blocked without option for unblocking because big tech companies which abuse privacy also own large network backbones and undersea cables and surely track the IP traffic on it.
You have filed your ticket in the wrong place. #783 should not have been filed as an UC issue, and you're the only one to pollute the thread with off-topic content.
You are the only one who claims that. I leave it to @Eloston to decide.
You're asking for a convenience feature that compromises the security of all U/C users.
No. I am asking for an *option* to be able open CF sites through Tor with UC and I welcome that option coming with a visual warning and with additional measures to reduce fingerprinting (which itself is a whole separate issue which you didn't open for discussion). Ideally we should be able to have the same or smaller fingerprint than TB + a visual warning. This is my suggestion. Yours so far is: "do nothing and believe in non-facts".
The request is inappropriate and to the extent that there is any discussion at all it's quite perverse of you to suggest that discussion of the privacy compromises entailed by your feature request should be silenced as "off-topic". How dare you.
Again: I invited you to open a separate issue to discuss fingerprinting reduction because it is a big one and not specific to Cloudflare only. You didn't do that. Instead you accuse me of "silencing" you and increase the arrogance and aggression in your replies.
Your screenshot only shows that you've failed to prevent google code from executing in your browser (evidenced by the status 200).
No. It shows what I said earlier: that Google's recaptcha is an iframe and that a successful HTTP request to Google is made. There is no "code execution" because JS is disabled completely (also visible on screenshot).
Maybe read what HTTP 200 means:
https://tools.ietf.org/html/rfc2616#section-10.2.1
|
That's a fallacy by analogy because the backbone providers are not centralizing just one portion, then blocking access contingent on privacy compromise in exchange for access. Furthermore, even if they were it would still not be the duty of UC to implement a privacy compromise in exchange for convenient access anyway. The analogy is broken no matter how you look at it.
Whether it's an "option" or not, it's still a privacy compromise in exchange for convenience nonetheless.
Actually I was the first to state the need to make the feature user-controllable and mindful of the fingerprinting issues -- an idea which you now claim as your own in your goalpost-shifting attempt.
Again, do your own homework. It's your problem not mine that #783 is blocked by underdeveloped anti-fingerprinting mechanisms. Every submitter is limited by their own motivation. I've gone as far as to raise lack of anti-fingerprinting as a barrier to #783, given rationale, and I'm content with that level of effort ATM. Support your own initiative or watch it die.
This reply only shows that you don't understand what HTTP 200 means. Obviously if you're successfully connecting to google.com, you've screwed up. As I demonstrated already with my screenshot, if you configure to block google.com j/s no connection to google.com is made on a CF "attention required" landing page. My screenshot proves that (note the status: failed in every row for google.com connections). |
|
|
|
I'm curious to know what the Tor devs have done; could you describe what changes they've done?
Finally I found something:
https://lists.torproject.org/pipermail/tor-access/
There are additional links inside the discussions. One of them is to a project by Cloudflare:
https://github.com/cloudflare/challenge-bypass-specification
which as per the top warning seems deprecated and recommends visiting:
https://privacypass.github.io/
which in its turn links to a paper named "Privacy Pass: Bypassing Internet Challenges Anonymously" describing the protocol:
https://www.petsymposium.org/2018/files/papers/issue3/popets-2018-0026.pdf
Just like I thought there is something done at level of cryptography. Perhaps you can understand what they are talking about.
Also the start of a related thread on tor-dev list:
https://lists.torproject.org/pipermail/tor-dev/2018-January/012776.html
|
|
@nchv That's a good find. I'll look into it more when I get a chance, but the latest post in the first link seems to suggest it doesn't actually help much... |
|
but the latest post in the first link seems to suggest it doesn't actually help much...
It is from March 2018 and I don't think it reflects the current state of things. Of course you can always test for yourself. Personally, as I also told Cloudflare, I don't see any recaptcha when using Tor browser.
|
|
Regarding curl... have a look at the TLS version and cipher suite that Tor Browser uses when connecting to any https site, then simply replicate this in curl:
Not a solution for UC, but does address the issue you had with the curl script. Works even without bothering to customise headers (apart from user agent). |
|
That curl command works fine! Thank you.
Regarding curl... have a look at the TLS version and cipher suite that Tor Browser uses when connecting to any https site
Where exactly do you look that up please?
|
|
Looking at: https://www.ssllabs.com/ssltest/viewMyClient.html it seems the culprit in this issue may be the TLS_GREASE_AA cipher suite which is at the top of "order of preference" for Chromium and cannot be disabled through --cipher-suite-blacklist command line option. I don't know what role this TLS_GREASE_AA plays (whether it is beneficial or not) but it doesn't appear at all in Tor Browser's cipher suite list. Perhaps making it possible to disable that cipher (e.g. through a flag) would result in using the other ciphers only which would make it possible to browse Cloudflare sites through Tor. @Eloston what do you say? |
|
I'm getting various results from So I looked up what this GREASE is:
Read more at: https://www.thesslstore.com/blog/google-wants-grease-chrome/
So this may not be related to Cloudflare's filtering, or maybe the error is on Cloudflare's end? Or they are deliberately using this to filter traffic (IMHO unlikely)? I think the TLS GREASE 'feature' in Chromium is unrelated but possibly also unwanted for Ungoogled Chromium as it was intended (presumably by Google) to
Not sure if Ungoogled Chromium should keep or remove the feature… BTW: Cloudflare clients can whitelist TOR traffic Also makes you wonder how privacy-friendly a joint proposal between Cloudflare and Google will turn out to be: The push to QUIC and HTTP/3 has come from two main companies - Google and Cloudflare |
|
This is still an issue. |
|
@emanruse Sorry for the late response. I think a flag is the right way to go. Feel free to submit a PR. |
|
I think a flag is the right way to go. Feel free to submit a PR.
I am not a programmer.
I was just wondering if it is in your (or other programmers') plans.
|
|
This is my successful patch to make chromium's http requests similar to tor-browser: --- chromium-96.0.4664.110/content/public/browser/frame_accept_header.cc
+++ chromium-96.0.4664.110/content/public/browser/frame_accept_header.cc
@@ -14,25 +14,7 @@ namespace content {
std::string FrameAcceptHeaderValue(bool allow_sxg_responses,
BrowserContext* browser_context) {
- std::string header_value = kFrameAcceptHeaderValue;
-
-#if BUILDFLAG(ENABLE_JXL_DECODER)
- // In case the buildflag and the runtime flag are enables, we insert
- // "image/jxl," into the header value at the correct place.
- if (base::FeatureList::IsEnabled(blink::features::kJXL)) {
- std::string header_value_prefix = header_value.substr(0, 54);
- std::string header_value_suffix = header_value.substr(54);
- header_value = header_value_prefix;
- header_value.append("image/jxl,");
- header_value.append(header_value_suffix);
- }
-#endif // BUILDFLAG(ENABLE_JXL_DECODER)
- if (allow_sxg_responses &&
- content::signed_exchange_utils::IsSignedExchangeHandlingEnabled(
- browser_context)) {
- header_value.append(kAcceptHeaderSignedExchangeSuffix);
- }
- return header_value;
+ return "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8";
}
} // namespace content
--- chromium-96.0.4664.110/net/socket/ssl_client_socket_impl.cc
+++ chromium-96.0.4664.110/net/socket/ssl_client_socket_impl.cc
@@ -302,7 +302,7 @@ class SSLClientSocketImpl::SSLContext {
SSL_CTX_sess_set_new_cb(ssl_ctx_.get(), NewSessionCallback);
SSL_CTX_set_timeout(ssl_ctx_.get(), 1 * 60 * 60 /* one hour */);
- SSL_CTX_set_grease_enabled(ssl_ctx_.get(), 1);
+ SSL_CTX_set_grease_enabled(ssl_ctx_.get(), 0);
// Deduplicate all certificates minted from the SSL_CTX in memory.
SSL_CTX_set0_buffer_pool(ssl_ctx_.get(), x509_util::GetBufferPool());With this patch, all is left is to start chromium with the command-line option |
|
Is this still an issue? |
|
@PF4Public Yes, cloudflare still gives Tor-browser beneficial treatement in this way. I have been successful with the previously posted patch, I haven't experienced any hickups with clouldflare when using tor since. This is a simple test using curl: $ curl https://1.1.1.1 -I -XGET
HTTP/2 403
date: Wed, 13 Apr 2022 22:28:50 GMT
content-type: text/html; charset=UTF-8
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=EvgZDDm26UXLY1Fzt062DVIfKVF7RA%2F43VfWA8ySQqzaMF7OCiLq0Aw4eFNk83uIdiyqF6UWbb6S1d2TaFyzFCYu1C%2FwjTXihZY5sFt7CGtOvQ4ofk%2ByKhg%3D"}],"group":"cf-nel","max_age":604800}
nel: {"report_to":"cf-nel","max_age":604800}
cf-chl-bypass: 1
permissions-policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
expires: Thu, 01 Jan 1970 00:00:01 GMT
x-frame-options: SAMEORIGIN
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
set-cookie: __cf_bm=ORxBDkpNOG_xWmyiIBsz5jdkhvuBBCxNLu60XjXcpD8-1649888930-0-AatASUT2kZiv3nvPkPDVrAc5Gy6rqLfMxxJ8Ove1pi13myVcsfCv/xUu8IKWByYOSu7bCUUv2PxymzsiJrTUopE=; path=/; expires=Wed, 13-Apr-22 22:58:50 GMT; domain=.every1dns.com; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 6fb7a318de7354a9-YYZ$ curl https://1.1.1.1 -I -XGET -A 'Mozilla/5.0 (Windows NT 10.0; rv:91.0) Gecko/20100101 Firefox/91.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --tlsv1.3 --tls13-ciphers TLS_AES_128_GCM_SHA256
HTTP/2 200
date: Wed, 13 Apr 2022 22:28:29 GMT
content-type: text/html
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=GzJOpBZX3dJB0R65clcg%2B4u%2B2QjGQBpH%2BokncgqDBXcTI3EtjmmtEeZTklG5IztVba6WTSZxJa1JXME%2FOOlru9T0RJY5Pulu9l7YfPTxrSE33ncxTheYrhc%3D"}],"group":"cf-nel","max_age":604800}
nel: {"report_to":"cf-nel","max_age":604800}
last-modified: Thu, 31 Mar 2022 18:35:55 GMT
x-rgw-object-type: Normal
x-amz-request-id: tx00000ebb0210041d04de3-0062574c6a-9ef0780-default
strict-transport-security: max-age=31536000
served-in-seconds: 0.003
cache-control: public, max-age=14400
cf-cache-status: HIT
age: 547
expires: Thu, 14 Apr 2022 02:28:29 GMT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
set-cookie: __cf_bm=DCDS6tkPUf8CFa.JOkFHAmOxfTkwk3P0gXG3beL7Dfs-1649888909-0-AfOv4s7Hgqf1iY9F/Vf7pEOYka8dPPDnXv7afsnef38C/TTia8Q4SaqbTTYHwLVq53+fiIKjgvNlG4ufCzAEAHU=; path=/; expires=Wed, 13-Apr-22 22:58:29 GMT; domain=.every1dns.com; HttpOnly; Secure; SameSite=None
server: cloudflare
cf-ray: 6fb7a2922ca03ff1-YYZ
alt-svc: h2="cflareusni3s7vwhq2f7gc4opsik7aa4t2ajedhzr42ez6uajaywh3qd.onion:443"; ma=86400; persist=1 |
|
Is it possible to rewrite this patch to make its effect be configurable by a flag? |
PF4Public
added
enhancement
help wanted
and removed
discussion
GREASE = Generate Random Extensions And Sustain Extensibility For more info see - ungoogled-software/ungoogled-chromium#783
|
I hope it is resolved by #1920. |
I would like to share my communication with Cloudflare's support in relation to problems with accessing sites using Cloudflare through Tor proxy.
There is definitely something which Tor Browser can do and which (ungoogled) Chromium cannot which results in the problems discussed below. I don't know what it is but I hope ungoogled-chromium will be able to provide a solution as CF's "protection" system right now is pretty much denying the possibility of using Tor + ungoogled-chromium for CF sites.
====================
From: **** <>
To: Cloudflare <>
Subject: Using Tor without Tor Browser
Hi,
I am facing the following problem:
I have tor systemd service installed and running and browser (Chromium) set to use the Tor Socks proxy. I also have separately Tor Browser Bundle (TBB).
Opening sites which use Cloudlflare often present the well known captcha challenge of Clouldflare. But that does not happen in TBB. All this has nothing to do with a particular Tor circuit - it happens always: all circuits work in TBB, no circuit works in any other HTTP client.
I even tried to mimic TBB by copying literally the HTTP request headers in a curl script (attached) but even that didn't work - the curl script also hits the captcha. You can test it with https://cloudflare.com as an argument.
Why is this happening and what can I do so that I can open Cloudflare'd sites with a curl script or with a browser different from TBB?
Please when answering consider that I don't want to enable JavaScript or to reveal my IP address to Google through recaptcha as it contradicts the essential reason for using Tor. Neither I am willing to install any extension to bypass this.
====================
****, Apr 7, 8:18 PM PDT
Hi ****,
Thanks for contacting Cloudflare support.
You may refer to the following article for existing possible solution to proceed with,
Does Cloudflare block Tor?
Cheers,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
Hi ****,
That article has nothing to do with my actual question. I am not asking how to tune my own website but how not to hit the captcha with an HTTP client different from TBB (as TBB doesn't hit it).
Could you please answer that?
====================
****, Apr 9, 8:11 PM PDT
Hi there,
In that case we will need more details on the exact steps for us to reproduce the issue faced, in case issue persist, please share with us:
Please respond with that information if you run into the issue again so we can continue to work with you to resolve your issue.
Best regards,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
Hi ****,
I am sharing the full STR, expected and actual behavior below. Please kindly pay equal attention to both (A) and (B).
STR:
I am using
tor-0.3.4.11-lp150.22.1.x86_64To generate HashedControlPassword use command:
Replace ████ with the output of the command in
/etc/torrcEXPECTED:
(A) tor-curl should be able to access the actual web page because it uses HTTP headers similar to those of Tor Browser (which can open the page)
(B) A security system using a challenge to protect the website should be GDPR compliant
ACTUAL:
(A) For every circuit without exception tor-curl hits the captcha and this happens for any website which uses this Cloudlflare feature (i.e. not only with https://cloudflare.com):
As you see: there is nothing malicious in using a script like tor-curl but Cloudflare is blocking ("challenging") it. Tor Browser does open the sites using Clouldflare though (and I don't know why).
(B) The current protection system of Cloudflare is non-GDPR compliant because:
It doesn't follow the principle of data minimisation:
Article 5(1)(c) https://gdpr-info.eu/art-5-gdpr/
"1. Personal data shall be:
(c) adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’);"
The purpose here is to protect a particular website from malicious activity. There is nothing relevant in sharing the fact of the visit with a third party (Google) or in otherwise reducing user's privacy (through enforcing JavaScript). So this doesn't minimize the data, on the contrary. To make clear: IP address is personal data, especially in combination with HTTP user agent. So every HTTP request should be processed in a GDPR compliant way.
The current challenge system contradicts also Article 6(1)(f) of the GDPR because the "legitimate interest" of the controller is overriding the fundamental right of the data subject, one of which is the right to data protection (defined in Article 8 of the Charter of fundamental rights of the EU). The current challenge system doesn't even give a possibility for consent to data processing beyond what is technically necessary. With JS enabled and without extensions blocking 3rd party requests (e.g. uMatrix) it just instantly shares personal data with Google. The web browser extension of Cloudflare does not resolve the issue because even the very process of installing an extension is additional data processing - totally unnecessary and an additional open door to various vulnerabilities.
A secure, privacy respecting and GDPR compliant challenge system would be simple, secure and privacy respecting. For example: A simple image or text/ASCII-art based captcha without 3rd party HTTP requests, cookies or JavaScript.
====================
****, Apr 11, 9:15 PM PDT
Hi ****,
You mentioned
For every circuit without exception tor-curl hits the captcha, can you share with us the rayID you have observed in plain text as well as the zone name where you were running your tests to, so we can help look further at your firewall event logs from there too.Best regards,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
Here are a few Ray IDs. For full command output including IP addresses check the attached file.
4c645a9a3c9358df
4c645ae56d639de5
4c645b317e739cbf
4c645b7d1cfa5ae5
4c645bc30dd3841e
4c645c0d6aa7c67d
4c645c56ad409499
4c645ccb3f11c859
4c645d107e20bf89
4c645d74b90dd113
I have given you full STR. I don't know what "zone name" means. Please clarify.
This is not about anything mine (firewall/sites) but a general issue with all sites which use Cloudflare's challenge mechanism.
====================
****, Apr 14, 8:32 PM PDT
Hi ****,
Zone name is referring to the domain you are running the test to, I understand that you believe this is a wider issue, but we can only discuss with you if you are running test to
your own zone(s)and encountering issue, as we can only discuss zone settings with owners.I would suggest to share with us the rayID that is associated with a zone you own, from there, we can verify the issue and run tests based on your own setting and move on from there
Best regards,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
On Mon, 15 Apr 2019 03:32:31 +0000 Cloudflare wrote:
I don't believe - I am showing facts. With the full STR you can reproduce the issue for yourself, with your own zone or with any other zone which has this "challenge" for Tor.
I don't see why you should ignore a wider issue yet suggest assistance for a limited manifestation of it.
====================
****, Apr 15, 7:59 PM PDT
Hi ****,
All rayID is associated with a specific zone, please let us know which zone you are testing with, that will help us with our investigation with the teams.
On the other hand, if these rayID are from
your own zone, it can allow us to communicate with you down the road as our goal is to ensure the security and privacy of all our customers, including yourself. We will only be able to discuss in details with the zone owner, and having verify that will definitely allow us to communicate more freely upon checking with our teams.Best regards,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
On Tue, 16 Apr 2019 02:59:06 +0000 Cloudflare wrote:
I have given you complete STR and the result of it. You keep asking for
additional info - there is no such info.
There is neither security, nor privacy in a captcha which shares user's IP address (and other details) with Google. I have explained that.
The shared STR show that the "zone owner" is you - cloudflare.com. So you have all details to investigate and fix the issue which applies to every site using this captcha system.
====================
****, Apr 16, 5:32 PM PDT
Hi ****,
I think we're just going in circle here and I should have better explain why we would need to focus on your own zone and check from there.
The key is, Cloudflare employees can not remove the captcha. The captcha is showing specifically because of security settings the site owner has turned on, due to whatever setting they might have on their zone, may that be
cloudflare.comor any other zones behind Cloudflare.And as shared, we will not be able to discuss what is the current setting of any zone that is not yours.
To share with you the generic information, a site visitor to a website that Cloudflare is protecting can see a captcha for the following reasons:
The IP address you are on has shown problematic activity online recently in one of our data sources. If you would like to look your IP up, then please look your IP up at Project Honeypot. If the IP address shows data for malicious activity, you can see why there. You can also attempt to whitelist your IP directly on that page by connecting from that IP.
The site owner decided to block the country you are visiting from.
Your actions are triggering a Web Application Firewall (WAF) rule that the website owner has turned on.
What to do:
You can still enter the site by passing the captcha. You can also request to the site owner that the whitelist your IP address. You must have cookies and JavaScript enabled in your browser settings to pass the captcha. If you are running any anti-virus or firewall programs on your computer, then you may also need to check to make sure that they aren't blocking access to the captcha image.
If your IP is showing that you are distributing malware or spam, your computer may have an infection and we would recommend running an anti-virus scan.
Having said that, we will not be able to change the setting of any zone, nor the infrastructure of Cloudflare WAF rule for any specific visitor.
However, if this is
your own zone, and you believe you have already got settings that should allow TBB (or any other TOR user) to bypass any challenge but your visitors are still observing difficulties, then, we can make use of the test rayID fromyour zone, and look further into your firewall setting.Useful resource
Does Cloudflare block Tor?
Why am I getting security captcha or challenge for normal administration posts/submits?
Best regards,
******** | Cloudflare Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
On Wed, 17 Apr 2019 00:32:54 +0000 Cloudflare wrote:
I agree.
I am afraid you are missing the actual big issue here and trying to turn it into a limited (zone) issue, then from that work on fixing the limited issue. Please understand:
This has nothing to do with having a particular country restriction, or specific IP address being marked by Cloudflare (or by a zone owner) as problematic. These factors are generic and common sense. What I am discussing as an issue is an overall pattern which has nothing to do with any of these factors.
To prove that I have recorded for you a screencast showing the actual issue with the same Tor exit node and the same IP address but with different browsers:
[Screencast]
As you can see - TBB (on the left) opens cloudflare.com, Chromium (on the right) doesn't. There is the Ray ID too.
So:
████ Issue 1 (Tor and browsers)
TBB opens all sites, other browsers face captcha. And that is regardless of the particular IP address of the Tor exit node. Also as demonstrated in the STR earlier: that does not depend on the HTTP request headers.
████ Issue 2 (Captcha implementation)
But they can make a more privacy respecting captcha which does not:
I hope that clarifies actual primary and secondary issues.
====================
******, Apr 19, 1:24 AM PDT
Hello there
Just for clarification purpose, are you trying to find a way to use the command line to query via TOR on Cloudflare website without solving the initial Captcha?
Best regards,
Mark
Cloudflare | Senior Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
On Fri, 19 Apr 2019 08:24:11 +0000 Cloudflare wrote:
I am willing to be able to access through Tor all websites which use Cloudflare with any modern browser without being forced to reduce the privacy of a legitimate visit through:
The curl example was aimed to demonstrate that the captcha doesn't depend on the HTTP request headers.
====================
***** M, Apr 20, 2:56 PM PDT
Hello ****,
Looking back over this issue it appears that Cloudflare's recently released "Onion Service"was meant to address some of your concerns. That is why you don't have issues with the Tor Browser.
https://blog.cloudflare.com/cloudflare-onion-service/
I don't believe that it's been adopted universally, but their are other options we just don't have insight into the status of all of them. And again as **** mentioned this is not something that we control, individual site owners can opt-out of this and specifically block Tor users if they want to.
Again we apologize for any inconvenience, please let us know if you have further questions.
Best regards,
*****- Cloudflare
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
Hi *****,
You guys asked for and received from me quite detailed information and repeated clarifications about the actual issues and the goal.
How exactly does any of the replies given so far provide a practical solution?
You keep repeating that the captcha is a feature which you don't control and it is up to the site owners to enable it or not. But the enabling of the captcha is not the issue here. The issue is the captcha itself - the way it is implemented, the fact that it reduces privacy and all the rest of it (as discussed). So that is something that only you (Cloudflare) are in control of. That is the actual issue here, no what your clients do or not.
====================
***** M, Apr 21, 12:03 PM PDT
Hello ****,
Thanks for clarifying. I do apologize for the privacy trade offs that are involved with use of Google's re-captcha service.
Google's recaptcha is an industry leader and an effective tool for preventing the automated or spam access of sites for many site owners. Unfortunately simple methods are no longer sufficient to inhibit maliscous requests.
For end users with a browser including Tor they can have a secure experience and mitigate user disturbance.
Unfortunately as you noted this does not include all browsers, and can be problematic when not using a browser. This may or may not be the site owner's intent. We can't speak to that, but imagine different cases would have different intent.
We don't strictly enforce the use of any captcha it's strictly up to the site owner's discretion. We now offer our Edge Worker, and apps platform and so users and/or site owners can implement their own solutions for cases where our's don't fit.
https://blog.cloudflare.com/introducing-apps-with-workers/
https://blog.cloudflare.com/tag/workers/
https://developers.cloudflare.com/workers/
Again we apologize for the friction this has caused. I hope that the sites can adjust their configurations and/or assist with adjusting your use-case.
We recommend that you reach out to the administrators for the specific services or sites that you run into issue with and share your experience.
Please let us know if you have further questions or concerns we can assist with.
Best regards,
*****- Cloudflare
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
Hi *****,
Google is an industry leader in making technically effective tools to break into everyone's privacy.
It is against the GDPR principles of data protection and minimisation to share data with other parties when it is possible not to do it. So although you may not enforce it, you don't take care to fix it either. It is practically a privacy back door - indeed a very effective one.
Additionally - in the process of enabling it there is no clear warning that informs the enabler that this tool is not GDPR compliant and about the consequences of if for the site visitor.
The final result: you are helping Google, not the web.
A simple method for the user doesn't necessarily mean a simple implementation. A complex image captcha without 3rd party requests and without JS is surely possible and can work fine. It is also possible to present an animated GIF as a more complex challenge or even an HTML video. None of that requires JS or "industry leaders".
So to answer the concern that JavaScript is a way to reduce user's privacy you suggest even more JS. Then what's the point of saying:
?
You speak of effectiveness, yet you suggest the worst possible "solution". I can assure you that nobody is going to contact a few million sites (or even just a few) because of this. I have seen people rage on forums and mailing lists against Clouldflare's recaptcha because of the concerns I am sharing with you. If you really care about the web and not just about corporate interest and industry leadership, you should approach this more seriously, rather than justify it.
If you don't - perhaps this whole communication is a waste of time. I already regret that I wasted a lot of mine just to read more that you are not going to do anything. I realize I am just one of thousands of non-paying users and perhaps of zero importance to you profit-wise. Yet the concerns I am raising are perfectly valid technically and I am quite disappointed that you are ignoring them.
====================
******, Apr 21, 10:12 PM PDT
Hello there
We appreciate the feedback. We take customer suggestions and input seriously and I have forwarded this to the appropriate team for consideration. Thank you again for sharing your ideas with us.
However, at this time, we do not have an immediate resolution for the concern you have brought up.
Best regards,
Cloudflare | Senior Support Engineer
====================
From: **** <>
To: Cloudflare <>
Subject: Re: [Cloudflare Support] **** - Using Tor without Tor Browser
You are welcome. I hope they will consider fixing it!
Thanks.
====================
***** M, Apr 22, 12:30 PM PDT
Hi ****,
One point of clarification, with Workers the JS runs on our servers not in the browser. It's used because it's a common language for web developer. It doesn't require JS execution on the client.
Thanks again for all your useful feedback please don't hesitate to reach out if you have further questions.
Best regards,
*****- Cloudflare
====================
The text was updated successfully, but these errors were encountered: