Skip to content

Commit

Permalink
Address https://twitter.com/gorhill/status/1583581072197312512
Browse files Browse the repository at this point in the history 
Unfortunately, the flawed conclusions people are reaching because of
this flawed tool are all over the place, including some spamming results
from this pointless tools in uBO's own thread on Wilders Security despite
advises to refrain from using the tool to evaluate content blockers.
  • Loading branch information
@gorhill
gorhill committed Aug 12, 2023
1 parent a3dad14 commit cf9f577
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions filters/filters.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16518,6 +16518,11 @@ techdracula.com##ins.adsbygoogle
/^https?:\/\/[-a-z]{6,}\.pro\/[a-d][-\.\/_A-Za-z][DHWXm][-\.\/_A-Za-z][59FVZ][-\.\/_A-Za-z][6swyz][-\.\/_A-Za-z][-\/_0-9a-zA-Z][-\.\/_A-Za-z][-\/_0-9a-zA-Z]{22,162}$/$script,xhr,3p,match-case
/^https?:\/\/[-a-z]{6,}\.xyz\/[a-d][-\.\/_A-Za-z][DHWXm][-\.\/_A-Za-z][59FVZ][-\.\/_A-Za-z][6swyz][-\.\/_A-Za-z][-\/_0-9a-zA-Z][-\.\/_A-Za-z][-\/_0-9a-zA-Z]{22,162}$/$script,xhr,3p,match-case

! Defuse pointless online tool too causing hardship to volunteers
! https://twitter.com/gorhill/status/1583581072197312512
*$3p,domain=d3ward.github.io
d3ward.github.io##.textads

! As of 2020-04-09, new filters will be added to year-based sublists

!#include filters-2020.txt
Expand Down

32 comments on commit cf9f577

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gorhill Should we add @@*$redirect-rule,domain=d3ward.github.io to that site too?

@Yuki2718
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AG once locked down as well and the author came over there to complain: AdguardTeam/AdguardFilters#140769

@MasterKia
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gorhill I think this does a good job of informing the users to consider the test results with caution, something which the site itself fails to do:

d3ward.github.io##main > div._bg-orange._txt-white.better_r::before:style(content: 'Annoncement from uBlock Origin: Be aware that the result of this test is not be an actual representation of the power of content blockers.' !important; font-weight: bold !important; text-decoration: underline !important;)

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Current filters are fine, the goal is to get 100% to stop the misinformation spreading out there. If those filters break the site, it's still far less problematic from uBO's perspective than the site spreading flawed assessment of uBO. The filter AdGuard added acts strictly on that one site to address the flawed results, and yet the author of the site apparently still think its tests are valid. He is essentially asking filter list maintainers to create filters specific for his tool, which make it clear the tool is flawed and pointless, yet he persists.

@MasterKia Now the site shows 100%, this is what people who don't know better (majority) want to see, they don't care about technical warning, there is already one and making it more verbose won't help.

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gorhill Recently I have added

*$xhr,3p,method=head,redirect-rule=nooptext:-1

to address generic anti-adb using fetch HEAD

uAssets/filters/filters-2023.txt

Line 3895 in 264318c

*$xhr,3p,method=head,redirect-rule=nooptext:-1

This makes the test can't reach 100%. Should we change the filter to site-specific, or exclude redirect on this site?

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes the test can't reach 100%.

I tried your filter and I still get 100%.

Ok I see, results are different in Firefox, it does appear @@*$redirect-rule,domain=d3ward.github.io is necessary.

@ghajini

This comment was marked as resolved.

Sign in to view

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add @@*$redirect-rule,domain=d3ward.github.io to that site too?

Added for the sake of Firefox.

@MasterKia
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Firefox xhr can't be redirected.

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Firefox xhr can't be redirected.

Oh really? It means our XHR redirecting filters does not work on firefox?

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Firefox xhr can't be redirected.

They are redirected using data: URI.

@MasterKia
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the third result in:
uBlockOrigin/uBlock-issues#2526 (comment)

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it depends on site's CORS policy? I don't see those errors / warnings on this site.

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reasons, Chromium can deal with redirecting to data: URI, but Firefox fails with the same.

Entering the following at the console:

fetch('https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js')
    .then(response => {
        if (response.ok && response.status === 200) {
            return response;
        };
    }).then(data => {
        console.log(data);
    }).catch(e => {
        console.log('error:', e);
    });

Shows that it goes redirecting to data:... URI through fine in Chromium, but not in Firefox.

You can see the result of redirecting to data: URI:

fetch('https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js')
    .then(response => {
        return response.text();
    }).then(data => {
        console.log(data);
    }).catch(e => {
        console.log('error:', e);
    });

In Chromium, I see the content of adsbygoogle surrogate.

All redirects for xhr always fallback on using data: URI as redirect URL, this is to avoid leaking extension URL, which on Firefox is unique and could be used to fingerprint.

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, if I redirect a xhr normally using an extension URL, I get the following result:

Response {
  type: 'basic',
  url: 'chrome-extension://neglonbndnocgncbfmmeodhbkbildmm…es/googlesyndication_adsbygoogle.js?secret=ukj71r',
  redirected: true,
  status: 200,
  ok: true,
  …
}
body: (...)
bodyUsed: false
headers: Headers {}
ok: true
redirected: true
status: 200
statusText: "OK"
type: "basic"
url: "chrome-extension://neglonbndnocgncbfmmeodhbkbildmml/web_accessible_resources/googlesyndication_adsbygoogle.js?secret=ukj71r"
[[Prototype]]: Response

Not ideal as this allows a site to easily detect that uBO specifically redirected the network request, hence why data: URIs are used in case of xhr.

@MasterKia
Copy link
Member

@MasterKia MasterKia commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I redirect a xhr normally using an extension URL

Does it work on Firefox too?

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does, but see what sites can see:

Response {
  body: ReadableStream { locked: false }
  ​bodyUsed: false
  ​headers: Headers { "content-type" → "application/x-javascript" }
  ​ok: true
  ​redirected: true
  ​status: 200
  ​statusText: "OK"
  ​type: "cors"
  ​url: "moz-extension://d8cdf3b0-3d73-421b-ae24-f190c353c52c/web_accessible_resources/googlesyndication_adsbygoogle.js?secret=llplub"
}

moz-extension://... is unique to every installation, so it's even worse for Firefox, hence even more justified to redirect to data: URI.

There are issues in somewhere bugzilla regarding leaking extension id and also redirecting to data: URI.

@MasterKia
Copy link
Member

@MasterKia MasterKia commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the issue here is the same as uBlockOrigin/uBlock-issues#235 or gorhill/uBlock#2823 but for redirects?

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a CSP thing, it's a Firefox-specific issue regarding redirecting to data: URI. I would have to find the bugzilla entry which explains it.

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incidentally, I see that on Chromium, AdGuard does not redirect to a data: URI, thus leaking the extension id as per cf9f577#commitcomment-124370102. The same fails in Firefox though, but in a different way than with uBO, the error message does not complain about data: URI.

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also incidentally, when using uBO Lite, the redirect leaks the unique extension id, so this is something that will have to be addressed by Firefox's DNR implementation.

Response {
  type: "cors",
  url: "moz-extension://c9d21eec-a0a0-4779-96da-e7822d7da1f4/web_accessible_resources/googlesyndication_adsbygoogle.js",
  redirected: true,
  status: 200,
  ok: true,
  statusText: "OK",
  headers: Headers(1),
  body: ReadableStream,
  bodyUsed: false
}

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but in a different way than with uBO

I got curious about this and it's because it's simply blocked with no redirection in Firefox. When I add the following filter in AdGuard:

||pagead2.googlesyndication.com/pagead/js/adsbygoogle.js$redirect=googlesyndication_adsbygoogle.js,important

I get leaked unique extension id as a result:

Response {
  body: ReadableStream { locked: false }
  ​bodyUsed: false
  ​headers: Headers { "content-type" → "application/x-javascript" }
  ​ok: true
  ​redirected: true
  ​status: 200
  ​statusText: "OK"
  ​type: "cors"
  ​url: "moz-extension://5965554d-1e28-47b1-ba04-70b0e2774001/web-accessible-resources/redirects/googlesyndication-adsbygoogle.js?secret=zJZC9pgMIC2eVbQSSEcw3"
}

So it looks like AdGuard is disregarding leaking the unique extension id.

@MasterKia
Copy link
Member

@MasterKia MasterKia commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for Firefox the options are:

  1. Redirect using a data: URI: might sometimes fail, but does not leak the extension unique id

  2. Redirect without using a data: URI: always works, but leaks the extension unique id

If the extension id was not unique, it wouldn't have been that bad to leak[*]. But it is unique for every user...

* The site can already know the request is redirected, probably by a content blocker.

@gorhill
Copy link
Member Author

@gorhill gorhill commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related bugzilla issue regarding redirecting to data: URI: https://bugzilla.mozilla.org/show_bug.cgi?id=1645683

And the leak issue via fetch(): https://bugzilla.mozilla.org/show_bug.cgi?id=1405971

@stephenhawk8054
Copy link
Member

@stephenhawk8054 stephenhawk8054 commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually even if Firefox ID is not unique (like chromium), it's not a good idea to leak the ID this easy. Because currently, it's already quite complicated for the websites on chromium to detect if uBO is installed on users' machines thanks to uBO's secret token for web accessible resources. If we ignore the ID-leak issue, the whole point of the token implementation is defeated.

@MasterKia
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's already quite complicated for the websites on chromium to detect if uBO is installed on users' machines

I think this PoC isn't that complicated:
uBlockOrigin/uBlock-issues#1572

@stephenhawk8054
Copy link
Member

@stephenhawk8054 stephenhawk8054 commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not always work. I tested that website for a few times, sometimes it works, but sometimes it does not: https://github.com/uBlockOrigin/uAssets/assets/66517106/a0fb55a7-a791-4612-8ef3-ddc5bbb85a84

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But anyway, so does the error throw at every site when redirecting, or does it work on some sites, and not on some others?

@stephenhawk8054
Copy link
Member

@stephenhawk8054 stephenhawk8054 commented on cf9f577 Aug 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm... When I add no-cors to mode, I don't see the error

fetch('https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js', {mode: 'no-cors'})
    .then(response => {
        if (response.ok && response.status === 200) {
            return response;
        };
    }).then(data => {
        console.log(data);
    }).catch(e => {
        console.log('error:', e);
    });

?

@gorhill
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get an empty string though as a result, not the content of the resource, i.e. with *$3p,xhr,redirect=googlesyndication_adsbygoogle.js I get an empty string in Firefox when using {mode:'no-cors'} even though the fetch() no longer throws.

@stephenhawk8054
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for the site like this: https://ozulmanga.com/ with this kind of fetch

function checkAdBlocker2() {
  try {
    fetch(
      new Request("https://static.jubnaadserve.com/api/widget.js", {
        method: 'HEAD',
        mode: 'no-cors'
      })).catch(error => {
        showNotification()
      });
  } catch (e) {
    // Request failed, likely due to ad blocker
    showNotification()
  }
}

redirect-rule still solves anti-adb on Firefox, don't know why.

@JobcenterTycoon
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The leak issue also got reported to Adguard already AdguardTeam/AdguardBrowserExtension#2278

Please sign in to comment.