Wrong browser from ua strings

[nvysyaraju-sift]

Hello,

While testing the parser with several ua strings, I found that for strings of this type

eg: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 bs=95 ts=0 ja3=67dfa73ac3b4f43834428b0274105be9 as=22616"

Initialized with all the Fields, the extracted browser value from Yauaa is bs=95, ts=0 instead of Chrome, for all the ua strings of this type, Yauaa seems to be extracting "bs=<>, ts=<>" inplace of the actual browser. Can you help me know how to get this resolved and get the proper browser name as there will be lot of string of this type.

Thank you.

[nielsbasjes]

Thanks for the example. I'll have a look.
Do you have examples with other 'noise' instead of the shown 'bs=' ,'ts=', 'ja3=' and 'as=' ?
Do you know what causes this?

[nielsbasjes]

Fixed in 7.12.2

https://try.yauaa.basjes.nl/?ua=Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%29+AppleWebKit%2F537.36+%28KHTML%2C+like+Gecko%29+Chrome%2F108.0.0.0+Safari%2F537.36+bs%3D95+ts%3D0+ja3%3D67dfa73ac3b4f43834428b0274105be9+as%3D22616

[nvysyaraju-sift]

Thank you for the quick response, I am not sure what causes these tags to be at the tail end of ua string.

Another thing is that, I tried parsing similar type of strings with 7.12.2, for a lot of them proper browser names are getting parsed but still there are some ua strings that return "bs=<>", "ts=<>" as browser name and version respectively.

For example,

browser value is getting parsed for string like these

Mozilla/5.0 (Linux; Android 13; SM-S906U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Mobile Safari/537.36 bs=97 ts=0 ja3=598872011444709307b861ae817a4b60 as=20001

Mozilla/5.0 (Linux; Android 12; SAMSUNG SM-A536U1) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/19.0 Chrome/102.0.5005.125 Mobile Safari/537.36 bs=96 ts=0 ja3=cd08e31494f9531f560d64c695473da9 as=21928

Lou's Pizza/3.16.0 iOS/16.2 CFNetwork/1402.0.8 Darwin/22.2.0 HQ:10214 bs=99 ts=0 ja3=b76d503360ae441d410a85a7f8d648ab as=22394

But browser is returning incorrect name for strings like these

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0 bs=98 ts=0 ja3=579ccef312d18482fc42e2b822ca2430 as=11878

Mozilla/5.0 (iPhone; CPU iPhone OS 16_0_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Mobile/15E148 Safari/604.1 bs=99 ts=0 ja3=773906b0efdefa24a7f2b8eb6985bf37 as=10796

[nielsbasjes]

I fixed those in the latest release (yesterday).
I found that the Lou's Pizza example was actually also wrong: It did not report that it was an app on an iOS system, now it does do that.

[nvysyaraju-sift]

Thank you so much for the fix, will try it with new version. And the live version (https://try.yauaa.basjes.nl/) is not yet using the latest version right?

[nielsbasjes]

Yes, forgot to load the new version ... fixed.

[nielsbasjes]

https://try.yauaa.basjes.nl/?ua=Lou%27s+Pizza%2F3.16.0+iOS%2F16.2+CFNetwork%2F1402.0.8+Darwin%2F22.2.0+HQ%3A10214+bs%3D99+ts%3D0+ja3%3Db76d503360ae441d410a85a7f8d648ab+as%3D22394

[nvysyaraju-sift]

@nielsbasjes Sry for repeated questions, thought this is interesting

I found that for invalid ua strings such as "this is an invalid ua string", all the standard fields are filled with "Hacker" value but when it comes to having a slash in the string, it starts filling Agent fields with values

Eg: "This is an invalid string/ this is an invalid string" returns AgentName (This is an invalid string)

When we have something like "Firefox 4.0" as who ua string it is returning parser is returning Firefox as both browser and os

When we have something like "Firefox 5/ Windows 10", it still returns Firefox as both browser and os

I know these are odd and not seen in general, is it just better to not worry about these or is there anyway to have these parsed as Invalid/Hacker?

Thank you for your quick responses for all the queries.

[nielsbasjes]

Yes, so the reality is that any string can appear in the user-agent field.
The goal of this library is to return the best possible result under the available specifications.
This starts with https://www.rfc-editor.org/rfc/rfc7231#section-5.5.3 and also contains the specifications and exceptions documented by the commonly used browsers and robots.

The effect is that for the 99.9+ % of the requests this analyzer will give you the correct result.
Because it is based on string pattern analysis it can even (without having seen it before) successfully extract something like "Lou's Pizza".

And yes there are many many strings that are total nonsense where it does not give the correct result; these are in the traffic of real websites < 0.1% of the traffic.

Because of the documented rules and conventions:

• "This is an invalid string" does not follow any of the patterns --> hacker.
• "This is an invalid string/ this is an invalid string" follows the basic pattern as documented in rfc7231 --> You get an agent and version.

[nvysyaraju-sift]

Thank you again for your response, I agree with the points you listed here, there will be these invalid and strings with noise in traffic but following the pattern and not missing normal or valid ua strings is our main focus.

[nvysyaraju-sift]

Hi @nielsbasjes

Thank you for resolving above issue with noise in UA strings, after using the updated version the detections improved well and the percentage of browser that still returns something like "bs=<>, ts=<>" came down. But still there are some non-negligible amount of strings that returns this.

Form my observation after you added updates to handle the above noise mostly strings with Safari browser are are getting extracted with this noise as browser value.

Strings like this just different Mac OS X versions and Safari versions.

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6.1 Safari/605.1.15 bs=99 ts=0 ja3=773906b0efdefa24a7f2b8eb6985bf37 as=29933

[nielsbasjes]

I have added some more rules to drop this noise.
Do you have any idea where these extra tags come from and where they can be expected also?

[nvysyaraju-sift]

Honestly not really sure on why we are seeing strings with these extra noise tags. I will try to get more info about this and get back here if I find anything understandable about this. Thank you