Fingerprint Generator to initially generate only the user agent?

[Biswajit96]

Could you clarify if it's possible to use the Apify Fingerprint Generator to initially generate only the user agent, perform some tests, and then generate the full fingerprint for that user agent if needed?

The reason for this request is that the Apify Fingerprint Generator is consuming a lot of CPU. Generating just the user agent first would use less CPU, allowing me to test efficiently. If the tests pass, I would then generate the complete fingerprint and proceed.

[barjin]

Hello and thank you for your interest in this project.

In fact, the fingerprint-generator first generates a set of HTTP headers (including the user-agent string) using the header-generator package and only then goes on to generate other stuff (like screen resolution, GPU vendor etc.). This happens in the getFingerprint() method here:

fingerprint-suite/packages/fingerprint-generator/src/fingerprint-generator.ts

Lines 159 to 163 in f512962

	const headers = super.getHeaders(options, requestDependentHeaders, partialCSP?.userAgent);
	const userAgent = 'User-Agent' in headers ? headers['User-Agent'] : headers['user-agent'];
	// Generate fingerprint consistent with the generated user agent
	const fingerprint: Record<string, any> = this.fingerprintGeneratorNetwork.generateConsistentSampleWhenPossible({

Since this project is open-source, you can easily hack this (e.g. first generate your HTTP headers with the header-generator and then pass the user-agent as a new parameter for the fingeprint-generator - overriding the current generation on line 159).

What I'm more curious about - you're saying the fingerprint-generator package is using too much CPU. This is something we haven't experienced yet. Can you please provide some more details about your setup?

Cheers!

[Biswajit96]

i will try this thanks for your kind response.

[Biswajit96]

Hello,
Thanks for your response

Here is the example how i use

const { FingerprintGenerator } = require('fingerprint-generator');
const axios = require('axios');

const axiosInstance = axios.create({
     timeout: 3000, // Timeout after 3 seconds
     headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json',
          'Accept-Encoding': 'gzip',
          'Cache-Control': 'no-cache, no-store, must-revalidate',
          'Pragma': 'no-cache',
          'Expires': '0',
     },
});

async function getRandomFingerprint(devicesList = '', operatingSystemsList = '', browserListQuery = 'defaults') {
     const devices = devicesList && devicesList !== '' ? devicesList.split(',').map(item => item.toLowerCase().trim()) : undefined;
     const operatingSystems = operatingSystemsList && operatingSystemsList !== '' ? operatingSystemsList.split(',').map(item => item.toLowerCase().trim()) : undefined;

     const fingerprintOptions = {
          browserListQuery,
     };

     if (devices) {
          fingerprintOptions.devices = devices;
     }

     if (operatingSystems) {
          fingerprintOptions.operatingSystems = operatingSystems;
     }

     return new FingerprintGenerator().getFingerprint(fingerprintOptions);
}

function withTimeout(ms, promise) {
     return new Promise((resolve, reject) => {
          const timeoutId = setTimeout(() => {
               reject(new Error("timeout"));
          }, ms);
          promise.then(
               (res) => {
                    clearTimeout(timeoutId);
                    resolve(res);
               },
               (err) => {
                    clearTimeout(timeoutId);
                    reject(err);
               }
          );
     });
}
async function main() {
     try {
          //"browserListQuery": "",
          //"devices": "desktop, mobile",
          //"operatingSystems": "windows, macos, android, ios"
          const { api, devices, operatingSystems, browserListQuery } = Config;
          const source = axios.CancelToken.source(); // create a cancel token

          const fingerprint = await getRandomFingerprint(devices, operatingSystems, browserListQuery);
          const apiurl = ${api}&sid=${tokenid}&ua=${encodeURIComponent(fingerprint.fingerprint.navigator.userAgent)};

          const response = await withTimeout(5000, axiosInstance.get(apiurl, {
               cancelToken: source.token,
          }));

          console.log(response.data);
     } catch (error) {
          console.error('Error:', error);
     }
}

main();

Benchmark :
with FingerprintGenerator = approx. 20000 api call per minute.
with user agent from a text file (200k UA) = approx. 35000 to 45000 api call per minute.
2x Intel Xeon Processor E5-2699 v4 with 128gb ram.