Browser crashes after 25 requests

[amanzrx4]

Which package is this bug report for? If unsure which one to select, leave blank

@crawlee/puppeteer (PuppeteerCrawler)

Issue description

I'm trying to scrap duckduckgo search results using puppeteer crawler, chromium browser and a user profile config. It suddenly just crashes after certain number of requests. I've set the maxConcurrency to 1

EDIT: It only crashes when i specify the user config for the chromium instance like this

launchContext: { launchOptions: { userDataDir: '/home/aman/.config/chromium', executablePath: '/usr/bin/chromium', args: ['--no-sandbox', '--disable-setuid-sandbox'], }, },

Code sample

No response

Package version

3.0.0

Node.js version

21.5.0

Operating system

linux

Apify platform

• Tick me if you encountered this issue on the Apify platform

I have tested this on the next release

No response

Other context

No response

[barjin]

This seems to be caused by the custom user profile configuration (see the same issue here).

It seems that it should be fixed in the latest versions of Puppeteer and Chromium. Can you share the Puppeteer / Chromium versions you're using (and try to update them to the latest?)

[amanzrx4]

hey @barjin thanks

i'm using the respective versions
chromium: Chromium 125.0.6422.141 built on Debian 12.5, running on Debian 12.5
puppeteer-core: 22.11.0

[codemem]

same error. crashed after 25 pages request

chrome: 126.0.6478.127 (Official Build) (arm64)
puppeteer: "22.12.1"

[barjin]

That's curious, could either of you post a reproduction code snippet?

Unfortunately, I haven't managed to reproduce this issue on my system yet (Linux on amd64).

[codemem]

Here is the pseudo snippets. Hope that helps

const crawlerOptions: Record<string, any> = {
      useSessionPool: true,
maxRequestsPerCrawl: 100,
      sessionPoolOptions: {
        sessionOptions: {},
      },
      launchContext: {
        launcher: puppeteerExtra,
        launchOptions: {
          headless: true,
          ignoreHTTPSErrors: true,
          executablePath: chromiumExecutionPath,
          args: ['--disable-web-security', '--no-sandbox', '--disable-setuid-sandbox'],
          ignoreDefaultArgs: ['--disable-extensions'],
          userDataDir: userDataDir // -> `./tmp/crawlee${uuid}`
        },
      },
      preNavigationHooks: {}

and then

const crawler = new PuppeteerCrawler(
      {
        ...crawlerOptions,
        async requestHandler
     ...

and finally

await crawler.run(
{
url: 'start_url',  // <<--- within this, there will be bunch of enqueueLinks call
userData: {
    customField: "value"
}
}
)