Persist localStorage/Cookies/Session data

[ankur22]

Feature Description

When running tests with the browser module, a need quickly arises to be able to setup the initial state of the website under test (WUT) (e.g. login) and then to be able to reuse that state in subsequent test runs across single or multiple scenarios.

Currently the browser module doesn't support any APIs to perform such state persistence and load. In Playwright, there are several ways in which it is possible to save and load the state of various bits of data, which we want to incorporate into the k6 browser module.

In this issue we explore each of the methods that Playwright has implemented to identify how they are used when running a test locally and how they might be used when ran in the cloud. The end goal of this issue is to try and identify how we could build a generic system to persist and load that works for local and cloud test runs. This means we're not interested in the Playwright APIs themselves, but instead only on the mechanism of how those APIs might internally perform the persisting and loading of the data in a generic way, so the APIs that interact with the persistence/fs layer.

Local FS

Currently it should be possible to retrieve the required data by working with page.evaluate to retrieve the cookie/localStorage/session data, and then working with a generic js fs module to persist/load the data locally to disk. However this will not work when ran in the cloud since fs APIs only work in the init context.

What would be useful here is a key-value API that stores the data somewhere when ran locally and somewhere different when ran in the cloud. That difference should be abstracted away from the user so that they don't need to worry about where the data is being stored (e.g. local disk or a key-value database).

The examples below use a fs modules, but we would want to replace it with a new key-value storage API.

(Untested snippet, but in theory it should work). Persisting data:

    // Save cookies, local storage and session
    const cookies = await context.cookies();
    const localStorage = await page.evaluate(() => JSON.stringify(localStorage));
    const sessionStorage = await page.evaluate(() => { return JSON.stringify(sessionStorage); });

    fs.writeFileSync('cookies.json', JSON.stringify(cookies));
    fs.writeFileSync('localStorage.json', localStorage);
    fs.writeFileSync('sessionStorage.json', sessionStorage);

(Untested snippet, but in theory it should work). loading data:

    // Load cookies, local storage and session
    const cookies = JSON.parse(fs.readFileSync('cookies.json', 'utf-8'));
    const localStorage = fs.readFileSync('localStorage.json', 'utf-8');
    const sessionStorage = fs.readFileSync('sessionStorage.json', 'utf-8');

    await context.addCookies(cookies);

    await page.evaluate((storage) => {
        const data = JSON.parse(storage);
        for (const [key, value] of Object.entries(data)) {
            localStorage.setItem(key, value);
        }
    }, localStorage);

    await page.evaluate((storage) => {
        const data = JSON.parse(storage);
        for (const [key, value] of Object.entries(data)) {
            sessionStorage.setItem(key, value);
        }
    }, sessionStorage);

browserContext.storageState

With this API, which is available in playwright but yet to be implemented in the browser module, it can store the cookie and localStorage data to disk when a path is supplied, otherwise is is returned as an object for the user to read and do whatever they wish with it. It saves the snapshot of the current state of the browserContext when storageState is called.

Storing the current snapshot of the browserContext's state.

    await page.goto('https://example.com');
    await context.storageState({ path: 'state.json' }); // Where the data will be saved

Loading an existing snapshot and overwriting the current browserContext state.

    const context = await browser.newContext({ storageState: 'state.json' }); // State is restored

    const page = await context.newPage();
    await page.goto('https://example.com');

Since the call to store the data is implicit, it will need be performed from the (Go based) browser module and not from the test script, so we will need a GO SDK/http API to interact with the persistence layer.

Persistent browserContext

With this API, which is also available in playwright but also yet to be implemented in the browser module, it stores all the user data (everything that chrome persists) in a directory that is not deleted once the test ends. It's the same directory that a normal chrome instance would persist your personal browsing data. Usually when a test ends the user data directory is removed by the browser module itself.

The user data dir is a directory with many sub-directories and files, how can we store such a directory in a key-value database? When the iteration ends (successful/fail) should the browser module archive the directory before trying to upload to somewhere?

To create a new user data directory:

    // Create a persistent context with a user data directory
    const browser = await chromium.launchPersistentContext('./user-data-dir', {
        headless: false // Set to true for headless mode
    });

    const page = await browser.newPage();
    await page.goto('https://example.com'); // Cache will persist for subsequent navigations

To load an existing user data directory (same as above):

    // Create a persistent context with a user data directory
    const browser = await chromium.launchPersistentContext('./user-data-dir', {
        headless: false // Set to true for headless mode
    });

    const page = await browser.newPage();
    await page.goto('https://example.com'); // Cache will reuse and persist for subsequent navigations

NOTE: What makes this challenging is that the user data directory needs to be located in the same instance that is running chrome. At the moment chrome runs in a separate instance from where k6 is ran. How do we retrieve the user data dir? This is worth noting, however, it's not a problem that this issue needs to solve.

Due to the outlined challenges with the use case, it might be better to focus on the first two use cases and ignore this for now. In terms of functionality, i believe most users should be able to work with the first two methods of persisting and loading state.

Suggested Solution (optional)

No response

Already existing or connected issues / PRs (optional)

• Implement BrowserContext.storageState([options]) #4179
• Implement BrowserType.launchPersistentContext(userDataDir[, options]) #4180
• Filter out request metrics by their URL tag #4181