Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement page.on('response') #4234

Open
Tracked by #4232
ankur22 opened this issue Jan 11, 2024 · 1 comment
Open
Tracked by #4232

Implement page.on('response') #4234

ankur22 opened this issue Jan 11, 2024 · 1 comment

Comments

@ankur22
Copy link
Contributor

ankur22 commented Jan 11, 2024

Feature Description

When the browser module navigates to a website, the browser (Chrome etc.) will perform further requests to retrieve dependencies such as js files, images, css files and so on. With page.goto, the response that is returned to the caller is for the initial navigation to the the website. This response object doesn't contain any response details for the dependency requests that are made after the initial navigation.

A response of the dependencies is useful so that fine grained checks can be made:

  1. Ensure that certain important dependencies are retrieved;
  2. Ensure that certain dependencies aren't retrieved;
  3. Ensure that the status code match the expected value;
  4. Assert certain details within the payload body;
  5. Dynamically assert on several dependencies depending on what has already been retrieved.
  6. etc.

Thresholds can be used for this too, but having something inline with the code feels like it could be beneficial since programmatic actions can be taken when dependency responses are retrieved or aren't retrieved.

Suggested Solution (optional)

Already existing or connected issues / PRs (optional)

grafana/xk6-browser#1227

@inancgumus inancgumus mentioned this issue Jan 21, 2025
@andrewslotin andrewslotin changed the title Implement page.on('response') Implement page.on('response') Oct 11, 2024
@inancgumus inancgumus transferred this issue from grafana/xk6-browser Jan 21, 2025
@ankur22 ankur22 self-assigned this Jan 29, 2025
@ankur22
Copy link
Contributor Author

ankur22 commented Jan 29, 2025

I have started working on this and the implementation can be found here: #4296

I have tested the change with:

import { browser } from 'k6/browser'

export const options = {
  scenarios: {
    ui: {
      executor: 'shared-iterations',
      options: {
        browser: {
          type: 'chromium',
        },
      },
    },
  },
}

export default async function () {
  const page = await browser.newPage()

  const requestCounter = (function() {
    let count = 0;
    return () => ++count;
  })();

  page.on('response', async (response) => {
    const currentCount = requestCounter();

    const headers = response.headers();
    var json = null;
    if (headers["content-type"] === "application/json") {
      json = await response.json();
    }

    console.log(JSON.stringify({
      responseNumber: currentCount,
      allHeaders: await response.allHeaders(),
      body: await response.body() ? String.fromCharCode.apply(null, new Uint8Array(await response.body())) : null,
      frameUrl: response.frame().url(),
      acceptLanguageHeader: await response.headerValue('Accept-Language'),
      acceptLanguageHeaders: await response.headerValues('Accept-Language'),
      headers: response.headers(),
      headersArray: await response.headersArray(),
      json: json,
      ok: response.ok(),
      requestUrl: response.request().url(),
      securityDetails: await response.securityDetails(),
      serverAddr: await response.serverAddr(),
      size: await response.size(),
      status: response.status(),
      statusText: response.statusText(),
      url: response.url(),
      text: await response.text()
    }, null, 2));
  })

  await page.goto('https://quickpizza.grafana.com/', { waitUntil: 'networkidle' })

  await page.locator('//button[text()="Pizza, Please!"]').click();

  await page.waitForTimeout(1000);

  await page.close();
}

An example of one of the console logs:

INFO[0002] {
  "responseNumber": 46,
  "allHeaders": {
    "content-length": "754",
    "date": "Wed, 29 Jan 2025 18:58:37 GMT",
    "content-type": "application/json"
  },
  "body": "{\"pizza\":{\"ID\":531,\"name\":\"The Lucky Americana\",\"dough\":{\"ID\":2,\"name\":\"Thick\",\"caloriesPerSlice\":200},\"ingredients\":[{\"ID\":1,\"name\":\"Extra virgin olive oil\",\"caloriesPerSlice\":50,\"vegetarian\":true},{\"ID\":4,\"name\":\"Bianco di Nizza tomatoes\",\"caloriesPerSlice\":50,\"vegetarian\":true},{\"ID\":6,\"name\":\"Mozzarella\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":14,\"name\":\"Extra cheese\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":19,\"name\":\"Garlic\",\"caloriesPerSlice\":25,\"vegetarian\":true},{\"ID\":24,\"name\":\"Ricotta cheese\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":31,\"name\":\"Shrimp\",\"caloriesPerSlice\":100,\"vegetarian\":false},{\"ID\":34,\"name\":\"Ham\",\"caloriesPerSlice\":100,\"vegetarian\":false}],\"tool\":\"Scissors\"},\"calories\":625,\"vegetarian\":false}\n",
  "frameUrl": "https://quickpizza.grafana.com/",
  "acceptLanguageHeader": null,
  "acceptLanguageHeaders": [
    ""
  ],
  "headers": {
    "date": "Wed, 29 Jan 2025 18:58:37 GMT",
    "content-type": "application/json",
    "content-length": "754"
  },
  "headersArray": [
    {
      "name": "date",
      "value": "Wed, 29 Jan 2025 18:58:37 GMT"
    },
    {
      "name": "content-type",
      "value": "application/json"
    },
    {
      "name": "content-length",
      "value": "754"
    }
  ],
  "json": {
    "pizza": {
      "tool": "Scissors",
      "ID": 531,
      "name": "The Lucky Americana",
      "dough": {
        "ID": 2,
        "name": "Thick",
        "caloriesPerSlice": 200
      },
      "ingredients": [
        {
          "ID": 1,
          "name": "Extra virgin olive oil",
          "caloriesPerSlice": 50,
          "vegetarian": true
        },
        {
          "vegetarian": true,
          "ID": 4,
          "name": "Bianco di Nizza tomatoes",
          "caloriesPerSlice": 50
        },
        {
          "caloriesPerSlice": 100,
          "vegetarian": true,
          "ID": 6,
          "name": "Mozzarella"
        },
        {
          "ID": 14,
          "name": "Extra cheese",
          "caloriesPerSlice": 100,
          "vegetarian": true
        },
        {
          "ID": 19,
          "name": "Garlic",
          "caloriesPerSlice": 25,
          "vegetarian": true
        },
        {
          "vegetarian": true,
          "ID": 24,
          "name": "Ricotta cheese",
          "caloriesPerSlice": 100
        },
        {
          "ID": 31,
          "name": "Shrimp",
          "caloriesPerSlice": 100,
          "vegetarian": false
        },
        {
          "ID": 34,
          "name": "Ham",
          "caloriesPerSlice": 100,
          "vegetarian": false
        }
      ]
    },
    "calories": 625,
    "vegetarian": false
  },
  "ok": true,
  "requestUrl": "https://quickpizza.grafana.com/api/pizza",
  "securityDetails": {
    "subjectName": "grafana.com",
    "issuer": "DigiCert Global G2 TLS RSA SHA256 2020 CA1",
    "validFrom": 1706832000,
    "validTo": 1741132799,
    "protocol": "TLS 1.3",
    "sanList": [
      "grafana.com",
      "*.grafana.com",
      "*.grafana.net",
      "*.grafana.org",
      "*.hosted-metrics.grafana.net",
      "*.raintank.io",
      "grafana.net",
      "grafana.org",
      "raintank.io",
      "*.grafanalabs.com",
      "grafanalabs.com",
      "*.raintank.com",
      "raintank.com"
    ]
  },
  "serverAddr": {
    "ip_address": "18.221.113.152",
    "port": 443
  },
  "size": {
    "headers": 107,
    "body": 754
  },
  "status": 200,
  "statusText": "",
  "url": "https://quickpizza.grafana.com/api/pizza",
  "text": "{\"pizza\":{\"ID\":531,\"name\":\"The Lucky Americana\",\"dough\":{\"ID\":2,\"name\":\"Thick\",\"caloriesPerSlice\":200},\"ingredients\":[{\"ID\":1,\"name\":\"Extra virgin olive oil\",\"caloriesPerSlice\":50,\"vegetarian\":true},{\"ID\":4,\"name\":\"Bianco di Nizza tomatoes\",\"caloriesPerSlice\":50,\"vegetarian\":true},{\"ID\":6,\"name\":\"Mozzarella\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":14,\"name\":\"Extra cheese\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":19,\"name\":\"Garlic\",\"caloriesPerSlice\":25,\"vegetarian\":true},{\"ID\":24,\"name\":\"Ricotta cheese\",\"caloriesPerSlice\":100,\"vegetarian\":true},{\"ID\":31,\"name\":\"Shrimp\",\"caloriesPerSlice\":100,\"vegetarian\":false},{\"ID\":34,\"name\":\"Ham\",\"caloriesPerSlice\":100,\"vegetarian\":false}],\"tool\":\"Scissors\"},\"calories\":625,\"vegetarian\":false}\n"
}  source=console

Differences

Below are listed the differences between the current implementation in k6 browser and Playwright:

  1. In k6 browser there is a size() method on the response object. In Playwright there is no such method. The size of the request and response data is in request.sizes() in Playwright.
  2. We start to retrieve values from methods that we couldn't in request when working with page.on('request') as detailed in this comment:
    1. response.request().response() now returns the response, although there should be no need to do this.
    2. reponse.request().timing() now returns almost all the values, apart from reponse.request().timing().responseEnd.
  3. I had to fix fetchBody so that it retries if no body is returned one the first call to chrome to retrieve it. If we implement page.on('requestfinished') we should be able to retrieve the response.body() without having to do a retry on fetchBody.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests

2 participants