Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation Instructions #55

Open
BenVosper opened this issue Sep 14, 2023 · 26 comments
Open

Installation Instructions #55

BenVosper opened this issue Sep 14, 2023 · 26 comments
Labels
documentation Improvements or additions to documentation

Comments

@BenVosper
Copy link

Could some basic installation documentation be provided to show users how to make use of the provided binaries?

The blog page demonstrates the use of the @puppeteer/browsers package to install the binaries, but this isn't possible in all environments. And this repo doesn't mention anything about installation.

I appreciate that it's complex to provide detailed instructions for every platform and use-case, but even some simple guidelines would be helpful. eg. "Download these, unzip to , mark as executable...".

I've managed to get everything working in a Debian-based container, but I'm not sure if I'm doing it right at all. For example, it appears that I also need to install the main Debian chrome package to install various dependencies the Chrome for Testing binary needs. I'm not sure if this is expected or not.

Anyway, thanks for working on this! The APIs and binaries work well, just a thought on user-experience.

@thiagowfx
Copy link
Contributor

it appears that I also need to install the main Debian chrome package to install various dependencies the Chrome for Testing binary needs. I'm not sure if this is expected or not.

Related: https://stackoverflow.com/questions/76730257/how-to-install-chrome-for-testing-from-zip-file-on-debian

@thiagowfx
Copy link
Contributor

Also related (sorry, internal only): https://crbug.com/1433228 ("CfT cannot run on many cloud environments (same for Chromium)")

@BenVosper
Copy link
Author

@thiagowfx Yeah, I found that Stackoverflow link. That's where I got the suggestion about the dependencies. But I wasn't sure if that is the official suggested way of doing things. If that is the expected way, it would be great to have a brief explanation in the repo

@thiagowfx
Copy link
Contributor

It's not official, I was just linking it for completeness. Your original FR is still valid, docs regarding dependencies could be improved :-)

@thiagowfx thiagowfx added the documentation Improvements or additions to documentation label Sep 15, 2023
@guest271314
Copy link

I've managed to get everything working in a Debian-based container, but I'm not sure if I'm doing it right at all. For example, it appears that I also need to install the main Debian chrome package to install various dependencies the Chrome for Testing binary needs. I'm not sure if this is expected or not.

I ordinarily just fetch the zip file, extract, then use a shell script to launch Chrome or Chromium. I would not mix Debian (PPA or official releases) with the Chrome-For-Testing or Chromium Developer Build.

Something like

wget --show-progress --progress=bar --output-document chrome.zip <URL> && unzip chrome.zip && rm chrome.zip

Then create a shell script, chmod u+x launch_chrome_for_testing.sh including something like

#!/bin/sh
$HOME/chrome-for-testing/chrome \
--silent-debugger-extension-api \
--offscreen-document-testing \
--aggressive-cache-discard \
--profile-directory="Default" \
--install-isolated-web-app-from-file="/home/user/telnet-client/dist/telnet.swbn" \
--allow-file-access-from-files \
# ...

@BenVosper
Copy link
Author

@guest271314 Thanks for this. Absolutely agree that installing the Debian Chrome package is asking for trouble.

But just unzipping the zip and running the binary doesn't seem to work for me. I get various shared libraries errors (libcups, libatk etc.) which indicates that I'm missing the dependencies. Same for the chromedriver binary. I can use ldd to see what's missing. There are ~10 or so missing for chrome.

So some official info on how to run the provided binaries as-is, or some way to work out and install the required binaries would be great.

@guest271314
Copy link

If you are on Debian you shouldn't be getting errors. I do the same process for Chromium from https://download-chromium.appspot.com/dl/Linux_x64?type=snapshots.

Do you have snap installed?

sudo systemctl disable snapd.service \
systemctl disable snapd.socket \
systemctl disable snapd.seeded.service

sudo snap remove firefox snap-store gtk-common-themes \
&& snap remove gnome-3-38-2004 && snap remove snapd-desktop-integration core20

sudo rm -rf /var/cache/snapd/

@guest271314
Copy link

I've actually done the same thing with .deb archives. Extract, locate the chrome executable, create a shell script, run the executable via a launcher I create.

@guest271314
Copy link

which indicates that I'm missing the dependencies.

There shouldn't be any dependencies. chrome executable runs standalone, just like the node executable, deno executable, firefox executable.

Just make sure snap is not interfering with things. You don;t want to be doing any apt or apt-get or snap calls on the archive or at all relevant to Chrome or Chromium. You can run the executable standalone.

@BenVosper
Copy link
Author

@guest271314 No snap or anything. Should have mentioned I'm running in docker on the ubuntu:jammy image. Including my minimal dockerfile below:

# Dockerfile
FROM ubuntu:jammy

RUN apt-get -y update
RUN apt-get install -y wget unzip

RUN wget --output-document chrome.zip \
    https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/117.0.5938.88/linux64/chrome-linux64.zip \
    && unzip chrome.zip && rm chrome.zip

Building:

docker build -t cft-test .

Running the binary directly as in your example:

docker run --rm cft-test ./chrome-linux64/chrome

Output:

./chrome-linux64/chrome: error while loading shared libraries: libgobject-2.0.so.0: cannot open shared object file: No such file or directory

So it definitively seems that in docker at least there is a required undocumented "install dependencies" step. Unless I'm missing something.

@guest271314
Copy link

Never tried Docker. Can't help you there. What happens when you are not running in Docker?

@BenVosper
Copy link
Author

@guest271314 No worries. I haven't tried it outside of docker. We need a reproducible container with access to these Chrome for Testing binaries for use in automated tests. It's probably working on your machine because you've already got the dependencies installed from somewhere else.

@guest271314
Copy link

I'm pretty sure Chromium is a standalone executable.

@eliot1785
Copy link

I'm experiencing the same issue - same exact dependencies missing in Debian. I'm not really sure why it would matter that I'm running in Docker...?

@blazerzar
Copy link

Probably not all dependencies are installed, depending on the base image you are using. I got it working with python:slim-bookworm by installing dependencies mentioned in
https://stackoverflow.com/questions/76730257/how-to-install-chrome-for-testing-from-zip-file-on-debian.

In the end i settled for the ones listed in
https://source.chromium.org/chromium/chromium/src/+/main:chrome/installer/linux/debian/dist_package_versions.json
since the list is a little smaller and official.

@finalfantasia
Copy link

finalfantasia commented Jan 11, 2024

The following is what I did to have Chrome for Testing working on an Amazon EC2 instance running Amazon Linux 2023. I hope this helps:

With Node.js

# A hacky way to install all the package dependencies required for Chrome for Testing.
# See: https://github.com/GoogleChromeLabs/chrome-for-testing/issues/55
sudo dnf install --assumeyes --quiet findutils
sudo dnf deplist https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm | \
  grep provider | \
  sort --unique | \
  awk '{print $2}' | \
  xargs sudo dnf install --best --allowerasing --skip-broken --assumeyes --quiet > /dev/null 2>&1

npx --yes @puppeteer/browsers install chrome@stable | \
  awk '{print $2}' | \
  xargs -I {} sudo ln --symbolic {} /usr/local/bin/chrome

npx --yes @puppeteer/browsers install chromedriver@stable | \
  awk '{print $2}' | \
  xargs -I {} sudo ln --symbolic {} /usr/local/bin/chromedriver

Without Node.js

# A hacky way to install all the package dependencies required for Chrome for Testing.
# See: https://github.com/GoogleChromeLabs/chrome-for-testing/issues/55
sudo dnf install --assumeyes --quiet findutils jq unzip
sudo dnf deplist https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm | \
  grep provider | \
  sort --unique | \
  awk '{print $2}' | \
  xargs sudo dnf install --best --allowerasing --skip-broken --assumeyes --quiet > /dev/null 2>&1

CHROME_FOR_TESTING_RELEASE="$(curl --silent https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json | jq '.channels.Stable')"

CHROME_DOWNLOAD_URL="$(echo ""${CHROME_FOR_TESTING_RELEASE}"" | jq -r '.downloads.chrome[] | select(.platform == "linux64") | .url')"
curl --remote-name --silent "${CHROME_DOWNLOAD_URL}"
unzip -q chrome-linux64.zip
sudo ln --symbolic "${PWD}/chrome-linux64/chrome" /usr/local/bin/chrome

CHROMEDRIVER_DOWNLOAD_URL="$(echo ""${CHROME_FOR_TESTING_RELEASE}"" | jq -r '.downloads.chromedriver[] | select(.platform == "linux64") | .url')"
curl --remote-name --silent "${CHROMEDRIVER_DOWNLOAD_URL}"
unzip -q chromedriver-linux64.zip
sudo ln --symbolic "${PWD}/chromedriver-linux64/chromedriver" /usr/local/bin/chromedriver

@guest271314
Copy link

For Debian and Ubuntu - I am not talking about Docker - you can do this, here using Deno. I can adjust the code for use in Node.js, and Bun

// Fetch and unzip Chrome-For-Testing Canary Channel
// deno run -A fetch_unzip_chrome_for_testing_canary_linux.js
/* eslint-disable no-console */
/* global Deno, Intl */

"use strict";

// import { parse } from "https://deno.land/std/flags/mod.ts";
import { exists } from "https://deno.land/std/fs/mod.ts";
import { basename, dirname } from "https://deno.land/std/path/mod.ts";
import {
  ERR_HTTP_RANGE,
  HttpReader,
  terminateWorkers,
  Uint8ArrayReader,
  Uint8ArrayWriter,
  ZipReader,
} from "https://raw.githubusercontent.com/gildas-lormeau/zip.js/master/index.js";

const executables = new Set([
  "chrome",
  "chrome-wrapper",
  "chrome_crashpad_handler",
  "chrome_sandbox",
  "libEGL.so",
  "libGLESv2.so",
  "libvk_swiftshader.so",
  "libvulkan.so.1",
  //"nacl_helper",
  //"nacl_helper_bootstrap",
  //"nacl_irt_x86_64.nexe",
  "xdg-mime",
  "xdg-settings",
]);

const json = await (await fetch(
  "https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json",
)).json();
const {
  url,
} = json.channels.Canary.downloads.chrome.find(({
  platform,
}) => platform === "linux64");

console.log(`Fetch ${url}`);

const response = await fetch(url);

const length = response.headers.get("content-length");

const ab = new ArrayBuffer(length);

const uint8 = new Uint8Array(ab);

const encoder = new TextEncoder();

let offset = 0;

async function log(bytes) {
  // https://medium.com/deno-the-complete-reference/deno-nuggets-overwrite-a-console-log-line-2513e52e264b
  await Deno.stdout.write(
    encoder.encode(`${bytes} of ${length} bytes written.\r`),
  );
}
// Just so we see what's going on
await response.body.pipeTo(
  new WritableStream({
    start() {
      console.log("Start reading stream.");
    },
    async write(value) {
      uint8.set(value, offset);
      await log(offset += value.length);
    },
    close() {
      console.log("\nDone reading stream.");
    },
  }),
);

// const ab = await response.arrayBuffer();
// console.log(ab.byteLength);

await Deno.writeFile("chrome-linux64.zip", uint8);

unzip({ _: ["chrome-linux64.zip"] }).catch((error) =>
  console.error(error.toString())
);

async function unzip(args) {
  if (args.l) {
    await listEntries(args.l || args._[0]);
  } else {
    const archive = args._.shift();
    if (archive) {
      await unzipEntries(archive, args._);
      await Deno.remove("chrome-linux64.zip");
    }
  }
}

async function unzipEntries(archive, filenames) {
  const zipReader = new ZipReader(await getReader(archive));
  const entries = await zipReader.getEntries();
  let selectedEntries;
  if (filenames.length) {
    selectedEntries = entries.filter((entry) =>
      filenames.includes(entry.filename)
    );
  } else {
    selectedEntries = entries;
  }
  await Promise.all(selectedEntries.map(async (entry) => {
    const entryDirectory = dirname(entry.filename);
    if (!await exists(entryDirectory)) {
      await Deno.mkdir(entryDirectory, { recursive: true });
    }
    if (!entry.directory) {
      await Deno.writeFile(
        entry.filename,
        await entry.getData(new Uint8ArrayWriter()),
      );
    }
  }));
  await terminateWorkers();
  for (const file of executables) {
    await Deno.chmod(`chrome-linux64/${file}`, 0o764);
  }
}

async function listEntries(archive) {
  const zipReader = new ZipReader(await getReader(archive));
  const entries = await zipReader.getEntries();
  let totalSize = 0;
  console.log("Archive: ", archive);
  let maxNameLength = 0;
  const formattedData = entries.map((entry) => {
    const length = formatLength(entry.uncompressedSize);
    const splitDate = entry.lastModDate.toISOString().split("T");
    const date = splitDate[0].padStart(11);
    const time = splitDate[1].match(/([^:]+:[^:]+):/)[1].padEnd(7);
    const name = entry.filename;
    totalSize += entry.uncompressedSize;
    maxNameLength = Math.max(maxNameLength, length.length);
    return { length, date, time, name };
  });
  console.log(
    "Length".padStart(maxNameLength - 1, " "),
    "     Date    Time    Name",
  );
  const lengthSeparator = "-".padStart(maxNameLength, "-");
  console.log(lengthSeparator, " ---------- -----   ----");
  formattedData.forEach(({ length, date, time, name }) =>
    console.log(length.padStart(maxNameLength), date, time, name)
  );
  console.log(lengthSeparator, "                    ----");
  console.log(formatLength(totalSize));
}

function formatLength(length) {
  return new Intl.NumberFormat().format(length);
}

async function getReader(archive) {
  if (/^https?:/.test(archive)) {
    try {
      return new HttpReader(archive, {
        useRangeHeader: true,
        forceRangeRequests: true,
      });
    } catch (error) {
      if (error.message == ERR_HTTP_RANGE) {
        try {
          return new HttpReader(archive, { useRangeHeader: true });
        } catch (error) {
          if (error.message == ERR_HTTP_RANGE) {
            return new HttpReader(archive);
          } else {
            throw error;
          }
        }
      } else {
        throw error;
      }
    }
  } else {
    if (!basename(archive).includes(".")) {
      archive += ".zip";
    }
    return new Uint8ArrayReader(await Deno.readFile(archive));
  }
}

@dagguh
Copy link

dagguh commented May 28, 2024

@guest271314 that's way too much code. Chrome for Testing is for testing. And most testing in the world is automatic. The testing environments are lightweight, come with minimal libs preinstalled. Current CtF dists do not work out of the box, they're not automation-friendly.

@guest271314
Copy link

@dagguh

And most testing in the world is automatic.

You don't get "automatic" testing automatically. You have to write code first.

I don't think there is a single URL that we can do for Chrome-For-Testing what we can do for Chromium Developer Channel.

wget --show-progress --progress=bar --output-document chrome.zip https://download-chromium.appspot.com/dl/Linux_x64?type=snapshots && unzip chrome.zip && rm chrome.zip

@guest271314
Copy link

@dagguh Technically we can do this in the browser. However, the relevant files in executables variable above will not be executable - unless we create the flat files first with touch, set them executable with chmod and overwrite the files to retain permissions

var text = await (await fetch('https://raw.githubusercontent.com/Stuk/jszip/main/dist/jszip.min.js')).blob();
await import(URL.createObjectURL(new Blob([text], {
  type: 'text/javascript'
})));

var json = await (await fetch('https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json')).json();
var {
  url
} = json.channels.Canary.downloads.chrome.find(({
  platform
}) => platform === 'linux64');

console.log(url);

var dir = await showDirectoryPicker({
  mode: 'readwrite',
});

var handle = await showSaveFilePicker({
  startIn: 'downloads',
  mode: 'readwrite',
  suggestedName: 'chrome-linux64.zip'
});


async function handleFile(handle) {

  console.log(dir.name, dir);
  var file = await handle.getFile();
  var files = await JSZip.loadAsync(file, {
    createFolders: true
  }); 
  for (const [key, value] of Object.entries(files.files)) {
    try {
      if (value.dir && value.name.slice(0, -1) !== dir.name) {
        var path = value.name.split('/').filter(Boolean).pop();
        console.log('dir', path, value.name);

        await dir.getDirectoryHandle(path, {
          create: true
        });

      } else {
        if (!value.dir) {
          var path = value.name.split('/').filter(Boolean);
          console.log('file', path, path[path.length - 2]);

          var folder = path[path.length - 2] === dir.name ? dir : await dir.getDirectoryHandle(path[path.length - 2], {
            create: false
          });
          var curr = await folder.getFileHandle(path[path.length - 1], {
            create: true
          });
          var writable = await curr.createWritable();
          var writer = writable.getWriter();
          await writer.write(value._data.compressedContent);
          await writer.close();

        }
      }
    } catch (e) {
      console.log(e, value);
    }
  }
  console.log(dir);
  await handle.remove();
  return `Done writing ${dir.name}`;
}

var {
  resolve,
  promise
} = Promise.withResolvers();

var fso = new FileSystemObserver(
  async ([{
    changedHandle,
    root,
    type
  }], record) => {
    try {
      console.log((await changedHandle.getFile()).size);
      fso.disconnect();
      fso.unobserve(handle);
      resolve(changedHandle);
    } catch (e) {
      console.warn(e);
    }
  },
);

fso.observe(handle).then(() => console.log(`Observing ${handle.name}`));

open(url);

handleFile(await promise).then(console.log).catch(console.error);

@guest271314
Copy link

@dagguh If you want "automatically" you can run the code above in the browser, then again use Native Messaging from the browser to set the relevant files executable. All of the above can be automated.

The "too much code" is N/A. I didn't know we were golfing here. If amount of code is an issue, we can golf this.

@armahillo
Copy link

I appreciate this thread, because we are having very similar issues.

What’s the easiest way to download Chrome for Testing binaries?

It's a bit silly to note the "easiest" way when not acknowledging alternate, ostensibly less-easy, methods, eh? This all pre-supposes that the user has node/npm installed already, which I do not, so this is actually notably harder than providing a URL to a list of downloadable binaries, eg.

@hyyrdbzm
Copy link

hyyrdbzm commented Sep 23, 2024

$ cd /opt
$ curl -L -O 'https://storage.googleapis.com/chrome-for-testing-public/129.0.6668.58/linux64/chrome-linux64.zip'
$ unzip chrome-linux64.zip
$ apt update
$ while read pkg ; do apt-get satisfy -y --no-install-recommends "${pkg}" ; done < chrome-linux64/deb.deps
$ chown root:root chrome-linux64/chrome_sandbox
$ chmod 4755 chrome-linux64/chrome_sandbox
$ export PATH="/opt/chrome-linux64:${PATH}"
$ export CHROME_DEVEL_SANDBOX="/opt/chrome-linux64/chrome_sandbox"

@artemmakur
Copy link

This issue is still affecting developers and testers -- I had to dig up install instructions for Ubuntu from an obscure StackOverflow thread. A little instruction on how to get Chrome for Testing to run would be much appreciated next time around.

@nekdan
Copy link

nekdan commented Nov 14, 2024

This issue is still affecting developers and testers -- I had to dig up install instructions for Ubuntu from an obscure StackOverflow thread. A little instruction on how to get Chrome for Testing to run would be much appreciated next time around.

Hi, I'm also trying to run Chrome for testing on Ubuntu, albeit in Docker. I was able to download the archive, but I don't understand what to do next. I found some instructions, but Chrome for testing still won't start. Could you send me a link to the instructions you found?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests