Skip to content

Latest commit

 

History

History
222 lines (155 loc) · 7.62 KB

e2e-node-tests.md

File metadata and controls

222 lines (155 loc) · 7.62 KB

WARNING WARNING WARNING WARNING WARNING

PLEASE NOTE: This document applies to the HEAD of the source tree

If you are using a released version of Kubernetes, you should refer to the docs that go with that version.

The latest release of this document can be found [here](http://releases.k8s.io/release-1.3/docs/devel/e2e-node-tests.md).

Documentation for other releases can be found at releases.k8s.io.

Node End-To-End tests

Node e2e tests are component tests meant for testing the Kubelet code on a custom host environment.

Tests can be run either locally or against a host running on GCE.

Node e2e tests are run as both pre- and post- submit tests by the Kubernetes project.

Note: Linux only. Mac and Windows unsupported.

Running tests

Locally

Why run tests Locally? Much faster than running tests Remotely.

Prerequisites:

  • Install etcd on your PATH
    • Verify etcd is installed correctly by running which etcd
  • Install ginkgo on your PATH
    • Verify ginkgo is installed correctly by running which ginkgo

From the Kubernetes base directory, run:

make test_e2e_node

This will: run the ginkgo binary against the subdirectory test/e2e_node, which will in turn:

  • Ask for sudo access (needed for running some of the processes)
  • Build the Kubernetes source code
  • Pre-pull docker images used by the tests
  • Start a local instance of etcd
  • Start a local instance of kube-apiserver
  • Start a local instance of kubelet
  • Run the test using the locally started processes
  • Output the test results to STDOUT
  • Stop kubelet, kube-apiserver, and etcd

Remotely

Why Run tests Remotely? Tests will be run in a customized pristine environment. Closely mimics what will be done as pre- and post- submit testing performed by the project.

Prerequisites:

  • join the googlegroup [email protected]
    • This provides read access to the node test images.
  • Setup a Google Cloud Platform account and project with Google Compute Engine enabled
  • Install and setup the gcloud sdk
    • Verify the sdk is setup correctly by running gcloud compute instances list and gcloud compute images list --project kubernetes-node-e2e-images

Run:

make test_e2e_node REMOTE=true

This will:

  • Build the Kubernetes source code
  • Create a new GCE instance using the default test image
    • Instance will be called test-e2e-node-containervm-v20160321-image
  • Lookup the instance public ip address
  • Copy a compressed archive file to the host containing the following binaries:
    • ginkgo
    • kubelet
    • kube-apiserver
    • e2e_node.test (this binary contains the actual tests to be run)
  • Unzip the archive to a directory under /tmp/gcloud
  • Run the tests using the ginkgo command
    • Starts etcd, kube-apiserver, kubelet
    • The ginkgo command is used because this supports more features than running the test binary directly
  • Output the remote test results to STDOUT
  • scp the log files back to the local host under /tmp/_artifacts/e2e-node-containervm-v20160321-image
  • Stop the processes on the remote host
  • Leave the GCE instance running

Note: Subsequent tests run using the same image will reuse the existing host instead of deleting it and provisioning a new one. To delete the GCE instance after each test see DELETE_INSTANCE.

Additional Remote Options

Run tests using different images

This is useful if you want to run tests against a host using a different OS distro or container runtime than provided by the default image.

List the available test images using gcloud.

make test_e2e_node LIST_IMAGES=true

This will output a list of the available images for the default image project.

Then run:

make test_e2e_node REMOTE=true IMAGES="<comma-separated-list-images>"

Run tests against a running GCE instance (not an image)

This is useful if you have an host instance running already and want to run the tests there instead of on a new instance.

make test_e2e_node REMOTE=true HOSTS="<comma-separated-list-of-hostnames>"

Delete instance after tests run

This is useful if you want recreate the instance for each test run to trigger flakes related to starting the instance.

make test_e2e_node REMOTE=true DELETE_INSTANCES=true

Keep instance, test binaries, and processes around after tests run

This is useful if you want to manually inspect or debug the kubelet process run as part of the tests.

make test_e2e_node REMOTE=true CLEANUP=false

Run tests using an image in another project

This is useful if you want to create your own host image in another project and use it for testing.

make test_e2e_node REMOTE=true IMAGE_PROJECT="<name-of-project-with-images>" IMAGES="<image-name>"

Setting up your own host image may require additional steps such as installing etcd or docker. See setup_host.sh for common steps to setup hosts to run node tests.

Create instances using a different instance name prefix

This is useful if you want to create instances using a different name so that you can run multiple copies of the test in parallel against different instances of the same image.

make test_e2e_node REMOTE=true INSTANCE_PREFIX="my-prefix"

Additional Test Options for both Remote and Local execution

Only run a subset of the tests

To run tests matching a regex:

make test_e2e_node REMOTE=true FOCUS="<regex-to-match>"

To run tests NOT matching a regex:

make test_e2e_node REMOTE=true SKIP="<regex-to-match>"

Run tests continually until they fail

This is useful if you are trying to debug a flaky test failure. This will cause ginkgo to continually run the tests until they fail. Note: this will only perform test setup once (e.g. creating the instance) and is less useful for catching flakes related creating the instance from an image.

make test_e2e_node REMOTE=true RUN_UNTIL_FAILURE=true

Notes on tests run by the Kubernetes project during pre-, post- submit.

The node e2e tests are run by the PR builder for each Pull Request and the results published at the bottom of the comments section. To re-run just the node e2e tests from the PR builder add the comment @k8s-bot node e2e test this issue: #<Flake-Issue-Number or IGNORE> and include a link to the test failure logs if caused by a flake.

The PR builder runs tests against the images listed in jenkins-pull.properties

The post submit tests run against the images listed in jenkins-ci.properties

Analytics