Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test(analyses-snapshots): update analyses snapshot local instructions #16918

Merged
merged 6 commits into from
Nov 21, 2024
Merged

test(analyses-snapshots): update analyses snapshot local instructions #16918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d332872
test(analyses-snapshot): add to docs and update Makefile
y3rsh Nov 20, 2024
47c6819
more
y3rsh Nov 20, 2024
d1e6027
better variable names and consolidate targets
y3rsh Nov 20, 2024
4449e76
Update analyses-snapshot-testing/README.md
y3rsh Nov 20, 2024
b79d31f
Merge branch 'edge' into update-analyses-snapshot-local-instructions
y3rsh Nov 21, 2024
ffd02e0
format readme
y3rsh Nov 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 38 additions & 14 deletions analyses-snapshot-testing/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@ CACHEBUST ?= $(shell date +%s)
ANALYSIS_REF ?= edge
PROTOCOL_NAMES ?= all
OVERRIDE_PROTOCOL_NAMES ?= all
OPENTRONS_VERSION ?= edge
LOCAL_IMAGE_TAG ?= local
ANALYZER_IMAGE_NAME ?= opentrons-analysis

export OPENTRONS_VERSION # used for server
export ANALYSIS_REF # used for analysis and snapshot test
export PROTOCOL_NAMES # used for the snapshot test
export OVERRIDE_PROTOCOL_NAMES # used for the snapshot test
export ANALYSIS_REF # tag, branch or commit for the opentrons repository. Used as the image tag for the analyzer image
export PROTOCOL_NAMES # tell the test which protocols to run
export OVERRIDE_PROTOCOL_NAMES # tell the test which override protocols to run

ifeq ($(CI), true)
PYTHON=python
Expand Down Expand Up @@ -93,23 +93,47 @@ build-base-image:

.PHONY: build-opentrons-analysis
build-opentrons-analysis:
@echo "Building docker image for $(ANALYSIS_REF)"
@echo "The image will be named opentrons-analysis:$(ANALYSIS_REF)"
@echo "If you want to build a different version, run 'make build-opentrons-analysis ANALYSIS_REF=<version>'"
docker build --build-arg BASE_IMAGE_NAME=$(BASE_IMAGE_NAME) --build-arg ANALYSIS_REF=$(ANALYSIS_REF) --build-arg CACHEBUST=$(CACHEBUST) -t opentrons-analysis:$(ANALYSIS_REF) -f citools/Dockerfile.analyze citools/.
@echo "Building docker image for opentrons repository reference$(ANALYSIS_REF)"
@echo "The image will be named $(ANALYZER_IMAGE_NAME):$(ANALYSIS_REF)"
@echo "If you want to build a different version, run 'make build-opentrons-analysis ANALYSIS_REF=<tag, branch, or commit>'"
docker build --build-arg BASE_IMAGE_NAME=$(BASE_IMAGE_NAME) --build-arg ANALYSIS_REF=$(ANALYSIS_REF) --build-arg CACHEBUST=$(CACHEBUST) -t $(ANALYZER_IMAGE_NAME):$(ANALYSIS_REF) -f citools/Dockerfile.analyze citools/.

.PHONY: local-build
local-build:
.PHONY: build-local
build-local:
@echo "Building docker image for your local opentrons code"
@echo "The image will be named opentrons-analysis:local"
@echo "For a fresh build, run 'make local-build NO_CACHE=1'"
docker build --build-arg BASE_IMAGE_NAME=$(BASE_IMAGE_NAME) $(BUILD_FLAGS) -t opentrons-analysis:local -f citools/Dockerfile.local .. || true
@echo "This image will be named $(ANALYZER_IMAGE_NAME):$(LOCAL_IMAGE_TAG)"
docker build --build-arg BASE_IMAGE_NAME=$(BASE_IMAGE_NAME) -t $(ANALYZER_IMAGE_NAME):$(LOCAL_IMAGE_TAG) -f citools/Dockerfile.local ..
@echo "Build complete"

.PHONY: snapshot-test-local
snapshot-test-local: ANALYSIS_REF=$(LOCAL_IMAGE_TAG)
snapshot-test-local: build-base-image build-local
@echo "This target is overriding the ANALYSIS_REF to the LOCAL_IMAGE_TAG: $(LOCAL_IMAGE_TAG)"
@echo "ANALYSIS_REF is $(ANALYSIS_REF). The the test maps this env variable to the image tag."
@echo "The image the test will use is $(ANALYZER_IMAGE_NAME):$(LOCAL_IMAGE_TAG)"
@echo "PROTOCOL_NAMES is $(PROTOCOL_NAMES)"
@echo "OVERRIDE_PROTOCOL_NAMES is $(OVERRIDE_PROTOCOL_NAMES)"
$(PYTHON) -m pipenv run pytest -k analyses_snapshot_test -vv

.PHONY: snapshot-test-update-local
snapshot-test-update-local: ANALYSIS_REF=$(LOCAL_IMAGE_TAG)
snapshot-test-update-local: build-base-image build-local
@echo "This target is overriding the ANALYSIS_REF to the LOCAL_IMAGE_TAG: $(LOCAL_IMAGE_TAG)"
@echo "ANALYSIS_REF is $(ANALYSIS_REF). The the test maps this env variable to the image tag."
@echo "The image the test will use is $(ANALYZER_IMAGE_NAME):$(LOCAL_IMAGE_TAG)"
@echo "PROTOCOL_NAMES is $(PROTOCOL_NAMES)"
@echo "OVERRIDE_PROTOCOL_NAMES is $(OVERRIDE_PROTOCOL_NAMES)"
$(PYTHON) -m pipenv run pytest -k analyses_snapshot_test --snapshot-update

.PHONY: generate-protocols
generate-protocols:
$(PYTHON) -m pipenv run python -m automation.data.protocol_registry

# Tools for running the robot server in a container

OPENTRONS_VERSION ?= edge
export OPENTRONS_VERSION # used for the robot server image as the tag, branch or commit for the opentrons repository

.PHONY: build-rs
build-rs:
@echo "Building docker image for opentrons-robot-server:$(OPENTRONS_VERSION)"
Expand Down
17 changes: 12 additions & 5 deletions analyses-snapshot-testing/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

1. Follow the instructions in [DEV_SETUP.md](../DEV_SETUP.md)
1. `cd analyses-snapshot-testing`
1. use pyenv to install python 3.12 and set it as the local python version for this directory
1. use pyenv to install python 3.13 and set it as the local python version for this directory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do we use that requires Python 3.13 here, and why do they other dev environment setup instructions recommend 3.10 instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are locked to 3.10 for all python on robots because of reasons I don't know specifically ,but that have to do with the burden of upgrading python version in buildroot and oe-core.
The image running the code under test is 3.10, just like it is in the app and on the robots.
The use of 3.13 is only for the testing tools in this folder. It is faster and is a place we can explore latest and greatest features 😄

1. `make setup`
1. Have docker installed and ready

Expand Down Expand Up @@ -72,10 +72,17 @@ cd analyses-snapshot-testing \

> This copies in your local code to the container and runs the analyses battery against it.

1. `make build-base-image`
1. `make build-local`
1. `make local-snapshot-test`
`cd PYENV_ROOT && git pull` - make sure pyenv is up to date so you may install python 3.13.0
`pyenv install 3.13.0` - install python 3.13.0
`cd <OPENTRONS_REPO_ROOT>/analyses-snapshot-testing` - navigate to the analyses-snapshot-testing directory
`pyenv local 3.13.0` - set the local python version to 3.13.0
`make setup` - install the requirements
`make snapshot-test-local` - this target builds the base image, builds the local code into the base image, then runs the analyses battery against the image you just created

You have the option to specify one or many protocols to run the analyses on. This is also described above [Running the tests against specific protocols](#running-the-tests-against-specific-protocols)

- `make local-snapshot-test PROTOCOL_NAMES=Flex_S_v2_19_Illumina_DNA_PCR_Free OVERRIDE_PROTOCOL_NAMES=none`
- `make snapshot-test-local PROTOCOL_NAMES=Flex_S_v2_19_Illumina_DNA_PCR_Free OVERRIDE_PROTOCOL_NAMES=none`

### Updating the snapshots locally

- `make snapshot-test-update-local` - this target builds the base image, builds the local code into the base image, then runs the analyses battery against the image you just created, updating the snapshots by passing the `--update-snapshots` flag to the test
Loading