Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor scheduled workflow for stability #134

Merged
merged 2 commits into from
Sep 25, 2024
Merged

Refactor scheduled workflow for stability #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e66e791
Update scheduled workflow cron
yanghua Sep 25, 2024
8f07ed2
Refactor scheduled workflow for stability
yanghua Sep 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/scheduled_workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ name: Scheduled Workflow

on:
schedule:
# Runs every day at midnight
- cron: '0 0 * * *'
# Runs every day at midnight and noon (beijing time)
- cron: '0 4,16 * * *'

jobs:
build:
Expand Down Expand Up @@ -50,5 +50,5 @@ jobs:
echo "TOS_REGION=${{ vars.TOS_REGION }}" >> $GITHUB_ENV
echo "TOS_ENDPOINT=${{ vars.TOS_ENDPOINT }}" >> $GITHUB_ENV
echo "TOSFS_LOGGING_LEVEL=${{ vars.TOSFS_LOGGING_LEVEL }}" >> $GITHUB_ENV
- name: Run tests
run: make test
- name: Run stability tests
run: make test_stability
9 changes: 7 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ help: ## Show the help.
@echo "install: ## Install the project in dev mode."
@echo "fmt: ## Format code using black & isort."
@echo "lint: ## Run pep8, black, mypy linters."
@echo "test: lint ## Run tests and generate coverage report."
@echo "test: ## Run tests and generate coverage report."
@echo "test_stability: ## Run stability tests."
@echo "watch: ## Run tests on every change."
@echo "clean: ## Clean unused files."
@echo "release: ## Create a new tag for release."
Expand Down Expand Up @@ -47,7 +48,11 @@ lint: ## Run pep8, black, mypy linters.

.PHONY: test
test: ## Run tests and generate coverage report.
$(ENV_PREFIX)pytest -vv -s --cov-config .coveragerc --cov=tosfs -l --tb=short --maxfail=1 ${TEST_DIR}
$(ENV_PREFIX)pytest -vv -s --cov-config .coveragerc --cov=tosfs -l --tb=short --maxfail=1 ${TEST_DIR} --ignore=${TEST_DIR}/test_stability.py

.PHONY: test_stability
test_stability: ## Run stability tests.
$(ENV_PREFIX)pytest -vv -s --cov-config .coveragerc --cov=tosfs -l --tb=short --maxfail=1 ${TEST_DIR}/test_stability.py

.PHONY: watch
watch: ## Run tests on every change.
Expand Down
4 changes: 2 additions & 2 deletions tosfs/tests/test_stability.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def test_write_breakpoint_continuation(tosfs, bucket, temporary_workspace):
with tosfs.open(f"{bucket}/{temporary_workspace}/{file_name}", "w") as f:
f.write(first_part)
# mock a very long block(business processing or network issue)
sleep(60)
sleep(60 * 15)
f.write(second_part)

assert tosfs.info(f"{bucket}/{temporary_workspace}/{file_name}")["size"] == len(
Expand All @@ -49,7 +49,7 @@ def test_read_breakpoint_continuation(tosfs, bucket, temporary_workspace):
read_first_part = f.read(10 * 1024 * 1024)
assert read_first_part == first_part
# mock a very long block(business processing or network issue)
sleep(60)
sleep(60 * 15)
read_second_part = f.read(10 * 1024 * 1024)
assert read_second_part == second_part
assert read_first_part + read_second_part == first_part + second_part