Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: use uv to install packages #11679

Merged
merged 1 commit into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,9 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Used versions
run: ./ci/print-versions
- name: Install pip dependencies
- name: Install Python dependencies
run: ./ci/pip-install

- name: Initialize CodeQL
Expand Down
15 changes: 8 additions & 7 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: docs/requirements.txt
- name: Install apt dependencies
run: |
sudo apt update
sudo apt install -y graphviz
- name: Install pip dependencies
run: pip install -r docs/requirements.txt
- name: Install Python dependencies
run: |
pip install uv
uv pip install --system -r docs/requirements.txt
- name: Sphinx build
run: |
echo "::add-matcher::.github/matchers/sphinx.json"
Expand All @@ -54,10 +54,11 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: docs/requirements.txt
- name: Install pip dependencies
run: pip install -r docs/requirements.txt
- name: Install Python dependencies
run: |
pip install uv
uv pip install --system -r docs/requirements.txt
- name: Sphinx linkcheck
run: |
./ci/run-docs linkcheck
Expand Down
9 changes: 3 additions & 6 deletions .github/workflows/licenses-update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,16 @@ jobs:
submodules: true
- uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.cache/pre-commit
path: ~/.cache/pre-commit
key: ${{ runner.os }}-pre-commit-${{ hashFiles('pyproject.toml') }}-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install $(sed -n 's/.*"\(pip==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(wheel==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
- run: ./scripts/generate-license-data
- name: Update renovate branch
if: github.ref != 'refs/heads/main'
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/migrations.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,7 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: 3.11
cache: pip
- name: Install pip dependencies
- name: Install Python dependencies
run: ./ci/pip-install migrations
- name: Check missing migrations
run: ./ci/run-checkmigrate
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Install dependencies
run: ./ci/pip-install mypy

Expand Down
14 changes: 6 additions & 8 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,22 +24,20 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/cache@v4
with:
path: |
~/.cache/pip
~/.cache/pre-commit
key: ${{ runner.os }}-pre-commit-${{ hashFiles('**/requirements*.txt') }}-${{ hashFiles('.pre-commit-config.yaml') }}-${{ hashFiles('pyproject.toml') }}
path: ~/.cache/pre-commit
key: ${{ runner.os }}-pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Update pip
run: python -m pip install --upgrade pip wheel
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install dependencies
run: |
if [ -f requirements-lint.txt ] ; then
pip install -r requirements-lint.txt
uv pip install --system -r requirements-lint.txt
else
pip install $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
fi
- name: pre-commit
run: pre-commit run --all
Expand Down
6 changes: 2 additions & 4 deletions .github/workflows/pylint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,10 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Install dependencies
run: |
python -m pip install $(sed -n 's/.*"\(pip==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(wheel==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(pylint==\([^"]*\)\)".*/\1/p' pyproject.toml)
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(pylint==\([^"]*\)\)".*/\1/p' pyproject.toml)
- name: Add pylint annotator
uses: pr-annotators/[email protected]

Expand Down
9 changes: 4 additions & 5 deletions .github/workflows/schema-update.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,11 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install pip dependencies
- name: Install Python dependencies
run: |
python -m pip install $(sed -n 's/.*"\(pip==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(wheel==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(weblate-schemas==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(weblate-schemas==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(pre-commit==\([^"]*\)\)".*/\1/p' pyproject.toml)
- run: make -C docs update-schemas
- run: pre-commit run --files $(git diff --name-only)
continue-on-error: true
Expand Down
28 changes: 13 additions & 15 deletions .github/workflows/setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Used versions
run: ./ci/print-versions
- name: Install pip dependencies
- name: Install Python dependencies
run: ./ci/pip-install
- name: Install setuptools
run: uv pip install --system $(sed -n 's/.*"\(setuptools[<>=]=\([^"]*\)\)".*/\1/p' pyproject.toml)
- name: Install Weblate
run: coverage run ./setup.py install
- name: Check difference
Expand Down Expand Up @@ -67,13 +68,12 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Used versions
run: ./ci/print-versions
- name: Install pip dependencies
- name: Install Python dependencies
run: ./ci/pip-install
- name: Install Weblate
run: pip install '.[all]'
run: uv pip install --system '.[all]'
- name: Check difference
run: diff -ruNqp weblate $(python -c 'import weblate; import os; print(os.path.dirname(weblate.__file__))')

Expand All @@ -92,13 +92,11 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
- name: Install pip dependencies
- name: Install Python dependencies
run: |
python -m pip install $(sed -n 's/.*"\(pip==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(wheel==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(twine==\([^"]*\)\)".*/\1/p' pyproject.toml)
pip install $(sed -n 's/.*"\(build==\([^"]*\)\)".*/\1/p' pyproject.toml)
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(twine==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system $(sed -n 's/.*"\(build==\([^"]*\)\)".*/\1/p' pyproject.toml)
- name: build
run: python -m build
- name: Twine check
Expand All @@ -122,14 +120,14 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: pip
cache-dependency-path: docs/requirements.txt
- name: Install apt dependencies
run: |
sudo apt update
sudo apt install -y graphviz pandoc
- name: Install pip dependencies
run: pip install -r docs/requirements.txt
- name: Install Python dependencies
run: |
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)
uv pip install --system -r docs/requirements.txt
- name: Sphinx build
run: ./ci/run-docs
- name: Convert release notes
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,10 +68,9 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip
- name: Used versions
run: ./ci/print-versions
- name: Install pip dependencies
- name: Install Python dependencies
run: ./ci/pip-install ${{ matrix.requirements }}
- name: Prepare database
run: ./ci/prepare-database
Expand Down
24 changes: 11 additions & 13 deletions ci/pip-install
Original file line number Diff line number Diff line change
Expand Up @@ -8,36 +8,34 @@ set -e -x

# Install CI requirements
# shellcheck disable=SC2046
python -m pip install $(sed -n 's/.*"\(pip==\([^"]*\)\)".*/\1/p' pyproject.toml)
# shellcheck disable=SC2046
pip install $(sed -n 's/.*"\(wheel==\([^"]*\)\)".*/\1/p' pyproject.toml)
python -m pip install $(sed -n 's/.*"\(uv==\([^"]*\)\)".*/\1/p' pyproject.toml)

# Workaround for
# SystemError: ffi_prep_closure(): bad user_data (it seems that the version of the libffi library seen at runtime is different from the 'ffi.h' file seen at compile-time)
pip install --no-binary :all: cffi
uv pip install --system --no-binary :all: cffi

if [ "${1:-latest}" = migrations ] ; then
pip install -e ".[all,mysql,ci]"
uv pip install --system -e ".[all,mysql,ci]"
else
if [ "${1:-latest}" = minimal ] ; then
# Adjust deps to force minimal version
sed -i '/^ *"/ s/>=/==/' pyproject.toml
fi

if [ "${1:-latest}" = mypy ] ; then
pip install -e ".[all,mysql,ci,mypy,test]"
uv pip install --system -e ".[all,mysql,ci,mypy,test]"
else
pip install -e ".[all,mysql,ci,test]"
uv pip install --system -e ".[all,mysql,ci,test]"
fi
if [ "${1:-latest}" = edge ] ; then
pip install --no-binary=lxml --upgrade-strategy eager -U -e ".[all,mysql,test,ci]"
uv pip install --system --no-binary=lxml --upgrade -e ".[all,mysql,test,ci]"
# Install from git / pre-release
pip install --no-deps --upgrade --force-reinstall https://github.com/translate/translate/archive/master.zip
pip install --no-deps --upgrade --force-reinstall https://github.com/WeblateOrg/language-data/archive/main.zip
pip install --no-deps --upgrade --force-reinstall https://github.com/WeblateOrg/translation-finder/archive/main.zip
pip install --upgrade --pre Django
uv pip install --system --no-deps --upgrade --force-reinstall https://github.com/translate/translate/archive/master.zip
uv pip install --system --no-deps --upgrade --force-reinstall https://github.com/WeblateOrg/language-data/archive/main.zip
uv pip install --system --no-deps --upgrade --force-reinstall https://github.com/WeblateOrg/translation-finder/archive/main.zip
uv pip install --system --upgrade --pre Django
fi
fi

# Verify that deps are consistent
pip check
uv pip check
8 changes: 4 additions & 4 deletions ci/run-migrate
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@ fi
check
git checkout "$TAG"
check
# Use clean virtualenv for each version, this avoids pip problems when trying to downgrade from current versions
virtualenv --python python3.11 ".venv-$TAG"
# Use clean virtualenv for each version, this avoids problems when trying to downgrade from current versions
uv venv --python python3.11 ".venv-$TAG"
check
# shellcheck source=/dev/null
. ".venv-$TAG/bin/activate"
# Workaround for SystemError: ffi_prep_closure(): bad user_data
pip install --no-binary :all: cffi
pip install -e ".[all,mysql]"
uv pip install --no-binary :all: cffi
uv pip install -e ".[all,mysql]"
check
echo "DATABASES['default']['HOST'] = '$CI_DB_HOST'" >> weblate/settings_test.py
check
Expand Down
7 changes: 2 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -113,11 +113,8 @@ antispam = [
ci = [
"build==1.2.1",
"coverage==7.5.1",
"pip==24.0",
"setuptools==69.5.1",
"twine==5.1.0",
"virtualenv==20.26.2",
"wheel==0.43.0"
"uv==0.1.45",
"twine==5.1.0"
]
dev = [
"Weblate[all,ci,test,lint,mypy]",
Expand Down
Loading