Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pip cache dir #1646

Closed
hugovk opened this issue Feb 18, 2024 · 6 comments · Fixed by #1734
Closed

Add pip cache dir #1646

hugovk opened this issue Feb 18, 2024 · 6 comments · Fixed by #1734
Assignees
Labels
compatibility Compatibility with a specification or another tool enhancement New feature or improvement to existing functionality

Comments

@hugovk
Copy link

hugovk commented Feb 18, 2024

Short version

Add a cross-platform way to find out the uv cache directory (like pip has) for use on GitHub Actions.

Long version

If we want to cache things installed by uv pip install on GitHub Actions, we need to do something like this:

- uses: actions/cache@v4
  if: startsWith(runner.os, 'Linux')
  with:
    path: ~/.cache/uv
    key: ${{ runner.os }}-uv-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-uv-

- uses: actions/cache@v4
  if: startsWith(runner.os, 'macOS')
  with:
    path: ~/Library/Caches/uv
    key: ${{ runner.os }}-uv-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-uv-

- uses: actions/cache@v4
  if: startsWith(runner.os, 'Windows')
  with:
    path: ~\AppData\Local\uv\Cache
    key: ${{ runner.os }}-uv-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-uv-

Adapted from https://github.com/actions/cache/blob/main/examples.md#multiple-oss-in-a-workflow

Or:

jobs:
  build:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
        include:
        - os: ubuntu-latest
          path: ~/.cache/uv
        - os: macos-latest
          path: ~/Library/Caches/uv
        - os: windows-latest
          path: ~\AppData\Local\uv\Cache
    steps:
    - uses: actions/cache@v4
      with:
        path: ${{ matrix.path }}
        key: ${{ runner.os }}-uv-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
         ${{ runner.os }}-uv-

Adapted from https://github.com/actions/cache/blob/main/examples.md#multiple-oss-in-a-workflow-with-a-matrix

Or we may be able to choose a common directory using the --cache-dir flag or UV_CACHE_DIR environment variable.

Originally we had to do the same thing with pip (hence the examples at actions/cache).

But then pip 20.1 added a pip cache dir command (pypa/pip#7350 / pypa/pip#8095):

pip cache dir
/Users/hugo/Library/Caches/pip

If we add uv pip cache dir, we would similarly be able to simplify the config:

- name: Get uv cache dir
  id: uv-cache
  run: |
    echo "dir=$(uv cache dir)" >> $GITHUB_OUTPUT

- name: pip cache
  uses: actions/cache@v4
  with:
    path: ${{ steps.pip-cache.outputs.dir }}
    key: ${{ runner.os }}-uv-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-uv-

Adapted from https://github.com/actions/cache/blob/main/examples.md#using-pip-to-get-cache-location

@charliermarsh
Copy link
Member

Yeah we should add this, it's trivial.

@charliermarsh charliermarsh added enhancement New feature or improvement to existing functionality compatibility Compatibility with a specification or another tool labels Feb 18, 2024
@charliermarsh
Copy link
Member

As part of this, I'll probably move uv clean to uv cache clean, so we can centralize the cache commands.

@hugovk
Copy link
Author

hugovk commented Feb 18, 2024

Out of scope for this issue, could be useful to add some other pip cache subcommands:

pip cache --help

Usage:
  pip3 cache dir
  pip3 cache info
  pip3 cache list [<pattern>] [--format=[human, abspath]]
  pip3 cache remove <pattern>
  pip3 cache purge


Description:
  Inspect and manage pip's wheel cache.

  Subcommands:

  - dir: Show the cache directory.
  - info: Show information about the cache.
  - list: List filenames of packages stored in the cache.
  - remove: Remove one or more package from the cache.
  - purge: Remove all items from the cache.

  ``<pattern>`` can be a glob expression or a package name.

Cache Options:
  --format <list_format>      Select the output format among: human (default) or abspath

Especially pip cache info, because we will end up with large pip and uv caches on some systems:

pip cache info
Package index page cache location (pip v23.3+): /Users/hugo/Library/Caches/pip/http-v2
Package index page cache location (older pips): /Users/hugo/Library/Caches/pip/http
Package index page cache size: 187.6 MB
Number of HTTP files: 870
Locally built wheels location: /Users/hugo/Library/Caches/pip/wheels
Locally built wheels size: 5.9 MB
Number of locally built wheels: 10

Shall I open a new issue for pip cache info?

@charliermarsh
Copy link
Member

Yeah feel free to create a separate issue for pip cache info. That looks useful. (I don't think we'd attempt to mimic the exact same output (in terms of format, etc.), but we could report the same information.)

@hugovk
Copy link
Author

hugovk commented Feb 18, 2024

Thanks, I created #1655 and sneaked pip cache list in there too.

@akx
Copy link
Contributor

akx commented Feb 19, 2024

Ah, heh, related 😁 actions/setup-python#818

@charliermarsh charliermarsh self-assigned this Feb 20, 2024
charliermarsh added a commit that referenced this issue Feb 20, 2024
## Summary

Like `pip cache dir`, merely prints out the cache directory.

Closes #1646.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compatibility Compatibility with a specification or another tool enhancement New feature or improvement to existing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants