Merge branch 'release/4.1.x'

* release/4.1.x:
  Bump version to 4.1.0
  Fix python 2 compatibility
  Ensure Python 2.7 compatibility
  Import force/smart_str instead of force/smart_text
  Run coverage-report (combined coverage) on Python 3.10
  Add myself to the list of contributors
  Add more Python versions to the classifiers
  Remove PIL from the installation instructions
  Fix spelling / rst errors
  Use GitHub actions workflow badge
  Minor improvements to the tests
  Add coverage reporting
  Add Python 3.10 to the test matrix
  Switch from django-nose to pytest-django
  Remove Python 3.10 from the test matrix because nose doesn't work
  Upgrade pip before installing other dependencies
  Django master branch has been renamed to main at some point
  Use tox-gh-actions
  Workaround SuspiciousFileOperation
  Remove nose-progressive, it doesn't seem to work with recent Python versions
  Only test on currently supported versions of Python and Django
  Switch to GitHub Actions
  chore: Set django-dramatiq min version
  chore: Add 'async_dramatiq' extra, django-dramatiq dependency
  feat: Add Dramatiq backend
  Accept cachefile_storage that is callable
  Add support of celery 5.*
  Use gettext_lazy in Django 2.0+ (ugettext_lazy is deprecated)
  Fix deprecation warning regarding invalid escape sequences.
  docs: Fix simple typo, assigment -> assignment
  fix broken links in documentation (#319)
  Pass features to BeautifulSoup constructor
  Do not check for existence if name is None
  Python 3.6 and Django 2.1
  Fix pickle serialization for ImageCacheFile
  Enhance condition in _get_size (#463)
  Test against Django 2.0

.coveragerc

@@ -0,0 +1,9 @@
+[run]
+branch = true
+parallel = true
+
+[report]
+show_missing = true
+skip_empty = true
+skip_covered = true
+precision = 2

.github/workflows/python.yml

@@ -0,0 +1,31 @@
+name: Python CI
+
+on:
+  push:
+    branches: [ develop ]
+  pull_request:
+    branches: [ develop ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      max-parallel: 4
+      matrix:
+        python-version: ['2.7', '3.6', '3.7', '3.8', '3.9', '3.10']
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install tox tox-gh-actions
+
+    - name: Run tests
+      run: tox

AUTHORS

@@ -29,6 +29,7 @@ Contributors
 * `Sean Bell`_
 * `Saul Shanabrook`_
 * `Venelin Stoykov`_
+* `Jaap Roes`_
 
 .. _Justin Driscoll: http://github.com/jdriscoll
 .. _HZDG: http://hzdg.com
@@ -51,3 +52,4 @@ Contributors
 .. _Sean Bell: https://github.com/seanbell
 .. _Saul Shanabrook: https://github.com/saulshanabrook
 .. _Venelin Stoykov: https://github.com/vstoykov
+.. Jaap Roes: https://github.com/jaap3

CONTRIBUTING.rst

@@ -11,7 +11,7 @@ contributions merged as quickly as possible:
    tests, first install tox, ``pip install tox``, then use ``tox``. This will let you know about any errors or style
    issues.
 4. While we're talking about tests, creating new ones for your code makes it
-   much easier for us to merge your code quickly. ImageKit uses nose_, so
+   much easier for us to merge your code quickly. ImageKit uses pytest_, so
    writing tests is painless. Check out `ours`__ for examples.
 5. It's a good idea to do your work in a branch; that way, you can work on more
    than one contribution at a time without making them interdependent.
@@ -20,5 +20,5 @@ contributions merged as quickly as possible:
 __ http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
 __ https://groups.google.com/forum/#!forum/django-imagekit
 __ irc://irc.freenode.net/imagekit
-.. _nose: https://nose.readthedocs.org/en/latest/
+.. _pytest: https://docs.pytest.org/en/latest/
 __ https://github.com/matthewwithanm/django-imagekit/tree/develop/tests

MANIFEST.in

@@ -1,6 +1,8 @@
 include AUTHORS
 include LICENSE
 include README.rst
+include pytest.ini
+include test-requirements.txt
 include testrunner.py
 include setup.cfg
 include tests/*.py

README.rst

@@ -1,11 +1,11 @@
 |Build Status|_
 
-.. |Build Status| image:: https://travis-ci.org/matthewwithanm/django-imagekit.svg?branch=develop
-.. _Build Status: https://travis-ci.org/matthewwithanm/django-imagekit
+.. |Build Status| image:: https://github.com/matthewwithanm/django-imagekit/actions/workflows/python.yml/badge.svg?branch=develop
+.. _Build Status: https://github.com/matthewwithanm/django-imagekit/actions/workflows/python.yml
 
 ImageKit is a Django app for processing images. Need a thumbnail? A
 black-and-white version of a user-uploaded image? ImageKit will make them for
-you. If you need to programatically generate one image from another, you need
+you. If you need to programmatically generate one image from another, you need
 ImageKit.
 
 ImageKit comes with a bunch of image processors for common tasks like resizing
@@ -22,23 +22,18 @@ __ https://github.com/fish2000/instakit
 Installation
 ============
 
-1. Install `PIL`_ or `Pillow`_. (If you're using an ``ImageField`` in Django,
+1. Install `Pillow`_. (If you're using an ``ImageField`` in Django,
    you should have already done this.)
 2. ``pip install django-imagekit``
-3. Add ``'imagekit'`` to your ``INSTALLED_APPS`` list in your project's settings.py
+3. Add ``'imagekit'`` to your ``INSTALLED_APPS`` list in your project's ``settings.py``
 
-.. note:: If you've never seen Pillow before, it considers itself a
-   more-frequently updated "friendly" fork of PIL that's compatible with
-   setuptools. As such, it shares the same namespace as PIL does and is a
-   drop-in replacement.
-
-.. _`PIL`: http://pypi.python.org/pypi/PIL
 .. _`Pillow`: http://pypi.python.org/pypi/Pillow
 
 
 Usage Overview
 ==============
 
+.. _specs:
 
 Specs
 -----
@@ -446,7 +441,7 @@ AdminThumbnail can even use a custom template. For more information, see
 Management Commands
 -------------------
 
-ImageKit has one management command—`generateimages`—which will generate cache
+ImageKit has one management command—``generateimages``—which will generate cache
 files for all of your registered image generators. You can also pass it a list
 of generator ids in order to generate images selectively.
 
@@ -472,9 +467,8 @@ people, open a pull request so we can take a look!
 You can also check out our list of `open, contributor-friendly issues`__ for
 ideas.
 
-Check out our `contributing guidelines`__ for more information about pitching in
+Check out our `contributing guidelines`_ for more information about pitching in
 with ImageKit.
 
-
 __ https://github.com/matthewwithanm/django-imagekit/issues?labels=contributor-friendly&state=open
-__ https://github.com/matthewwithanm/django-imagekit/blob/develop/CONTRIBUTING.rst
+.. _`contributing guidelines`: https://github.com/matthewwithanm/django-imagekit/blob/develop/CONTRIBUTING.rst

docs/caching.rst

@@ -3,7 +3,7 @@ Caching
 
 
 Default Backend Workflow
-================
+========================
 
 
 ``ImageSpec``
@@ -29,6 +29,8 @@ objects, but they've got a little trick up their sleeve: they represent files
 that may not actually exist!
 
 
+.. _cache-file-strategy:
+
 Cache File Strategy
 -------------------
 
@@ -55,6 +57,8 @@ The default strategy only defines the first two of these, as follows:
             file.generate()
 
 
+.. _cache-file-backend:
+
 Cache File Backend
 ------------------
 
@@ -185,6 +189,11 @@ Or, in Python:
             def on_source_saved(self, file):
                 file.generate()
 
+.. note::
+
+    If you use custom storage backend for some specs,
+    (storage passed to the field different than configured one)
+    it's required the storage to be pickleable
 
 
 __ https://pypi.python.org/pypi/django-celery

docs/conf.py

@@ -54,7 +54,7 @@
 # built documents.
 #
 # The short X.Y version.
-version = re.match('\d+\.\d+', pkgmeta['__version__']).group()
+version = re.match(r'\d+\.\d+', pkgmeta['__version__']).group()
 # The full version, including alpha/beta/rc tags.
 release = pkgmeta['__version__']
 

docs/upgrading.rst

@@ -79,12 +79,9 @@ IK3 provides analogous settings for cache file backends and strategies:
     IMAGEKIT_DEFAULT_CACHEFILE_BACKEND = 'path.to.MyCacheFileBackend'
     IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY = 'path.to.MyCacheFileStrategy'
 
-See the documentation on `cache file backends`_ and `cache file strategies`_
+See the documentation on :ref:`cache file backends <cache-file-backend>` and :ref:`cache file strategies <cache-file-strategy>`
 for more details.
 
-.. _`cache file backends`:
-.. _`cache file strategies`:
-
 
 Conditional model ``processors``
 --------------------------------
@@ -93,9 +90,7 @@ In IK2, an ``ImageSpecField`` could take a ``processors`` callable instead of
 an iterable, which allowed processing decisions to made based on other
 properties of the model. IK3 does away with this feature for consistency's sake
 (if one kwarg could be callable, why not all?), but provides a much more robust
-solution: the custom ``spec``. See the `advanced usage`_ documentation for more.
-
-.. _`advanced usage`:
+solution: the custom ``spec``. See the :doc:`advanced usage <advanced_usage>` documentation for more.
 
 
 Conditonal ``cache_to`` file names
@@ -109,9 +104,7 @@ There is a way to achieve custom file names by overriding your spec's
 ``cachefile_name``, but it is not recommended, as the spec's default
 behavior is to hash the combination of ``source``, ``processors``, ``format``,
 and other spec options to ensure that changes to the spec always result in
-unique file names. See the documentation on `specs`_ for more.
-
-.. _`specs`:
+unique file names. See the documentation on :ref:`specs` for more.
 
 
 Processors have moved to PILKit

imagekit/admin.py

@@ -1,4 +1,10 @@
-from django.utils.translation import ugettext_lazy as _
+from django import VERSION
+if VERSION[0] < 2:
+    # ugettext is an alias for gettext() since Django 2.0,
+    # and deprecated as of Django 3.0.
+    from django.utils.translation import ugettext_lazy as _
+else:
+    from django.utils.translation import gettext_lazy as _
 from django.template.loader import render_to_string
 
 

imagekit/cachefiles/__init__.py

@@ -22,8 +22,8 @@ def __init__(self, generator, name=None, storage=None, cachefile_backend=None, c
         """
         :param generator: The object responsible for generating a new image.
         :param name: The filename
-        :param storage: A Django storage object that will be used to save the
-            file.
+        :param storage: A Django storage object, or a callable which returns a
+            storage object that will be used to save the file.
         :param cachefile_backend: The object responsible for managing the
             state of the file.
         :param cachefile_strategy: The object responsible for handling events
@@ -40,9 +40,9 @@ def __init__(self, generator, name=None, storage=None, cachefile_backend=None, c
                 name = fn(generator)
         self.name = name
 
-        storage = storage or getattr(generator, 'cachefile_storage',
-            None) or get_singleton(settings.IMAGEKIT_DEFAULT_FILE_STORAGE,
-            'file storage backend')
+        storage = (callable(storage) and storage()) or storage or \
+            getattr(generator, 'cachefile_storage', None) or get_singleton(
+            settings.IMAGEKIT_DEFAULT_FILE_STORAGE, 'file storage backend')
         self.cachefile_backend = (
             cachefile_backend
             or getattr(generator, 'cachefile_backend', None)
@@ -144,8 +144,24 @@ def __getstate__(self):
         # file is hidden link to "file" attribute
         state.pop('_file', None)
 
+        # remove storage from state as some non-FileSystemStorage can't be
+        # pickled
+        settings_storage = get_singleton(
+            settings.IMAGEKIT_DEFAULT_FILE_STORAGE,
+            'file storage backend'
+        )
+        if state['storage'] == settings_storage:
+            state.pop('storage')
         return state
 
+    def __setstate__(self, state):
+        if 'storage' not in state:
+            state['storage'] = get_singleton(
+                settings.IMAGEKIT_DEFAULT_FILE_STORAGE,
+                'file storage backend'
+            )
+        self.__dict__.update(state)
+
     def __nonzero__(self):
         # Python 2 compatibility
         return self.__bool__()

imagekit/cachefiles/backends.py

@@ -110,7 +110,7 @@ def generate(self, file, force=False):
 
     def _exists(self, file):
         return bool(getattr(file, '_file', None)
-                    or file.storage.exists(file.name))
+                    or (file.name and file.storage.exists(file.name)))
 
 
 def _generate_file(backend, file, force=False):
@@ -140,7 +140,7 @@ def schedule_generation(self, file, force=False):
 
 
 try:
-    from celery import task
+    from celery import shared_task as task
 except ImportError:
     pass
 else:
@@ -193,3 +193,27 @@ def __init__(self, *args, **kwargs):
 
     def schedule_generation(self, file, force=False):
         _rq_job.delay(self, file, force=force)
+
+
+try:
+    from dramatiq import actor
+except ImportError:
+    pass
+else:
+    _dramatiq_actor = actor()(_generate_file)
+
+
+class Dramatiq(BaseAsync):
+    """
+    A backend that uses Dramatiq to generate the images.
+    """
+    def __init__(self, *args, **kwargs):
+        try:
+            import dramatiq  # noqa
+        except ImportError:
+            raise ImproperlyConfigured('You must install django-dramatiq to use'
+                                        ' imagekit.cachefiles.backends.Dramatiq.')
+        super(Dramatiq, self).__init__(*args, **kwargs)
+
+    def schedule_generation(self, file, force=False):
+        _dramatiq_actor.send(self, file, force=force)