Merge the Python3 Refactor code. Almost a complete re-write of non-co…

…re functionality.

This branch has been active on GitHub a year in the making... so there's probably too many changes to review based on commits (128 total commits on branch).

Read COMPATIBILITY.txt and browse the code to understand the full extent of the changes.  Documentation updates are coming soon.

Merge branch 'v0.5.x-py3-refactor'

.github/workflows/lint-and-test.yml

@@ -9,7 +9,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.7, 3.8, 3.9]
+        python-version: [3.6, 3.7, 3.8, 3.9]
 
     steps:
     - uses: actions/checkout@v2
@@ -22,7 +22,28 @@ jobs:
         python -m pip install --upgrade pip
         python -m pip install flake8 pytest
         if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+
+        # latest version not yet available on Ubuntu Focal 20.04 LTS, but it's better to install it with all dependencies first
         sudo apt-get install -qq libimage-exiftool-perl
+        exiftool -ver
+
+        # get just the minimum version to build and compile, later we can go with latest version to test
+        export EXIFTOOL_VER=12.15
+        wget http://backpan.perl.org/authors/id/E/EX/EXIFTOOL/Image-ExifTool-$EXIFTOOL_VER.tar.gz
+        tar xf Image-ExifTool-$EXIFTOOL_VER.tar.gz
+        cd Image-ExifTool-$EXIFTOOL_VER/
+
+        # https://exiftool.org/install.html#Unix
+        perl Makefile.PL
+        make test
+
+        export PATH=`pwd`:$PATH
+        cd ..
+        exiftool -ver
+
+        # save this environment for subsequent steps
+        # https://brandur.org/fragments/github-actions-env-vars-in-env-vars
+        echo "PATH=`pwd`:$PATH" >> $GITHUB_ENV
     - name: Install pyexiftool
       run: |
         python -m pip install .

.gitignore

@@ -5,3 +5,12 @@ dist/
 MANIFEST
 
 *.egg-info/
+
+# pytest-cov db
+.coverage
+
+# tests will be made to write to temp directories with this prefix
+tests/exiftool-tmp-*
+
+# IntelliJ
+.idea

CHANGELOG.md

@@ -3,7 +3,7 @@
 Date (Timezone)              | Version | Comment
 ---------------------------- | ------- | -------
 07/17/2019 12:26:16 AM (PDT) | 0.2.0   | Source was pulled directly from https://github.com/smarnach/pyexiftool with a complete bare clone to preserve all history.  Because it's no longer being updated, I will pull all merge requests in and make updates accordingly
-07/17/2019 12:50:20 AM (PDT) | 0.2.1   | Convert leading spaces to tabs
+07/17/2019 12:50:20 AM (PDT) | 0.2.1   | Convert leading spaces to tabs.  (I'm aware of [PEP 8](https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces) recommending spaces over tabs, but I <3 tabs)
 07/17/2019 12:52:33 AM (PDT) | 0.2.2   | Merge [Pull request #10 "add copy_tags method"](https://github.com/smarnach/pyexiftool/pull/10) by [Maik Riechert (letmaik) Cambridge, UK](https://github.com/letmaik) on May 28, 2014<br> *This adds a small convenience method to copy any tags from one file to another. I use it for several month now and it works fine for me.*
 07/17/2019 01:05:37 AM (PDT) | 0.2.3   | Merge [Pull request #25 "Added option for keeping print conversion active. #25"](https://github.com/smarnach/pyexiftool/pull/25) by [Bernhard Bliem (bbliem)](https://github.com/bbliem) on Jan 17, 2019<br> *For some tags, disabling print conversion (as was the default before) would not make much sense. For example, if print conversion is deactivated, the value of the Composite:LensID tag could be reported as something like "8D 44 5C 8E 34 3C 8F 0E". It is doubtful whether this is useful here, as we would then need to look up what this means in a table supplied with exiftool. We would probably like the human-readable value, which is in this case "AF-S DX Zoom-Nikkor 18-70mm f/3.5-4.5G IF-ED".*<br>*Disabling print conversion makes sense for a lot of tags (e.g., it's nicer to get as the exposure time not the string "1/2" but the number 0.5). In such cases, even if we enable print conversion, we can disable it for individual tags by appending a # symbol to the tag name.*
 07/17/2019 01:20:15 AM (PDT) | 0.2.4   | Merge with slight modifications to variable names for clarity (sylikc) [Pull request #27 "Add "shell" keyword argument to ExifTool initialization"](https://github.com/smarnach/pyexiftool/pull/27) by [Douglas Lassance (douglaslassance) Los Angeles, CA](https://github.com/douglaslassance) on 5/29/2019<br>*On Windows this will allow to run exiftool without showing the DOS shell.*<br>**This might break Linux but I don't know for sure**<br>Alternative source location with only this patch: https://github.com/blurstudio/pyexiftool/tree/shell-option
@@ -35,18 +35,22 @@ Date (Timezone)              | Version | Comment
 08/22/2021 08:34:45 PM (PDT) | 0.4.11  | no functional code changes.  Changed setup.py with updated version and Documentation link pointed to sylikc.github.io -- as per issue #27 by @derMart
 08/22/2021 09:02:33 PM (PDT) | 0.4.12  | fixed a bug ExifTool.terminate() where there was a typo.  Kept the unused outs, errs though. -- from suggestion in pull request #26 by @aaronkollasch
 02/13/2022 03:38:45 PM (PST) | 0.4.13  | (NOTE: Barring any critical bug, this is expected to be the LAST Python 2 supported release!)  added GitHub actions.  fixed bug in execute_json_wrapper() 'error' was not defined syntactically properly -- merged pull request #30 by https://github.com/jangop
+03/13/2021 01:54:44 PM (PST) | 0.5.0a0 | no functional code changes ... yet.  this is currently on a separate branch referring to [Break down Exiftool into 2+ classes, a raw Exiftool, and helper classes](https://github.com/sylikc/pyexiftool/discussions/10) and [Deprecating Python 2.x compatibility](https://github.com/sylikc/pyexiftool/discussions/9) .  In time this refactor will be the future of PyExifTool, once it stabilizes.  I'll make code-breaking updates in this branch from build to build and take comments to make improvements.  Consider the 0.5.0 "nightly" quality.  Also, changelog versions were modified because I noticed that the LAST release from smarnach is tagged with v0.2.0
+02/28/2022 12:39:57 PM (PST) | 0.5.0   | complete refactor of the PyExifTool code.  Lots of changes.  Some code breaking changes.  Not directly backwards-compatible with v0.4.x.  See COMPATIBILITY.TXT to understand all the code-breaking changes.
 
 
-On version changes, update setup.py to reflect version
+On version changes, update __int__.py to reflect version
+
 
 # Changes around the web
 
-Check for changes at the following resources to make sure we have the latest and greatest.  While we have the most active fork, I'm just one of the many forks, spoons, and knives!
+Check for changes at the following resources to see if anyone has added some nifty features.  While we have the most active fork, I'm just one of the many forks, spoons, and knives!
+
+We can also direct users here or answer existing questions as to how to use the original version of ExifTool.
 
-(last checked 5/19/2021 all)
+(last checked 2/28/2022 all)
 
 search "pyexiftool github" to see if you find any more random ports/forks
 check for updates https://github.com/smarnach/pyexiftool/pulls
-check for updates on elodie https://github.com/jmathai/elodie/commits/master/elodie/external/pyexiftool.py
 check for new open issues https://github.com/smarnach/pyexiftool/issues?q=is%3Aissue+is%3Aopen
 

COMPATIBILITY.txt

@@ -0,0 +1,55 @@
+PyExifTool does not guarantee source-level compatibility from one release to the next.
+
+That said, efforts will be made to provide well-documented API-level compatibility,
+and if there are major API changes, migration documentation will be provided, when
+possible.
+
+----
+
+v0.1.x - v0.2.0  = smarnach code, API compatible
+v0.2.1 - v0.4.13 = code with all PRs, a superset of functionality on Exiftool class
+v0.5.0 -         = not API compatible with the v0.4.x series.  See comments below:
+
+
+----
+API changes between v0.4.x and v0.5.0:
+
+	PYTHON CHANGE: Old: Python 2.6 supported.  New: Python 3.6+ required
+
+	CHANGED: Exiftool constructor:
+		RENAME:   "executable_" parameter to "executable"
+		DEFAULT BEHAVIOR: "common_args" defaults to ["-G", "-n"] instead of None.  Old behavior set -G and -n if "common_args" is None.  New behavior "common_args" = [] if common_args is None.
+		DEFAULT: Old: "win_shell" defaults to True.  New: "win_shell" defaults to False.
+		NEW: "encoding" parameter
+		NEW: "logger" parameter
+
+	NEW PROPERTY GET/SET: a lot of properties were added to do get/set validation, and parameters can be changed outside of the constructor.
+
+	METHOD RENAME: starting the process was renamed from "start" to "run"
+
+	MINIMUM TOOL VERSION: exiftool command line utility minimum requirements.  Old: 8.60.  New: 12.15
+
+	ENCODING CHANGE: execute() and execute_json() no longer take bytes, but is guided by the encoding set in constructor/property
+
+	ERROR CHANGE: execute_json() when no json was not returned (such as a set metadata operation) => Old: raised an error.  New: returns custom ExifToolException
+
+	FEATURE REMOVAL: execute_json() no longer detects the '-w' flag being passed used in common_args.
+		If a user uses this flag, expect no output.
+		(detection in common_args was clunky anyways because -w can be passed as a per-run param for the same effect)
+
+
+	all methods other than execute() and execute_json() moved to ExifToolHelper or ExifToolAlpha class.
+
+	ExifToolHelper adds methods:
+		get_metadata()
+		get_tags()
+
+		NEW CONVENTION: all methods take "files" first, "tags" second (if needed) and "params" last
+
+
+	ExifToolAlpha adds all remaining methods in an alpha-quality way
+
+	NOTE: ExifToolAlpha has not been updated yet to use the new convention, and the edge case code may be removed/changed at any time.
+		If you depend on functionality provided by ExifToolAlpha, please submit an Issue to start a discussion on cleaning up the code and moving it into ExifToolHelper
+----
+

LICENSE

@@ -1,6 +1,6 @@
 PyExifTool <http://github.com/sylikc/pyexiftool>
 
-Copyright 2012 Sven Marnach, Kevin M (sylikc)
+Copyright 2012-2014 Sven Marnach, 2019-2022 Kevin M (sylikc)
 
 PyExifTool is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by

README.rst

@@ -1,5 +1,6 @@
+**********
 PyExifTool
-==========
+**********
 
 PyExifTool is a Python library to communicate with an instance of Phil
 Harvey's excellent ExifTool_ command-line application.  The library
@@ -11,70 +12,200 @@ single instance needs to be launched and can be reused for many
 queries.  This is much more efficient than launching a separate
 process for every single query.
 
-.. _ExifTool: http://www.sno.phy.queensu.ca/~phil/exiftool/
+.. _ExifTool: https://exiftool.org/
+
+Example Usage
+=============
+
+.. code-block:: python
+	:caption: Quick Sample
+	:linenos:
+
+	import exiftool
+
+	files = ["a.jpg", "b.png", "c.tif"]
+	with exiftool.ExifToolHelper() as et:
+		metadata = et.get_metadata(files)
+	for d in metadata:
+		print("{:20.20} {:20.20}".format(d["SourceFile"],
+										 d["EXIF:DateTimeOriginal"]))
+
 
 Getting PyExifTool
-------------------
+==================
+
+PyPI
+------------
+
+Easiest: Install a version from the official `PyExifTool PyPI`_
+
+::
+
+    python -m pip install -U pyexiftool
+
+.. _PyExifTool PyPI: https://pypi.org/project/PyExifTool/
+
+
+From Source
+------------
 
 The source code can be checked out from the github repository with
 
 ::
 
     git clone git://github.com/sylikc/pyexiftool.git
 
-Alternatively, you can download a tarball_.  
+Alternatively, you can download a tarball_.
+
+.. _tarball: https://github.com/sylikc/pyexiftool/tarball/master
 
-Official releases are on PyPI
+Run
 
 ::
 
-    https://pypi.org/project/PyExifTool/
+    python setup.py install [--user|--prefix=<installation-prefix]
 
+to automatically install this module from source.
 
-.. _tarball: https://github.com/sylikc/pyexiftool/tarball/master
 
-Installation
-------------
+PyExifTool Dependencies
+=======================
+
+Python
+------
+
+PyExifTool runs on **Python 3.6+**.  (If you need Python 2.6 support,
+please use version v0.4.x).  PyExifTool has been tested on Windows and
+Linux, and probably also runs on other Unix-like platforms.
 
-PyExifTool runs on Python 2.6 and above, including 3.x.  It has been
-tested on Windows and Linux, and probably also runs on other Unix-like
-platforms.
+Phil Harvey's exiftool
+----------------------
 
-You need an installation of the ``exiftool`` command-line tool.  The
-code has been tested with version 8.60, but should work with version
-8.40 or above (which was the first production version of exiftool
-featuring the ``-stay_open`` option for batch mode).
+For PyExifTool to function, ``exiftool`` command-line tool must exist on
+the system.  If ``exiftool`` is not on the ``PATH``, you can specify the full
+pathname to it by using ``ExifTool(executable=<full path>)``.
 
-PyExifTool currently only consists of a single module, so you can
-simply copy or link this module to a place where Python finds it, or
-you can call
+PyExifTool requires a **minimum version of 12.15** (which was the first
+production version of exiftool featuring the options to allow exit status
+checks used in conjuction with ``-echo3`` and ``-echo4`` parameters).
 
-(It is being slowly re-factored to a module to keep exiftool.py from 
-growing astronomically)
+To check your ``exiftool`` version:
 
 ::
 
-    python setup.py install [--user|--prefix=<installation-prefix]
+    exiftool -ver
 
-to automatically install that module.
 
-Testing
--------------
+Windows/Mac
+^^^^^^^^^^^
 
-Run tests to make sure it's functional
+Windows/Mac users can download the latest version of exiftool:
 
 ::
 
-    python -m unittest -v tests/test_exiftool.py
+    https://exiftool.org
+
+Linux
+^^^^^
+
+Most current Linux distributions have a package which will install ``exiftool``.
+Unfortunately, some do not have the minimum required version, in which case you
+will have to `build from source`_.
+
+* Ubuntu
+  ::
+
+    sudo apt install libimage-exiftool-perl
+
+* CentOS/RHEL
+  ::
+
+    yum install perl-Image-ExifTool
+
+.. _build from source: https://exiftool.org/install.html#Unix
+
 
 Documentation
--------------
+=============
+
+The documentation is available at `sylikc.github.io`_.
+It is slightly outdated at the moment but will be improved as the
+project moves forward
+
+::
+
+    http://sylikc.github.io/pyexiftool/
+
+.. _sylikc.github.io: http://sylikc.github.io/pyexiftool/
+
+
+Package Structure
+-----------------
+
+PyExifTool consists of a few modules, each with increasingly more features.
+
+The base ``ExifTool`` class is the most rudimentary, and each successive class
+inherits and adds functionality.
+
+* ``ExifTool`` is the base class with functionality which will not likely change.
+  It contains the core features with no extra fluff.  The API is considered stable
+  and should not change much with new versions.
+
+* ``ExifToolHelper`` adds the most commonly used functionality.  It overloads
+  some functions to turn common errors into warnings or makes checks to make
+  ``ExifTool`` easier to use.  More methods may be added or slight tweaks may
+  come with new versions.
+
+* ``ExifToolAlpha`` includes some of the community functionality that contributors
+  added for edge use cases.  It is *not* up to the rigorous testing standard of both
+  ``ExifTool`` or ``ExifToolHelper``.  There may be old or defunct code at any time.
+  This is the least polished of the classes and functionality/API may be
+  changed/added/removed at any time.
+
+
+Brief History
+=============
+
+PyExifTool was originally developed by `Sven Marnach`_ in 2012 to answer a
+stackoverflow question `Call exiftool from a python script?`_.  Over time,
+Sven refined the code, added tests, documentation, and a slew of improvements.
+While PyExifTool gained popularity, Sven `never intended to maintain it`_ as
+an active project.  The `original repository`_ was last updated in 2014.
+
+In early 2019, `Martin Čarnogurský`_ created a `PyPI release`_ from the
+2014 code.  Coincidentally in mid 2019, `Kevin M (sylikc)`_ forked the original
+repository and started merging PR and issues which were reported on Sven's
+issues/PR page.
+
+In late 2019 and early 2020 there was a discussion started to
+`Provide visibility for an active fork`_.  There was a conversation to
+transfer ownership of the original repository, have a coordinated plan to
+communicate to PyExifTool users, amongst other things, but it never materialized.
+
+Kevin M (sylikc) made the first release to PyPI repository in early 2021.
+At the same time, discussions were starting revolving around
+`Deprecating Python 2.x compatibility`_ and `refactoring the code and classes`_.
+
+The latest version is the result of all of those discussions, designs,
+and development.  Special thanks to the community contributions, especially
+`Jan Philip Göpfert`_, `Seth P`_, and `Kolen Cheung`_.
 
-The documentation is available at
-http://sylikc.github.io/pyexiftool/.
+.. _Sven Marnach: https://github.com/smarnach/pyexiftool
+.. _Call exiftool from a python script?: https://stackoverflow.com/questions/10075115/call-exiftool-from-a-python-script/10075210#10075210
+.. _never intended to maintain it: https://github.com/smarnach/pyexiftool/pull/31#issuecomment-569238073
+.. _original repository: https://github.com/smarnach/pyexiftool
+.. _Martin Čarnogurský: https://github.com/RootLUG
+.. _PyPI release: https://pypi.org/project/PyExifTool/0.1.1/#history
+.. _Kevin M (sylikc): https://github.com/sylikc
+.. _Provide visibility for an active fork: https://github.com/smarnach/pyexiftool/pull/31
+.. _Deprecating Python 2.x compatibility: https://github.com/sylikc/pyexiftool/discussions/9
+.. _refactoring the code and classes: https://github.com/sylikc/pyexiftool/discussions/10
+.. _Jan Philip Göpfert: https://github.com/jangop
+.. _Seth P: https://github.com/csparker247
+.. _Kolen Cheung: https://github.com/ickc
 
 Licence
--------
+=======
 
 PyExifTool is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
@@ -85,4 +216,4 @@ PyExifTool is distributed in the hope that it will be useful,
 but WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
-See COPYING.GPL or COPYING.BSD for more details.
+See ``LICENSE`` for more details.