Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 2.6 and 3.x support fixes and enhanced versions testing #73

Closed
wants to merge 9 commits into from
Closed

Python 2.6 and 3.x support fixes and enhanced versions testing #73

wants to merge 9 commits into from

Conversation

dupuy
Copy link

@dupuy dupuy commented Jul 3, 2013

This pull request is mostly a superset of Bernardo Heynemann's PR #70 (I omitted the last two commits in that PR as they seemed off-topic, worthwhile as they might otherwise be).

In addition to his fixes for 2.6 and py3k bugs, using the excellent six compatibility library, I enhanced the testing setup for slumber, updating the .travis.yml configuration to add testing for all the Python versions (except 2.5) that it supports, and as you can see from the Travis report on this PR, all tests are passing on all those versions. I also made some tweaks to the tox.ini he created, so that it doesn't install tox itself (and all of its dependencies) in the virtualenvs that it creates for testing slumber on different Python versions, and so that it tests Python 3.1 if available (Travis no longer supports Python 3.1, so it's commented out in .travis.yml).

I also fixed some other Py3K issues that I ran into while using slumber (before I saw his PR). In particular:

  • Use of iteritems() method in several places was not compatible with py3k; I used six.iteritems() function, which is defined appropriately for Python 2 or 3.
  • The url_join() function did not properly concatenate path components passed as bytes in py3k - they would be formatted as /b'url-path-comp'/ rather than as /url-path-comp/. Passing them as bytearray made it worse, as they would then be formatted as /bytearray(b'url-path-comp')/! There were also some latent issues with encoding compatibility and mixing Unicode and bytes, where urlsplit or urlunsplit could raise UnicodeDecodeError exceptions, and other cases where users could pass in bytes data with incompatible encodings that would not raise exceptions.

For the second issue, I added several cases to the test suite for url_join, including checks that TypeError exceptions are raised for potentially incompatibly encoded data passed to url_join (if you need non-ASCII data in the extra arguments, you need to pass them (and the base URL) as Unicode, not bytes).

In @merwok's comment on PR #70, issue #56 is mentioned, specifically the possibility of syncing tox.ini and .travis.yml using the panci tool. I installed panci and ran it against those files in this PR, but I found the output to be somewhat inferior to the versions of those files in this PR; the resulting files lose functionality and/or cause test failures, and I didn't feel like fixing panci too.

--- .travis.yml 2013-06-05 02:16:45 -0400
+++ tox-to-travis.yml   2013-07-03 01:16:43 -0400
@@ -1,13 +1,14 @@
 language: python
 python:
-  - 2.6
-  - 2.7
-  # 3.1 # Travis no longer supports Python 3.1
-  - 3.2
-  - 3.3
+- '2.6'
+- '2.7'
+- '3.1'
+- '3.2'
+- '3.3'
   - pypy
-install:
-  - pip install -r requirements.txt && pip install -r requirements-test.txt
-  - case `python -V 2>&1` in *2.6*) pip install unittest2;; esac
-script:
-  - python setup.py test
+script: '
+
+  pip install -r requirements.txt
+
+  python setup.py test'
+
--- tox.ini     2013-07-03 00:32:56 -0400
+++ travis-to-tox.ini   2013-07-03 01:16:52 -0400
@@ -4,16 +4,11 @@
 # and then run "tox" from this directory.

 [tox]
-envlist = py26, py27, py31, py32, py33, pypy
+envlist = py26, py27, py32, py33, pypy

 [testenv]
 commands =
-    pip install -r requirements.txt
+    pip install -r requirements.txt && pip install -r requirements-test.txt
+    case `python -V 2>&1` in *2.6*) pip install unittest2;; esac
     python setup.py test
-deps =
-    mock

-[testenv:py26]
-deps =
-    mock
-    unittest2

If you really want to generate one from the other, generate tox.ini from .travis.yml since the only lost functionality is Python 3.1 testing, and the useless installation of tox and its dependencies inside the virtualenvs for testing slumber (which I fixed in d534651) is annoying but harmless, and I don't think you'll get any test failures.

heynemann and others added 9 commits May 2, 2013 17:31
This commit adds python 2.6-3.3 support (as well as tests through tox) to slumber.

Some additional dependencies were introduced, though.
This commit adds python 2.6-3.3 support (as well as tests through tox) to slumber.

Some additional dependencies were introduced, though. Six was added to support conditional treatment of strings.

A test dependency to unittest2 was added to tox in order to add support to the "discover" method of TestCase (though this dependency only happens in python26).

All tests are passing in the following python versions: 2.6, 2.7, 3.2, 3.3 and pypy.
To allow joining args of different types, url_join() uses '%s' % arg
to format the arguments; while in Python 2 this works, in Python 3
bytearray and bytes arguments are formatted as /bytearray(b'abc')/
and /b'arg'/ rather than /arg/, so more sophisticated code is needed.

Mixing Unicode and byte data is risky as you don't know the encoding
of the bytes arguments, but as long as args are ASCII (or non-ASCII
Unicode data) it is okay.  We can even allow non-ASCII bytes in the
base URL if we specify that the returned joined URL has the same type
(Unicode or bytes) and in the latter case, encoding, as the base URL.

To allow a base URL to have non-ASCII bytes, we have to play encoding
tricks on urlsplit and urlunsplit, "decoding" bytes as latin1 before
passing the resulting "Unicode" to the function, and then re-encoding
the result(s) as latin1 to reverse the transformation.

A TypeError is raised if a non-ASCII bytes argument is passed, or a
non-ASCII Unicode argument is passed with a bytes base URL.  This is
by analogy with posixpath.join(), which doesn't care about or check
for non-ASCII, but rejects mixed strings and bytes with a TypeError.
And test joining mixed encoded and decoded unicode as well, actually
passing bytes and bytearray arguments for encoded unicode strings.

And test that non-ASCII args requiring conversion, or non-ASCII bytes
args, raise a TypeError.
Although Travis no longer supports Python 3.1, slumber should support it
(with 'six' compatibility) so there's no need to omit it from tox.ini file.
@cclauss
Copy link

cclauss commented Feb 20, 2019

Can this issue be closed?

@dupuy dupuy closed this by deleting the head repository Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants