Chapter 5: Modules and packages

Modules

A Python module, simply put, is a .py file. A module can contain any Python code we like. I created e.g.the Python program mCipher, which contain some encrypting/decrypting methods and can be imported. The module must be placed within PATH, best in the Lib directory.

->>> import mCipher as m
->>> m.vigenere("I have a cat","Python",1)
'XFTCSNRYM'
->>> m.vigenere('XFTCSNRYM',"Python",0)
'IHAVEACAT'
->>> 

The key difference is that programs are designed to be run, whereas modules are designed to be imported and used by programs. See - mCipher has only functions. Not all modules have associated .py files—for example, the sys module is built into Python, and some modules are written in other languages (most commonly, C). However, much of Python’s library is written in Python, so, for example, if we write import collections we can create named tuples by calling collections.namedtuple(), and the functionality we are accessing is in the collections.py module file.

->>> import os
->>> os.listdir
<built-in function listdir>
->>> os.listdir()
['01-KfN1400010001-watermarked.tif', '01-KfN1400010001.tif', '01-KfN1400010001_watermark.jpg', '01-KfN1400010001_watermark.tif', '01-KfN14B02720001.txt', '01-KfN14B02720001.xml', '12345_zmena_pixelu.py', '2cervence']
->>> list(os.walk("."))

for root, dirs, files in os.walk("."):
    path = root.split(os.sep)
    print((len(path) - 1) * '***', os.path.basename(root))
    for file in files:
        print(len(path) * '---', file)

->>> import os,mCipher
->>> type(mCipher)
<class 'module'>
->>> from mCipher import vigenere, trans_cipher

->>> import mCipher
->>> dir(mCipher)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'affine_cipher', 'attrgetter', 'caesar_cipher', 'getKey', 'itemgetter', 'm', 'makeVernamCypher', 'trans_cipher', 'vigenere']
->>> from mCipher import *
->>> 

These syntaxes can cause name conflicts since they make the imported objects (variables, functions, data types, or modules) directly accessible. The from importable import * syntax imports all the objects from the module (or all the modules from the package). For example, if we write from os.path import dirname, we can conveniently call dirname() without qualification. But if further on in our code we write dirname = “.”, the object reference dirname will now be bound to the string “.” instead of to the dirname() function, so if we try calling dirname() we will get a TypeError exception because dirname now refers to a string and strings are not callable. Bu we can write e.g.

->>> from os.path import dirname as my_dirs
->>> 

Here are a few import examples:
->>> import os
print(os.path.basename(filename)) # safe fully qualified access
->>> import os.path as path
print(path.basename(filename)) # risk of name collision with path
->>> from os import path
print(path.basename(filename)) # risk of name collision with path
->>> from os.path import basename
print(basename(filename)) # risk of name collision with basename
->>> from os.path import *
print(basename(filename)) # risk of many name collisions

How the Python knows where the module is? The built-in sys module has a list called sys.path that holds a list of the directories that constitute the Python path. Normally modules and packages are in: 1.C:\Python31\Lib - Windows

1. usr/lib - Linux

but it must be in path.

->>> import mCipher
->>> print(mCipher.__file__)
C:/JCU/IOS 655 Python a Bash/pokusy\mCipher.py
->>> 

ImportError exception

->>> import nonsense
Traceback (most recent call last):
  File "<pyshell#80>", line 1, in <module>
    import nonsense
ImportError: No module named 'nonsense'
->>> 

At every subsequent import of the module Python will detect that the module has already been imported and will do nothing.

Compiling modules

When Python needs a module’s byte-code compiled code, it generates it automatically—this differs from, say, Java, where compiling to byte code must be done explicitly. 1.First Python looks for a file with the same name but with the extension .pyo — optimized byte-code compiled version of the module. If there is no .pyo file (or if it is older than the .py file, that is, if it is out of date),

1. Python looks for a file with the extension .pyc—this is a nonoptimized byte-code compiled version of the module. If

Python finds an up-to-date byte-code compiled version of the module, it loads it;

1. otherwise, Python loads the .py file and compiles a byte-code compiled version.

Either way, Python ends up with the module in memory in byte-code compiled form. If Python had to byte-compile the .py file, it saves a .pyc version (or .pyo if -O was specified on Python’scommand line, or is set in the PYTHONOPTIMIZE environment variable), providing the directory is writable. Saving the byte code can be avoided by using the -B command line option, or by setting the PYTHONDONTWRITEBYTECODE environment variable. When Python is installed, the standard library modules are usually byte-code compiled as part of the installation process.

---

Package

Packages are namespaces which contain multiple packages and modules themselves. They are simply directories, but with a twist. Can be installed e.g. through pip or easy_install etc. Each package in Python is a directory which MUST contain a special file called __init__.py. This file can be empty, and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported. If we create a directory called marta, which marks the package name, we can then create a module inside that package called bar. We also must not forget to add the *__init__.py* file inside the marta directory.

---

Deep copy and Shallow copy

Shallow Copy

A shallow copy creates a new object which stores the reference of the original elements.

So, a shallow copy doesn’t create a copy of nested objects, instead it just copies the reference of nested objects. This means, a copy process does not recurse or create copies of nested objects itself.

->>> import copy
->>>old_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
->>>new_list = copy.copy(old_list)

->>> print("Old list:", old_list)
->>> print("New list:", new_list)

->>> Old list: [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
->>> New list: [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

->>> old_list = [[1, 1, 1], [2, 2, 2], [3, 3, 3]]
->>> new_list = copy.copy(old_list)

#append - add to the end

->>> old_list.append([4, 4, 4])

->>> print("Old list:", old_list)
->>> print("New list:", new_list)

Old list: [[1, 1, 1], [2, 2, 2], [3, 3, 3], [4, 4, 4]]
New list: [[1, 1, 1], [2, 2, 2], [3, 3, 3]]

#Adding new nested object using Shallow copy
->>> old_list = [[1, 1, 1], [2, 2, 2], [3, 3, 3]]
->>> new_list = copy.copy(old_list)

->>> old_list[1][1] = 'AA'

->>> print("Old list:", old_list)
->>> print("New list:", new_list)

Deep copy

A deep copy creates a new object and recursively adds the copies of nested objects present in the original elements.

->>> import copy

->>> old_list = [[1, 1, 1], [2, 2, 2], [3, 3, 3]]
->>> new_list = copy.deepcopy(old_list)

->>> print("Old list:", old_list)
->>> print("New list:", new_list)

Old list: [[1, 1, 1], [2, 2, 2], [3, 3, 3]]
New list: [[1, 1, 1], [2, 2, 2], [3, 3, 3]]


# add nested data 
->>> old_list = [[1, 1, 1], [2, 2, 2], [3, 3, 3]]
->>> new_list = copy.deepcopy(old_list)

->>> old_list[1][0] = 'BB'

->>> print("Old list:", old_list)
->>> print("New list:", new_list)

Old list: [[1, 1, 1], ['BB', 2, 2], [3, 3, 3]]
New list: [[1, 1, 1], [2, 2, 2], [3, 3, 3]]

# and see
->>> a
[2, 3, 4, 88, 9999]
->>> b=copy.copy(a)
->>> c=deepcopy(a)
->>> id(a)
140569946565704
->>> id(b)
140569921928520
->>> id(c)
140569947011400
->>> d=a
->>> id(d)
140569946565704
->>> a.append(123456)
->>> print(a,b,c,d)
[2, 3, 4, 88, 9999, 123456] [2, 3, 4, 88, 9999] [2, 3, 4, 88, 9999] [2, 3, 4, 88, 9999, 123456]
->>>