Python Notes : Modules
Modules
work done on a terminal is lost when the terminal is shut down
better to write scripts and save them as files
such files are called modules
definitions from a module can be imported into other modules
fibo.py
# Fibonacci numbers module

def fib(n):    # write Fibonacci series up to n
    a, b = 0, 1
    while a < n:
        print(a, end=' ')
        a, b = b, a+b
    print()

def fib2(n):   # return Fibonacci series up to n
    result = []
    a, b = 0, 1
    while a < n:
        result.append(a)
        a, b = b, a+b
    return result
through the interpreter
>>> import fibo
>>> fibo.fib(1000)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987
>>> fibo.fib2(100)
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
>>> fibo.__name__
'fibo'

>>> fib = fibo.fib
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
More on Modules
module can contain executable statements as well as function definitions
these statements are intended to initialize the module
they are executed only the first time the module name is encountered in an import statement

each module has its own private namespace
is used as the global namespace by all functions defined in the module

modules can import other modules
customary but not required to place all import statements at the beginning of a module
the imported module names, if placed at the top level of a module, are added to the module's global namespace

variant of the import statement that imports names from a module directly into the importing module's namespace

>>> from fibo import fib, fib2
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
variant of the import statement that imports all names that a module defines
imports all names except those beginning with an underscore
>>> from fibo import *
>>> fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
if the module name is followed by as, then the name following as is bound directly to the imported module
>>> import fibo as fib
>>> fib.fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
can also use aliases
>>> from fibo import fib as fibonacci
>>> fibonacci(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377
Executing Modules as Scripts
run a module as a script using the interpreter
>>> py fibo.py <arguments>
code that parses the command line only runs if the module is executed as the 'main' file
to permit a module to be both imported and run as a script add to end of module
if __name__ == "__main__":
    import sys
    fib(int(sys.argv[1]))
The Module Search Path
when a module named xxx is imported, the interpreter first searches for a built-in module with that name
names of built-in modules are listed in sys.builtin_module_names
if not found, it then searches for a file named xxx.py in a list of directories given by the variable sys.path
sys.path is initialized from
  • the directory containing the input script (or the current directory when no file is specified)
  • PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH)
  • the installation-dependent default (by convention including a site-packages directory, handled by the site module)
"Compiled" Python files
to speed up loading modules, Python caches the compiled version of each module in the __pycache__ directory under the name module.version.pyc
the version encodes the format of the compiled file
generally contains the Python version number
in CPython release 3.3 the compiled version of xxx.py would be cached as __pycache__/xxx.cpython-33.pyc
naming convention allows compiled modules from different releases and different versions of Python to coexist

Python checks the modification date of the source against the compiled version to see if it's out of date and needs to be recompiled
an automatic process
the compiled modules are platform-independent

two circumstances when Python does not check the cache

  • it always recompiles and does not store the result for the module loaded directly from the command line
  • it does not check the cache if there is no source module
    to support a non-source (compiled only) distribution
    • the compiled module must be in the source directory
    • there must not be a source module
Standard Modules
Python comes with a library of standard modules
some modules are built into the interpreter
these provide access to operations that are not part of the core of the language but are nevertheless built in
sys module is built into every Python interpreter
the variables sys.ps1 and sys.ps2 define the strings used as primary and secondary prompts
>>> import sys
>>> sys.ps1
'>>> '
>>> sys.ps2
'... '
>>> sys.ps1 = 'C> '
C> print('Yuck!')
Yuck!
C>
the variable sys.path is a list of strings that determines the interpreter's search path for modules
is initialized to a default path taken from the environment variable PYTHONPATH
if PYTHONPATH is not set path taken from a built-in default
can modify it using standard list operations
>>> import sys
>>> sys.path.append('/ufs/guido/lib/python')
The dir() Function
built-in function dir() is used to find out which names a module defines
returns a sorted list of strings
>>> import fibo, sys
>>> dir(fibo)
['__name__', 'fib', 'fib2']
dir() does not list the names of built-in functions and variables
to get a list of those use the standard module builtins
>>> import builtins
>>> dir(builtins)  
['ArithmeticError', 'AssertionError', 'AttributeError', 'BaseException',
 'BlockingIOError', 'BrokenPipeError', 'BufferError', 'BytesWarning',
 'ChildProcessError', 'ConnectionAbortedError', 'ConnectionError',
 'ConnectionRefusedError', 'ConnectionResetError', 'DeprecationWarning',
 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False',
 'FileExistsError', 'FileNotFoundError', 'FloatingPointError',
 'FutureWarning', 'GeneratorExit', 'IOError', 'ImportError',
 'ImportWarning', 'IndentationError', 'IndexError', 'InterruptedError',
 'IsADirectoryError', 'KeyError', 'KeyboardInterrupt', 'LookupError',
 'MemoryError', 'NameError', 'None', 'NotADirectoryError', 'NotImplemented',
 'NotImplementedError', 'OSError', 'OverflowError',
 'PendingDeprecationWarning', 'PermissionError', 'ProcessLookupError',
 'ReferenceError', 'ResourceWarning', 'RuntimeError', 'RuntimeWarning',
 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError',
 'SystemExit', 'TabError', 'TimeoutError', 'True', 'TypeError',
 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError',
 'UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', 'UserWarning',
 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__build_class__',
 '__debug__', '__doc__', '__import__', '__name__', '__package__', 'abs',
 'all', 'any', 'ascii', 'bin', 'bool', 'bytearray', 'bytes', 'callable',
 'chr', 'classmethod', 'compile', 'complex', 'copyright', 'credits',
 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'exec', 'exit',
 'filter', 'float', 'format', 'frozenset', 'getattr', 'globals', 'hasattr',
 'hash', 'help', 'hex', 'id', 'input', 'int', 'isinstance', 'issubclass',
 'iter', 'len', 'license', 'list', 'locals', 'map', 'max', 'memoryview',
 'min', 'next', 'object', 'oct', 'open', 'ord', 'pow', 'print', 'property',
 'quit', 'range', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice',
 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'vars',
 'zip']
Packages
Packages organize modules in a directory structure into a namespace by using 'dotted module names'
sound/                          Top-level package
      __init__.py               Initialize the sound package
      formats/                  Subpackage for file format conversions
              __init__.py
              wavread.py
              wavwrite.py
              aiffread.py
              aiffwrite.py
              auread.py
              auwrite.py
              ...
      effects/                  Subpackage for sound effects
              __init__.py
              echo.py
              surround.py
              reverse.py
              ...
      filters/                  Subpackage for filters
              __init__.py
              equalizer.py
              vocoder.py
              karaoke.py
              ...
__init__.py files are required to make Python treat directories containing the file as packages
a relatively advanced feature is using a namespace package
prevents directories with a common name from unintentionally hiding valid modules in the module search path
__init__.py can
  • contain initialization code
  • set the __all__ variable
  • be an empty file
users of the package can import individual modules from the package
loads the submodule sound.effects.echo
must be referenced with its full name
import sound.effects.echo
sound.effects.echo.echofilter(input, output, delay=0.7, atten=4)
can also import using aliases
from sound.effects import echo
echo.echofilter(input, output, delay=0.7, atten=4)
can import the desired function or variable directly using an alias
from sound.effects.echo import echofilter
echofilter(input, output, delay=0.7, atten=4)
if an import statement fails an ImportError exception is raised
Importing * From a Package
it's not wise to import all submodules from a package
from sound.effects import *
doing so could take a long time
importing all sub-modules might have side-effects which should only happen when the sub-module is explicitly imported

package author can provide an explicit index of the package
a package's __init__.py code can define a list of module names that should be imported when from package import * is encountered
the __all__ variable is a list of strings that defines the names that are exported when from package import * is used

__all__ = ["echo", "surround", "reverse"]
sound.effects import * would import the three named submodules of the sound.effects package
Intra-package References
modules within a package can reference each other using relative imports without circular references
below moduleA imports moduleC while moduleC imports moduleA
Import Absolute Relative
Import from moduleA in moduleB from package.moduleA import XXX from .moduleA import XXX
Import from moduleA in moduleC from package.moduleA import XXX from ..moduleA import XXX
Import from moduleC in moduleA from package.subpackage.moduleC import XXX from .subpackage.moduleC import XXX
Packages in Multiple Directories
packages support one more special attribute __path__
initialized to be a sequence of strings containing the name of the directory holding the package's __init__.py
not often needed
previous    index    next