Virtual Environments

Working with a Python Environment

Python's venv module is used to create virtual environments
this module is part of the standard library,
officially recommended way to create virtual environments

Creating a Virtual Environment

to create a virtual environment in a Windows shell use the command

py -m venv venv

Python launcher selects an appropriate version of Python to use
launcher is bundled with official installation
command creates a new virtual environment named venv using the built-in venv module
the first venv that you use in the command specifies the module
the second venv sets the name of the virtual environment

Activating a Virtual Environment

activate the environment by executing a script that comes with the VE installation

venv\Scripts\activate

must be in folder containing the VE
the VE name will be included in the command prompt as (venv)

Installing Packages into a Virtual Environment

once activated install any required dependencies using pip

python -m pip install <package-name>

using a VE creates an isolated environment in which packages can be installed
no dependency conflicts between projects in their own VE

Deactivating a Virtual Environment

can deactivate VE suing the command

deactivate

command prompt returns to normal
any interaction now with Python or pip is within the global Python environment

Enabling a Virtual Environment

working with virtual environments directly from an IDE can streamline the development process
VS Code provides built-in support for managing Python virtual environments
can create, activate, and manage VEs without leaving the editor

Create and Activate a Virtual Environment in VS Code

to create and activate a Python environment in VS Code

Open the Workspace - launch VS Code and open the folder that containing the Python project
Open the Integrated Terminal - navigate to View → Terminal in the menu to launch the integrated terminal
Create a Virtual Environment - in the terminal create a new virtual environment with the venv module
Activate the Virtual Environment - using the integrated terminal, use the platform-specific command to activate the virtual environment

Why Use a Virtual Environment?

Python isn't great at dependency management
if not specific pip will place all the external packages being installed in a folder called site-packages/ in the base Python installation
there are two site-package folders

purelib/ should contain only modules written in pure Python
platlib/ should contain binaries that aren't written in pure Python

most OSs implement Python's site-packages setting so that both locations point to the same path
effectively creates a single site-packages folder

>>> import sysconfig
>>> sysconfig.get_path("purelib")
'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages'
>>> sysconfig.get_path("platlib")
'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages'

Avoid System Pollution

installing packages to the OS's global site-packages will mix the packages with the system-relevant packages
mix-up could have unexpected side effects on tasks crucial to the OS's normal behavior
updates to OS could overwrite or lose any installed packages

Sidestep Dependency Conflicts

different projects may require different versions of the same external library
if two different versions of the same package are installed into the global Python environment, the second installation overwrites the first one
each VE can contain a specific version of the same external library

Minimize Reproducibility Issues

make the VE reproducible by documenting its contents
create a requirements.txt file while the virtual environment is active

(venv) PS> python -m pip freeze > requirements.txt

command pipes the output of pip freeze into a file called requirements.txt
file contains a list of the external dependencies currently installed in the VE

keep the requirements.txt file up to date
can always re-create the virtual environment even after deleting the venv/ folder or moving to a different computer altogether

(venv) PS> deactivate
PS> py -m venv new-venv\
PS> new-venv\Scripts\activate
(new-venv) PS> python -m pip install -r requirements.txt

above a new virtual environment called new-venv is created and activated
all external dependencies in the requirements.txt file are installed

Dodge Installation Privilege Lockouts

may need administrator privileges on a computer to install packages into the host Python's site-packages directory
with virtual environments a new installation location is created
can install and work with external packages

What is a Virtual Environment?

a VE is a folder structure providing everything needed to run a lightweight yet isolated Python environment

A Folder Structure

a new venv-created VE includes a self-contained folder structure and symlinks the Python executable files into the structure

A Folder Structure in Detail

in the terminal navigate to the folder that containing the VE
execute the tree command to display the contents of the directory

PS> tree venv\ /F

a VE folder contains a lot of files and folders
most of what makes this tree structure so long rests inside the site-packages/ folder

venv\
│
├── Include\
│
├── Lib\
│   │
│   └── site-packages\
│       │
│       ├── pip\
│       │
│       └── pip-24.2.dist-info\
│
│
├── Scripts\
│   ├── Activate.ps1
│   ├── activate
│   ├── activate.bat
│   ├── deactivate.bat
│   ├── pip.exe
│   ├── pip3.12.exe
│   ├── pip3.exe
│   ├── python.exe
│   └── pythonw.exe
│
└── pyvenv.cfg

folder/file	description
Include\	is an initially empty folder used to include C header files for installed packages which depend on C extensions
Lib\	contains the site-packages\ folder is where external packages are installed within the VE
Scripts\	contains the executable files of your virtual environment includes the Python interpreter (python.exe) the pip executable (pip.exe) the activation script for your virtual environment comes in a couple of different flavors to allow working with different shells
a {name}-{version}.dist-info/	get by default for pip contains package distribution information which records information about installed packages
pyvenv.cfg	is a crucial file for the VE contains only a couple of key-value pairs Python uses to set variables in the sys module variables determine which Python interpreter and site-packages directory the current Python session will use home = C:\Users\Name\AppData\Local\Programs\Python\Python312 include-system-site-packages = false version = 3.12.5 executable = C:\Users\Name\AppData\Local\Programs\Python\Python312\python312.exe command = C:\Users\Name\AppData\Local\Programs\Python\Python312\python312.exe -m venv C:\Users\Name\path\to\project\venv

three essential parts to a virtual environment

a symlink of the Python binary
a pyvenv.cfg file
a site-packages directory

An Isolated Python Installation

want an isolated environment so that any installed external packages won't conflict with global site-packages
venv reproduces the folder structure that a standard Python installation creates
structure accounts for the location of the symlink of the Python binary and the site-packages directory

a VE has access to packages in the standard library
follow the venv command below to have a VE access to the base installation's site-packages
only get read access to the system site-packages directory

PS> py -m venv venv\ --system-site-packages

can change pyvenv.cfg instead

include-system-site-packages = true

How a Virtual Environment Works?

It Copies Structure and Files

when a VE is created using using venv, the module re-creates the file and folder structure of a standard installation on the OS
symlinks the Python executable into the folder structure with which venv was called

It Adapts the Prefix-Finding Process

venv produces the standard folder structure which lets the interpreter determine where all the VE's relevant files are located
instead of looking for the os module to determine the location of the standard library the interpreter first looks for a pyvenv.cfg file
if the file is found and it contains a home key the interpreter will use that key to set the value for two variables

sys.base_prefix will hold the path to the Python executable used to create the VE
sys.prefix will point to the directory containing pyvenv.cfg.

if the two variables have different values, Python adapts where it'll look for modules

It Links Back to the Standard Library

VEs aim to be a lightweight way to provide an isolated Python environment
VE can be quickly created and then deleted when it's no longer needed
venv copies only the minimally necessary files

It Modifies the PYTHONPATH

ensures the scripts needed to run use the Python interpreter within the VE
venv modifies the PYTHONPATH environment variable which can be accessed using sys.path
inspect the variable without an active VE and it shows the default path locations for the default Python installation

>>> import sys
>>> sys.path
['',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\DLLs',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\lib',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312',
 'C:\\Users\\Name\\AppData\\Roaming\\Python\\Python312\\site-packages',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\lib\\site-packages']

when the same command is made when a VE is running the output is

>>> import sys
>>> sys.path
['',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\python312.zip',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\DLLs',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312\\lib',
 'C:\\Users\\Name\\AppData\\Local\\Programs\\Python\\Python312',
 'C:\\Users\\Name\\path\\to\\venv',
 'C:\\Users\\Name\\path\\to\\venv\\lib\\site-packages']

Python replaced the default site-packages directory path with the one that lives inside the VE
the change means that Python will load any external packages installed in the VE
because the path to your base Python's site-packages directory is no longer in the list Python won't load modules from there
change in Python's path settings effectively creates the isolation of external packages in the VE

It Changes the Shell PATH Variable on Activation

the activation script performs two critical steps

Path	sets the VIRTUAL_ENV variable to the root folder path of the VE prepends the relative location of its Python executable to the PATH because the path to all the executables in the VE now lives at the front of the PATH, the shell will invoke the internal versions of pip or Python when pip or python is entered
Command prompt	changes the command prompt to the name passed when the VE was created takes that name and puts it into the prompt enclosed in parentheses the script changes the command prompt showing the VE is activated

Customizing a Virtual Environment

Change the Command Prompt

when creating a VE can use an argument to provide an alias to the VE's real name

PS> py -m venv venv\ --prompt dev-env
PS> venv\Scripts\activate
(dev-env) PS>PS> py -m venv venv\ --prompt dev-env
PS> venv\Scripts\activate
(dev-env) PS>

to use the name of the current working directory as the command prompt of the VE

PS> py -m venv venv\ --prompt .
PS> venv\Scripts\activate
(<current-folder-name>) PS>

when the dot operator is specified as the value for --prompt Python will use the output of os.path.basename(os.getcwd()) as the command prompt for the VE

Overwrite Existing Environments

venv does not overwrite an existing VE when the same venv command is made
below the list of install packages is unchanged after py -m venv venv\ is run a second time

PS> py -m venv venv\
PS> venv\Scripts\pip.exe install requests
PS> venv\Scripts\pip.exe list
Package            Version
------------------ ---------
certifi            2024.8.30
charset-normalizer 3.3.2
idna               3.8
pip                24.2
requests           2.32.3
urllib3            2.2.2

PS> py -m venv venv\
PS> venv\Scripts\pip.exe list
Package            Version
------------------ ---------
certifi            2024.8.30
charset-normalizer 3.3.2
idna               3.8
pip                24.2
requests           2.32.3
urllib3            2.2.2

to reinitialize a VE use the --clear argument

PS> py -m venv venv\
PS> venv\Scripts\pip.exe install requests
PS> venv\Scripts\pip.exe list
Package            Version
------------------ ---------
certifi            2024.8.30
charset-normalizer 3.3.2
idna               3.8
pip                24.2
requests           2.32.3
urllib3            2.2.2

PS> py -m venv venv\ --clear
PS> venv\Scripts\pip.exe list
Package    Version
---------- -------
pip        24.2

Create Multiple Virtual Environments at Once

can create multiple separate VEs in one go by passing more than one path to the command

PS> py -m venv venv\ C:\Users\Name\Documents\virtualenvs\venv-copy\

running the command creates separate VEs in different locations
the two folders are independent VE folders

above the first argument, venv/, represents a relative path
the second argument uses an absolute path

Update the Core Dependencies

when a package is installed into a VE using pip, may receive a message about updating pip
venv uses ensurepip to bootstrap pip into the VE

can specify for pip to contact PyPI and update itself right after installation by passing the --upgrade-deps argument

PS> py -m venv venv\ --upgrade-deps
PS> venv\Scripts\activate
(venv) PS> python -m pip install --upgrade pip

Include the System Site-Packages

can access all modules installed into the base Python's site-packages directory by using the --system-site-packages flag when creating the VE
installing any additional external packages will put them into the VE's site-packages directory
only get read access to the system site-packages directory

PS> py -m venv venv\ --system-site-packages
PS> venv\Scripts\activate
(venv) PS>

Managing a Virtual Environment

Decide Where to Create the Environment Folders

two approaches to where VEs should be located

project-folder approach	create a new VE in the root folder of the project which will use the VE the VE folder then lives alongside any code that written for the project the advantage is knowing which VE belongs to which project can activate the VE using a short relative path after navigating into the project folder
single-folder approach	keep all of VEs in a single folder could be less effort to keep track of which VEs have been created can go to a single location to inspect all VEs to decide which ones to keep and which ones to delete

Treat Them as Disposables

VEs are disposable folder structures
should be able to safely delete and re-create at any time without losing information about the code project
generally don't put any additional code or information into the VE manually

anything that goes in there should be handled by your package manager

don't commit the VE to version control
don't ship it with the project

VEs aren't entirely self-sufficient Python installations
VEs rely on the base Python''s standard library
won't create a portable application by distributing the VE together with the code

Pin the Dependencies

to make a VE reproducible need a way to describe its contents
most common way to do this is by creating a requirements.txt file while the VE is active

(venv) PS> python -m pip freeze > requirements.txt

requirements.txt contains a list of the external dependencies currently installed in the VE

to recreate the VE

(venv) PS> deactivate
PS> py -m venv new-venv\
PS> new-venv\Scripts\activate
(new-venv) PS> python -m pip install -r requirements.txt

committing the requirements.txt file to version control means can ship the project code with the 'recipe' to re-create the same VE on their machines

while this is a widespread way to ship dependency information with a code project, it isn't deterministic because of these issues

Python Version	the requirements file doesn't include information about which version of Python used as the base interpreter when creating the VE
Sub-Dependencies	depending on how the requirements file is created, it may not include version information about sub-dependencies of the dependencies someone could get a different version of a subpackage if that package was silently updated after the requirements file as created