bic4color

The 4 colors BIC ball-pen allows a single pen to switch between different ball-pens. Likewise, your UNIX terminal can switch between several distinct Python virtual environments ( “venv” for short), with a total control over the packages loaded in each venv.

What is a Python virtual environment and why you should use it?

One big change when passing from Python 2 to Python 3 is the use of virtual environments (venv). The idea is simple: in a virtual environment, you may install a set of libraries exclusive to this environment. Venvs avoid compatibility issues (i.e. less problems with python), and help a lot all support operation (faster resolution of problems).

A collection of virtual environment tutorials and motivators

Virtual environment is not a COOP invention, but “the recommended way to master your python framework”. It can be related to your first contact with the UNIX environment variables (.bashrc,module load, etc…). See the following link for non-COOP venvs explanations:

Let us add two situations where venvs show their usefulness

Divide to Reign

When you you work in a venv, you are limiting the number of packages you are working with. Listing (pip list) and Checking (pip check) is just far easier and faster with a smaller number of packages.

For support, you can create an ID card of your python environment, which is simply the list of packages installed. The smaller the list is, the faster your dear support will be able to be in the exact state as you are on a different machine.

Avoiding conflicts

You work on a fist project “A” in the spring. Project “A” uses the package “awesome_stats 2.3”, but was not upgraded to support “awesome_stats 3.O” or later. You do not care about this menial detail.

You work on a second project “B” in summer. “B” is using the new features of “awesome_stats 3.O”. The installation will install seamlessly the version 3.0. (Actually, if you were doing a pip check on your terminal, project “A” would complain, but who does this regularly?)

You come back to project “A” in autumn, but it obviously does not work anymore. If you revert to “awesome_stats 2.3”, project “B” would meet the same fate. This is, technically speaking, a dependency hell of conflicting dependencies.

Two virtual environments would make this problem non-existent.

How to install a virtual environment?

Prerequisites

First you have to install Python 3, you can find how to do it on https://www.python.org/downloads/. Make sure you are using the right version of Python with the command ‘which python’

python -m venv

The command to install a virtual environment is

python -m venv -awesome_new_environement-

If you want to create a duplication of your base environment (copy of all pre-installed packages), you can use the --system-site-packages optional argument. Not recommended, since keeping your venvs as small as possible should be your top priority

If venv is not installed and you are on linux, you may need to ask for it with more conviction:

sudo apt-get install python3-venv
Worked example

Here is an example with the venv totoro created in a folder python_venvs at the root level:

mylogin@mymachine:~>python -m venv ~/python_venvs/totoro
mylogin@mymachine:~>ls totoro
bin/    include/    lib/    pyvenv.cfg

This command will create a folder totoro with inside three folders and a file: bin/, include/, lib/ and pyvenv.cfg. Now that you have created your environment, you have to activate it before installing the libraries you need on it. To activate your virtual environment, use the activate shell script inside the venv:

mylogin@mymachine:~>source ~/totoro/bin/activate
(totoro) mylogin@mymachine:~>

You should see at the beginning of the command line the name of your virtual environment. Now you can install every library you need:

(totoro) mylogin@mymachine:~>pip install numpy
(totoro) mylogin@mymachine:~>pip install scipy

You can quit your virtual environment and go back to the default one with deactivate:

(totoro) mylogin@mymachine:~>deactivate
mylogin@mymachine:~>

The packages available in totoro are no more visible. Try pip list to check.

Like this post? Share on: TwitterFacebookEmail


Antoine Dauptain is a research scientist focused on computer science and engineering topics for HPC.

Keep Reading


Published

Category

Pitch

Tags

Stay in Touch