Building Specific Python Versions from source on Linux Mint 20

If you come from a background in windows, and are an avid python developer, you may be used to chopping and changing python base installations. It can be quite jarring transferring to Linux development where python is much more closely integrated with the OS, and you generally don’t want to be using the system python installations for development. I myself like to keep several base-versions of python installed and spin up virtual environments for each project I am working on, depending on the nature of the project and due to certain package dependencies I may favor 3.6/3.7 over 3.8 ect.

So how does one obtain specific / unusual versions without relying on adding some 3rd party sketchy software sources, well how about compiling it yourself from source? While this may seem like a long winded process, it’s actually fairly straight forward especially once you have completed it once before, and it’ll provide you with a clean efficient python installation.

The first step is to obtain all the dependencies required to build your version, to do this you need figure out which version of Ubuntu your installation is based upon. For Mint 20 this is ‘focal’ but to be sure simply read the lsb-release file (Linux-Standard-Base release):

cat /etc/upstream-release/lsb-release

It’ll spit out the following information, and the key-value pair we are most interested in is ‘DISTRIB_CODENAME’.

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu Focal Fossa"

Currently Mint builds it’s OS upon the Long-Term-Support releases of Ubuntu, and so the base version only changes every 2 years. Now you need to modify the apt sources to include deb-src based upon the lsb-release code-name:

sudo nano /etc/apt/sources.list

You’ll need to use sudo to elevate your privileges so that you can modify this file. Use the down key and go to the line below the last # comment, insert the deb-src line using the code-name you found above, for Mint 20 see below. If you are using LMDE or another Debian derivative that doesn’t piggyback off Ubuntu the process will be similar but using the pure Debian lsb-release sources.

#deb cdrom:[Linux Mint 20 _Ulyana_ - Release amd64 20200624]/ focal contrib main

# This system was installed using small removable media
# (e.g. netinst, live or single CD). The matching "deb cdrom"
# entries were disabled at the end of the installation process.
# For information about how to configure apt package sources,
# see the sources.list(5) manual.

deb-src http://archive.ubuntu.com/ubuntu/ focal main

To save this ctrl+o, it’ll ask for a filename to save, hit enter so that it’s saved as the same file. If you didn’t sudo your command you’ll get a permission error here.

Now lets get the build dependencies, build-dep python3 is a good starting point (assuming you are build a python 3 version). Then it’s worth testing to see if there are additional build dependencies for your particular version by adding the minor code.

sudo apt update
sudo apt build-dep python3
sudo apt build-dep python3.7

In my case there is no build-dep in focal for python3.7 (However 3.8 does have). In python versions greater than 3.7 you’ll need to manually install libffi as it’s no longer automatically bundled. This will stop errors while building the ctypes module.

sudo apt install libffi-dev

The SSL module also requires additional build dependencies. The following command will ensure everything is in order.

sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev

Now with our build dependencies resolved, lets pull down the Gzipped source tarball for our particular version. The best is solution is to go onto the official python website and determine which version you are after and then locate the FTP link, this will be followed with an MD5 checksum which you can use to validate the download. For me I am after 3.7.8 (MD5=4d5b16e8c15be38eb0f4b8f04eb68cd0). (If you are really ambitious and want a hot-off-the press release you can clone a version from github https://github.com/python/cpython).

cd Downloads/
wget https://www.python.org/ftp/python/3.7.8/Python-3.7.8.tgz
md5sum Python-3.7.8.tgz

Provided that spits out the correct md5 sum you are good to go, lets unzip this archive and configure the build. The –enable-optimization call-parameter helps improve the speed of the final python installation on your machine using Profile Guided and Link Time optimizations (PGO/LTO). This comes at the cost of a bunch more work during the compilation, so dealers-choice when it comes to that parameter.

tar -xzf Python-3.7.8.tgz
cd Python-3.7.8/
./configure --enable-optimizations

If you requre an installation with shared libraries it’s a little more tricky. Since it’s an alt install you have to manually define the path prefix, as the relative location will otherwise be incorrect. You’ll also need to export the environment variable or add the path to the linker so python doesn’t throw an error at runtime. All in all, should be avoided unless specifically required for PyInstaller or similar packages (use the instructions below if this is the case).

./configure --enable-optimizations --enable-shared --prefix=/opt/python LDFLAGS=-Wl,-rpath=/opt/python/lib
sudo ldconfig /usr/local/lib 

Now let it finish churning some ‘hacker-typer’ println’s, once finished we can start compiling. This is where the real magic happens, but depending on how much grunt your PC has, and if you enabled the LTO/PGO optimizations, the compilation can take a long time.

make

Once this has finished instead of using the normal ‘make install’ command, use the ‘make altinstall’ command instead. Since the former can end up replacing / interfering with system python installs, and the latter will neatly nest interpreters in individual folders by version number.

sudo make altinstall

Now this should create a new version install in /exec_prefix/bin/python with all the related modules in /usr/local/lib/. Using LM20 this folder will already contain a 2.7 python2 installation and a 3.8 python3 installation, which will be undisturbed by the altinstall. On linux mint the executable will be located in the following location if you didn’t specify an alternate prefix.

/opt/python/bin/python3.7

Now there should be a correctly linked python3.7 environment on your machine, you can test this running python3.7 --version and ensuring python3 --version still points to the system 3.8 installation.

Now lets update the base packages, spin up some venvs and start developing!

python3.7 -m pip install --update pip
python3.7 -m pip install --update setuptools
python3.7 -m venv /home/steve/Documents/NewProject/venv/