Fixing Virtualenvs after System Python Upgrade

By | March 9, 2020

Recently I've been testing out the 20.04 beta for Ubuntu. It's real nice. But it recently upgraded python from python3.7 to python3.8 underneath me. It's honestly not such a bad thing. Python 3.8 has a lot of improvements that will make working with the language quite excellent. I'm particularly looking forward to utilizing Assignment
and getting my continue statementsback because I'm a curmudgeon at heart.

But when the python version changes it breaks things.

$ python
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f9b1f321740 (most recent call first):
Aborted (core dumped)

Nobody likes a core dump. Now if you can utilize python3.8 (like most things should be able to do looking at you saltstack. Your fix is simple. Just recreate your virutalenv (make sure you deactivate first so you're using the working, system python) and you should be back on track. Example :

(bch_dev_track) $ deactivate
$ cd ../
$ virtualenv -p python3 bch_dev_track/
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/chalbersma/Documents/src/github/chalbersma/bch_dev_track/bin/python3
Not overwriting existing python script /home/chalbersma/Documents/src/github/chalbersma/bch_dev_track/bin/python (you must use /home/chalbersma/Documents/src/github/chalbersma/bch_dev_track/bin/python3)
Installing setuptools, pkg_resources, pip, wheel...done.
$ cd bch_dev_track/
$ source bin/activate
(bch_dev_track) $ python --version
Python 3.8.2

Easy as pie! If you're in my boat you'll need to do a little bit more. First things first
the deadsneaks group is excellent
but it currently doesn't have release for 20.04 (because it's a beta which is understandable).
So we're going to need to compile python from source and install it. I followd the linked guide but the most important part to remember is to use altinstall. Use altinstall when making otherwise you will
overwrite your system python.

Once you have python3.7 you can confirm with the following:

$ python3.7 --version
Python 3.7.6

Now you can go ahead and we can fix your existing virtualenvs. If you try to do the above without cleanup you'll have some issues. So you need to do a small amount of cleanup first:

# From the root of your virtualenv
rm -r lib share include
unlink bin/python
unlink bin/python3
unlink bin/python3.7
# optionally unlink other version if this was (for example) a python3.6 installation

Then you can simply reinstall:

# From the parent directory
virtutualenv -p python3.7 $virtualenvloc

Now you should once again have a working virutalenv. At this point you should take the time to check and reinstall all of your dependencies and rerun all your tests. Best of luck!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.