building python 3.8.3 on jetpack 4.4 dp and the jetson xavier nx

This how-to is a refinement of my first Python 3.8.3 from 16 May build, built on the older Jetson Nano with JetPack 4.4 DP ( https://arcanesciencelab.wordpress.com/2020/05/16/building-python-3-8-3-on-jetpack-4-4-developer-preview/ ). Where changes have occurred, they are noted.

These instructions will help you build Python 3.8.3 on the Jetson Xavier NX Development Kit running JetPack 4.4 Developer Preview. There are two broad steps to building this version. The first is the installation of support developer libraries to allow as many Python modules as possible to successfully build. After the installation step, then comes the build steps. This was reversed in the 16 May build how-to.

To build as many modules as possible the following libraries need to be installed.

  • libgdm-dev
  • libnss3-dev
  • libssl-dev
  • libsqlite3-dev
  • libreadline-dev
  • libbz2-dev
  • libdb-dev, libdb++-dev (Berkeley db)
  • libgdbm-dev
  • libffi-dev

And for those of you following along who just want to copy and paste one time, the single line command to install all the libraries at once is:

sudo apt install libgdm-dev libnss3-dev libssl-dev libsqlite3-dev libreadline-dev libbz2-dev libdb-dev libdb++-dev libgdbm-dev libgdbm-dev libffi-dev

The package libffi-dev was not in the installation list from 16 May. If not installed then then module _ctypes will fail to build.

  1. If you haven’t done so already then download the latest Python, version 3.8.3, from https://www.python.org/downloads/.
  2. Untar the file (we’ll assume it’s downloaded to the default ~/Downloads):
    tar xvf Downloads/Python-3.8.3.tar.xz
  3. Make a build directory at the same level as the untarred source directory. In this case the build directory was named build-python-3.8.3
  4. Change directory into the build directory.
  5. From within the build directory execute the configure script:
    ../Python-3.8.3/configure --enable-optimizations
  6. Run make:
    make -j 6

    Note that we’re using all six cores of the Jetson Xavier NX. This means that all six cores should be enabled on the NX.

  7. Install into the alternate location for this version of Python:
    sudo -H make altinstall
  8. Check for the alternate location with ‘which python3.8’. It should return ‘/usr/local/bin/python3.8’.

I then created a Python 3.8.3 virtual work environment. These are the specific steps I used to create that environment.

  1. In your home directory, create a work folder. In this example it was named ‘venv383’. Change directory (cd) into it.
  2. Then create a Python VE:
    python3.8 -m venv app01

Once created, you should have a directory structure like this:

├── bin/
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── easy_install
│   ├── easy_install-3.8
│   ├── pip
│   ├── pip3
│   ├── pip3.8
│   ├── python -> python3.8
│   ├── python3 -> python3.8
│   └── python3.8 -> /user/local/bin/python3.8
├── include/
├── lib/
│   └── python3.8/
│       └── site-packages/
│           ├── easy_install.py
│           ├── pip/...
│           ├── pip-19.2.3.dist-info/...
│           ├── pkg_resources/...
│           ├── __pycache__/...
│           ├── setuptools/...
│           └── setuptools-41.2.0.dist-info/...
├── lib64 -> lib/
└── pyvenv.cfg

To active the virtual environment, you would execute

source bin/activate

In my environment I created a Bash alias to handle that:

alias activate='source $HOME/venv383/app01/bin/activate'

Note that this incorporates the folder and virtual environment name I personally created earlier. If you change any of that then you’ll incorporate what you created.

Here’s a quick run to see if those bits are working properly.

And so it would appear that they do.

building python 3.8.3 on jetpack 4.4 developer preview

We’re going to be installing Python 3.8.3 onto the Jetson Nano. Every current version of Jetson Nano’s JetPack Development Kit is built and hosted on Ubuntu 18.04. Ubuntu 18.04 comes stock with Python 3.6.9. We want to use a more up-to-date version of Python, but not clobber the default version. Installing the up-to-date version side-by-side with the default Ubuntu version will be accomplished with the alternate installation provided by Python’s make process.

  1. Download the source from https://www.python.org/downloads/. In this post it was Python 3.8.3
  2. Untar the file:
    tar xvf Downloads/Python-3.8.3.tar.xz
  3. Make a build directory at the same level as the source directory. In this case it was named build-python-3.8.3
  4. Change directory into the build directory.
  5. From within the build directory execute the configure script:
    ../Python-3.8.3/configure --enable-optimizations
  6. Run make:
    make -j 4
  7. Install into the alternate location for this version of Python:
    sudo -H make altinstall
  8. Check for the alternate location with ‘which python3.8’. It should return ‘/usr/local/bin/python3.8’.

Python 3.8.3 will build and be functional as is, but if you want all the features to build for Python you’ll need to install a list of additional libraries in order to enable all the features, or at least as many as you care about.

My list of imported libraries for this build includes:

  • libgdm-dev
  • libnss3-dev
  • libssl-dev
  • libsqlite3-dev
  • libreadline-dev
  • libbz2-dev
  • libdb-dev, libdb++-dev (Berkeley db)
  • libgdbm-dev

Why the additional libraries? Because this is what happened when I built in my unique environment the first time (First run).

# First run

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2                  _dbm                  _gdbm              
_hashlib              _sqlite3              _ssl               
_tkinter              readline                                 
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           


Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

# Second run

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_tkinter           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc                  atexit                pwd                
time                                                           

It took two attempts, the first to have make list the modules that wouldn’t be built at the end of a full build. I went looking in both the Bash script function ‘detect_modules()’ as well as on the web, and managed to install support for everything I cared about. The only module I didn’t care to build was tkinter. Tkinter is the interface to the Tk GUI toolkit. I use PyQt5 instead, so found no need to build it. Your needs, of course, may vary, and it may come to pass I need a package that uses Tcl/Tk GUI. But I doubt that.

Once Python 3.8.3 was built I created a virtual environment according to the directions here: https://docs.python.org/3/library/venv.html . Just as I didn’t want this latest version installed over the stock Python 3.6.9, I’m using a virtual environment to keep usage of 3.8.3 from corrupting the stock environment. In my environment I create an alias to execute ‘activate’. One nice feature of a virtual environment is that ‘python’ points to Python 3.8.3.

More to come…