My Python Setup
[…] because the language is easy to learn and put to use, many practicing Python programmers leverage only a fraction of its powerful features.
— Luciano Ramalho1
Is the thought of not fully leveraging the Python environment daunting you?2 We are not that different then. Thus my constant search for new tools and ways of improving my Python experience.
This is an opinionated summary-guide of my current setup for the future me. Feel free to follow along with me!
PyCharm integration with WSL and Docker is a Professional feature. Think about using Visual Studio Code instead.
The first step in setting up our dev environment is installing Python in our WSL. We use
sudo apt-get update && sudo apt-get dist-upgrade to re-synchronize the package index files, upgrade, and handles dependencies conflicts.
If you are using multiple versions of Python, think about using pyenv. Otherwise, we install our preferred version of Python using the following commands:
sudo apt install software-properties-common sudo add-apt-repository -y ppa:deadsnakes/ppa sudo apt install python3.8
Then we check everything went as expected:
$ python3 --version Python 3.8.5
I prefer having Python 3 as my default version – using
python instead of
Ubuntu 20.04+ you can do the following to achieve that:
sudo apt install python-is-python3
$ python --version Python 3.8.5
Poetry is a tool for dependency management and packaging in Python.
To install Poetry:
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
A quick health check:
$ poetry --version Poetry version 1.1.4
My playground is
laboratory on my
C drive. To create a new project:
$ cd /mnt/c/laboratory $ poetry new python-boilerplate Created package python_boilerplate in python-boilerplate
$ cd python-boilerplate $ poetry install Creating virtualenv python-boilerplate-zUD6aEZx-py3.8 in /home/ayoub/.cache/pypoetry/virtualenvs Updating dependencies Resolving dependencies... (1.7s) Writing lock file Package operations: 8 installs, 0 updates, 0 removals • Installing pyparsing (2.4.7) • Installing attrs (20.3.0) • Installing more-itertools (8.6.0) • Installing packaging (20.7) • Installing pluggy (0.13.1) • Installing py (1.9.0) • Installing wcwidth (0.2.5) • Installing pytest (5.4.3) Installing the current project: python-boilerplate (0.1.0)
Notice that Pytest is installed by default. The highlighted line contains the path to your virtual environments;
/home/ayoub/.cache/pypoetry/virtualenvs. We will use this path to set up PyCharm. Keep it!
Setting up PyCharm
We first open the project using your PyCharm. My project directory is
/mnt/c/laboratory inside WSL or
C:\laboratory\python-boilerplate in Windows.
To add a new Python interpreter to PyCharm:
CTRL + ALT + S >
Python Interpreter >
Then from the side menu:
/home/ayoub/.cache/pypoetry/virtualenvs/python-boilerplate-zUD6aEZx-py3.8/bin/python3.8. The path is to your virtual enviroment generated by Poetry. Use
poetry env info to print it again.
With Python interpreter set up. Let’s change the Terminal to WSL instead of CMD:
CTRL + ALT + S >
Shell path >
Installing development dependencies
Programmers! Cast out your guilt! Spend half your time in joyous testing and debugging! Stalk bugs with care, methodology, and reason. Build traps for them. Be more artful than those devious bugs and taste the joys of guiltless programming! 3
— Boris Beizer
Pytest, the framework of choice for testing in Python, is installed by default with Poetry. To keep an eye on coverage we use Coverage.py:
$ poetry add --dev coverage[toml] pytest-cov
[toml] extension allows us to configure coverage through
pyproject.toml file. While pytest-cov is the plugin to use Pytest to generate coverage reports like so:
pytest --cov=myproj tests/
-------------------- coverage: ... --------------------- Name Stmts Miss Cover ---------------------------------------- myproj/__init__ 2 0 100% myproj/myproj 257 13 94% myproj/feature4286 94 7 92% ---------------------------------------- TOTAL 353 20 94%
If you feel optimistic add the following configuration to your
# pyproject.toml [tool.coverage.report] fail_under = 100
For some “Mocking”, explore pytest-mock.
Flake8 is a wrapper around these tools:
- PyFlakes checks Python source files for errors.
- Pycodestyle checks your Python code against some of the style conventions in PEP 8.
- Ned Batchelder’s McCabe script checks McCabe complexity
Flake8 runs all those tools by launching a single
flake8 command. It displays the warnings in a per-file, merged output. Flake8 could be reinforced with additional extensions. We will use a few of those here.
$ poetry add --dev flake8
To configure your Flake8 create
.flake8 in the root of your project and add these lines:
# .flake8 [flake8] extend-ignore = E203, E266, E501 max-line-length = 88 max-complexity = 18
For Flake8 to play nice with Black we will be ignoring some rules:
E203: Whitespace before ‘:’
E266: Too many leading ‘#’ for block comment
E501: Line longer than 79 characters
We ignored the
E501 while setting the max line length to 88; the default in Black.
Speaking of which, Black is the uncompromising Python code formatter. Black just works. No need for configuration. We use Flake8’s extension flake8-black to run Black
--check ... from within the Flake8 plugin ecosystem.
$ poetry add --dev black flake8-black
The basic configuration for Black:
# Borrowed from: https://github.com/psf/black/blob/master/pyproject.toml # pyproject.toml [tool.black] include = '\.pyi?$' exclude = ''' /( \.git | \.hg | \.mypy_cache | \.tox | \.venv | _build | buck-out | build | dist )/ '''
We add isort a Python utility to sort imports alphabetically, and automatically separated into sections and by type.
$ poetry add --dev isort flake8-isort
# pyproject.toml [tool.isort] multi_line_output = 3 include_trailing_comma = true force_grid_wrap = 0 use_parentheses = true ensure_newline_before_comments = true line_length = 88
Every development pipeline has tasks, such as test, lint or publish. With taskipy, you can define those tasks in one file and run them with a simple command.
$ poetry add --dev taskipy
# myproject.toml [tool.taskipy.tasks] format = "black . && isort ."
I think we should use Git for this project, even though
python-boilerplate-v3 is a much more elegant solution than Git syntax.
Don't forget to set up your .gitignore.
To make the pipeline more robust we use pre-commit. A framework for managing and maintaining multi-language pre-commit hooks.
Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements.
$ poetry add --dev pre-commit
Place the configuration in the root directory inside
# .pre-commit-config.yaml repos: - repo: https://github.com/psf/black rev: 20.8b1 hooks: - id: black - repo: https://github.com/PyCQA/isort rev: '5.6.4' hooks: - id: isort - repo: https://gitlab.com/pycqa/flake8 rev: 3.8.4 hooks: - id: flake8
We run with this configuration Black, Isort, and Flake8 before each commit.
I always document my functions. To do the same, and once you are done writing your function in PyCharm, right after your function definition, start typing a multiline string starting with
""". PyCharm will generate the boilerplate for your function docstring documentation automatically.
To check for these docstrings we will use flake8-docstrings.
$ poetry add --dev flake8-docstrings
$ poetry add --dev sphinx sphinx-autodoc-typehints
docs directory to store your documentation:
$ mkdir docs && cd docs $ poetry run sphinx-quickstart
Then change the list of extensions inside of
# docs/cong.py extensions = [ "sphinx.ext.autodoc", "sphinx_autodoc_typehints", ]
After that let’s create the following file:
.. docs/api.rst API! =============================================== .. automodule:: src.python_boilerplate.main :members:
And reference it inside the
.. docs/index.rst .. toctree:: :maxdepth: 2 :caption: Contents: api
$ poetry run make html
Check the generated HTML files for the docs!
The combinations of Python libraries are truly endless. And that’s the power of Python — laugh in Elm. We didn’t get into CI/CD pipelines and containerization, I opted to keep those for future explorations. And with that, I wish you a Happy new year!