October 24, 2018

Python dev workflow on macOS

Here is the new workflow in place of the old popular one using virtualenv, pip, requirements.txt and virtualenvwrapper.

Tools

pyenv for Python version installation and switch
Pipenv for virtualenv creation and package management for each project
Pipenv-Pipes for Pipenv environment switch

Pipenv is the central tool. pyenv provides specific version of Python (and pip) for Pipenv to initialize its virtualenv. Pipes is just a handy tool for fast navigation between Pipenv’s virtualenvs (like virtualenvwrapper’s workon), which may be built into Pipenv in the future.

Basic workflow

pyenv

To install:

$ brew install pyenv

then add eval "$(pyenv init -)" to shell (e.g. ~/.zshrc)

To show available Python versions to install:

$ pyenv install --list

To install a Python version:

$ pyenv install <version>

Pipenv

Install Pipenv:

$ brew install pipenv

To install environment with specific Python version other than system default, this version must exist in system, so we can use pyenv to install the version first, then switch to its context.

$ pyenv shell <version>
$ pipenv --python <version>

After this, pyenv will be of no use because corresponding Python and pip have been installed into Pipenv’s context, unless you update to a new Python version.

Tips

Create Pipfile from requirements.txt

To use Pipenv for an old project with requirements.txt available, we can first

$ pipenv install

Pipfile will be automatically created from requirements.txt, with all packages installed with specific version numbers.

Second, run

$ pipenv graph

to show a dependency tree graph, from which we can identify less packages that we want. Then we can manually edit Pipfile to keep only these key packages. Note that it does not need to make a Pipfile without any redundant (i.e. all packages are leaf nodes in the dependency tree).

Last, run pipenv install again.

Time saving with `--skip-lock`

Uninstall packages that are not in Pipfile

Manually deleted packages in Pipfile would not be automatically uninstalled, we can use

$ pipenv clean

Sublime Text syntax highlighting of Pipfile

(Tested for ST 3)

Install package TOML for TOML syntax highlighting. (Pipfile uses TOML format.)
Install package ApplySyntax for custom syntax hightlighting for non-default format
Add the following to ApplySyntax settings:

"syntaxes": [
    {
        "syntax": "TOML/TOML",
        "rules": [
            {"file_path": ".*\\Pipfile$"}
        ]
    },
    {
        "syntax": "JavaScript/JSON",
        "extensions": ["Pipfile.lock"]
    },
]