Tools

Tools

Here are the tools we like. We will only discuss some of them in the tutorial.

make

We use GNU make to specify the whole sequence of operations from raw data to a pdf document. Makefiles record all parameter settings such as seeds for random number generators. We use make to separate source code from derived data.

Python

In 2001, we started using python because the code is easy to read. Since then it has become popular, and in some fields a standard. To separate stages of processing we write temporary pickle files. In particular we segregate all plotting into scripts that use matplotlib.

Git

We use git in our work generally. Today, we will ask you to use git branches to keep exercises using various tools separate.

yapf

(Yet Another Python Formatter) is an open-source code formatter that we use to automatically coerce code to follow Google’s Python style.

mypy

Mypy is an optional static type checker for Python.

pylint

Analyzes and scores python code against a user defined style guide, pylintrc.

sphinx

Sphinx is a documentation generator or a tool that translates a set of plain text source files into various output formats. We use the autodoc extension that can import the modules we are documenting, and pull in documentation from docstrings in a semi-automatic way.

pytest

The pytest framework makes it easy to write small, readable tests, and can scale to support complex functional testing for applications and libraries.

pdb

pdb is Python’s built-in interactive source code debugger.

cProfile

cProfile is Python’s built-in, recommended tool for deterministic profiling. We use it to find where long running programs are spending the most time.

Cython

Cython is an optimizing static compiler for both the Python programming language and the extended Cython programming language (based on Pyrex). It makes writing C extensions for Python easy enough to be plausible.

cython.parallel

Cython supports native parallelism through the cython.parallel module. It currently supports OpenMP. We use it ease critical bottlenecks.

sed

Sed is a command line stream editor that we use to extract documentation from makefiles in response to “make help”.

LaTeX

By using make to govern typesetting, we can ensure that reported computational results are products of the most recent version of our source code

Nix

Nix is a package manager and build system that parses reproducible build instructions specified in the Nix Expression Language. Nix stores results of builds in unique addresses specified by a hash of the complete dependency trees. By working in environments specified by such builds, we control the dependency tree and ensure reproducibility.