`from otherswork import wheel`: looking for useful Python packages

Python is a language where simplicity matters. Thus, many features are only accessible through built-in modules, called the standard library. Also, new useful packages are continuously written since Python emerged from the free software principle. In this talk, I will make a (very) short introduction to Python's standard library, and share my experience in finding and installing packages from Python community -- biased for use in astronomy.

Modules, Packages, and all that...

module is Python code file containing definitions, functions, statements...
package is a structure to share a set of modules.

import this

This talk was based on the python4astronomers article:

Where does Python look for modules?

Multiple Pythons on your computer (virtualenv)

Anaconda environments

Mandatory reading: the standard library documentation

"Python's standard library is very extensive, offering a wide range of facilities..."

—its docs

https://docs.python.org/2/library/

A good way of learning: PyMOTW (Python 2, Python 3).

A few examples

random

import random
import numpy.random as np_rand

abc = list('abcde')

random.sample(abc, 2)
# ['c', 'e']

# np_rand.sample(size=None)  # Return random floats in the half-open interval [0.0, 1.0)
np_rand.sample(2)
# array([ 0.79363627,  0.48012174])

time, datetime

import time

# Regular and 12 hour format
print (time.strftime("%H:%M:%S"),time.strftime("%I:%M:%S"))
# ('19:11:49', '07:11:49')

# Date with full and short year
print (time.strftime("%Y/%b/%d"), time.strftime("%y-%m-%d"))
# ('2016/Jun/15', '16-06-15')

linecache

import linecache

theline = linecache.getline(thefilepath, desired_line_number)

collections

from collections import defaultdict

dl = defaultdict(list)
dl['a'].append(1)

ds = defaultdict(set)
ds['a'].append(1)

from collections import OrderedDict

od = OrderedDict()
od['a'] = 1
od['b'] = 2
od['c'] = 3
od['d'] = 4

for it in od.items():
    print(it)

itertools
argparse

The Python Package Index: `PyPi`

"The Python Package Index (aka ``PyPI`` -- formerly known as the "Cheese Shop") is the preferred hub for publishing Python packages and modules. Python's standard library supports code uploads to PyPI through its ``distutils`` module."

https://pypi.python.org or http://cheeseshop.python.org

`pip`

pip install _package_
# --user : local install (no admin rights)
# -U or --upgrade : upgrade existing installation
# --no-deps : no install of dependencies packages (useful for upgrade)
# --install-option="--prefix=$PREFIX_PATH" redirects the install

pip in embedded in 2.7.9+. If you have an updated version of Python and don't find it, run this command:

python -m ensurepip

Remember: pip installs binaries in addition to the modules. Add this installation path to your $PATH (in unix, is is $HOME/.local/bin).

Lists of useful modules

In its wiki, the Python Software Foundation keeps a list of useful modules.
PyPi Ranking lists projects on PyPI ranked by downloads.

A few choices

tqdm

import time
from tqdm import tqdm

print('# Starting something low...')
for i in tqdm(range(5)):
    time.sleep(1)
print('# Done!')

joblib
tinyDB

virtualenv

virtualenv -p /usr/bin/python2.6 /path/to/myvirtualenv

source /path/to/myvirtualenv/bin/activate

deactivate

Packages for Science

Keynote: State of the Tools | SciPy 2015 | Jake VanderPlas.

ipython
Numpy
Scipy
Matplotlib
Pandas
SymPy
PyMC / emcee

Numba

#!/usr/bin/env python
# -*- coding:utf-8 -*-

import numpy as np
from numba import jit

size = long(1e8)


def my_sum(size):
    result = 0.0
    for i in range(size):
        result += i
    return result

print(my_sum(size))


# jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when function is called.
@jit
def nb_sum(size):
    result = 0.0
    for i in range(size):
        result += i
    return result

print(nb_sum(size))


def np_sum(size):
    return np.sum(np.arange(size).astype(float))

print(np_sum(size))

# running cmd
python -m cProfile -s cumulative numba_ex.py > numba_ex.txt

Python conferences:

Lists of astronomy packages

Guru Google

Google python blah blah or python astronomy blah blah works most of the time.

Remember: Python is not only functions

List of object-oriented features that you might want to look up as you become more experienced:

Generators
Class-level attributes, classmethods, staticmethods
Properties and accessors
Decorators
Metaclasses

Excellent references:

Warning! Be aware of the spirit of Python

"Break free from this subtle destroyer and reclaim a life of passion and purpose": The Spirit of Python.

from otherswork import wheel: looking for useful Python packages