Category Scientific
 

Posts about scientific Python packages or about using Python in science in general.

Subcategories:

None

Book review: NumPy 1.5 Beginner’s Guide

I recently got the chance to review the book NumPy 1.5 Beginner’s Guide by Ivan Idris and published by Packt Publishing.

It covers many aspects of NumPy and also introduces SciPy as well as Matplotlib. The author includes a lot of examples and exercises and also shows the effects of some not-so-easy-to-understand functions using matplotlib graphs.

The book is easy to read, so you should make fast progress in learning NumPy. Overall, it’s a good read for NumPy beginners. Advanced NumPy users, who just want to look up how specific things work, are better of with NumPy’s documentation, though.

Building NumPy, SciPy & Matplotlib for Python 2.7 on Snow Leopard

A few days ago I wrote about how to build SciPy for Python 2.7 on Mac OS 10.6 Snow Leopard.

Usually you want to install NumPy, SciPy and Matplotlib. After reading Installing SciPy/Mac OS X, the Matplotlib installation instructions and the HJBlog you might come to the conclusion, that it’s not trivial to build them on your own and that you better use the 32bit binaries for Python 2.6 or get them via MacPorts.

But actually it’s really easy. The only dependencies that you need to install are Xcode (for gcc and X11) and gfortran.

To simplify the the installation, I wrote a small Makefile that downloads all packages (except for Xcode) and builds/installs them:

  1. Rename* the downloaded makefile to Makefile, open Terminal and cd to the diretory with the makefile.

  2. Download gfortran and start the graphical installer:

    $ make fortran
    
  3. Download and install NumPy, SciPy and Matplotlib:

    $ make
    
  4. Run NumPy’s and SciPy’s test suite and delete the temporary build diretory:

    $ make test clean
    

That’s how the makefile looks like:

# Download all packages to this directory.
TMP_DIR=./PYTMP

# Select Python version.
PYVERSION=2.7
PYTHON=python${PYVERSION}

# Package to download frm http://r.research.att.com/
FORTRANPACKAGE=gfortran-42-5664.pkg

# Select which versions of the packages you want to install.
NUMPYVERSION=1.5.0
SCIPYVERSION=0.8.0
MATPLOTLIBMAJORVERSION=1.0
MATPLOTLIBVERSION=${MATPLOTLIBMAJORVERSION}.0

# Normally, you shouldn’t need change this.
OSX_SDK_VER=10.6
ARCH_FLAGS=-arch i386 -arch x86_64

# Values for some environment variables. You shouldn’t need to change this.
MACOSX_DEPLOYMENT_TARGET=${OSX_SDK_VER}
CFLAGS="${ARCH_FLAGS} -I/usr/X11/include -I/usr/X11/include/freetype2 -isysroot /Developer/SDKs/MacOSX${OSX_SDK_VER}.sdk"
LDFLAGS="-Wall -undefined dynamic_lookup -bundle ${ARCH_FLAGS} -L/usr/X11/lib -syslibroot,/Developer/SDKs/MacOSX${OSX_SDK_VER}.sdk"
FFLAGS="${ARCH_FLAGS}"


default: all

# Create a temporary directory for the build process
_tmp_dir:
    mkdir -p ${TMP_DIR}

# Clean the the temporary build directory
clean:
    rm -rf ${TMP_DIR}

# Download gfortran and start the installer
fortran: _tmp_dir
    cd ${TMP_DIR} && \
    ${PYTHON} -c "import urllib; urllib.urlretrieve('http://r.research.att.com/${FORTRANPACKAGE}', '${FORTRANPACKAGE}')" && \
    open ${FORTRANPACKAGE}

# Download all required packages
fetch: _tmp_dir
    cd ${TMP_DIR} && \
    ${PYTHON} -c "import urllib; urllib.urlretrieve('http://sourceforge.net/projects/numpy/files/NumPy/${NUMPYVERSION}/numpy-${NUMPYVERSION}.tar.gz/download', 'numpy-${NUMPYVERSION}.tar.gz')" && \
    ${PYTHON} -c "import urllib; urllib.urlretrieve('http://sourceforge.net/projects/scipy/files/scipy/${SCIPYVERSION}/scipy-${SCIPYVERSION}.tar.gz/download', 'scipy-${SCIPYVERSION}.tar.gz')" && \
    ${PYTHON} -c "import urllib; urllib.urlretrieve('http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-${MATPLOTLIBMAJORVERSION}/matplotlib-${MATPLOTLIBVERSION}.tar.gz/download', 'matplotlib-${MATPLOTLIBVERSION}.tar.gz')"

# Extract, build and install NumPy
numpy:
    cd ${TMP_DIR} && \
    rm -rf numpy-${NUMPYVERSION} && \
    tar -xf numpy-${NUMPYVERSION}.tar.gz && \
    cd numpy-${NUMPYVERSION} && \
    export MACOSX_DEPLOYMENT_TARGET=${MACOSX_DEPLOYMENT_TARGET} && \
    export CFLAGS=${CFLAGS} && \
    export LDFLAGS=${LDFLAGS} && \
    export FFLAGS=${FFLAGS} && \
    ${PYTHON} setup.py build && \
    ${PYTHON} setup.py install && \
    cd .. && \
    ${PYTHON} -c "import numpy; print 'Installed NumPy', numpy.__version__"

# Extract, build and install SciPy
scipy:
    cd ${TMP_DIR} && \
    rm -rf scipy-${SCIPYVERSION} && \
    tar -xf scipy-${SCIPYVERSION}.tar.gz && \
    cd scipy-${SCIPYVERSION} && \
    export MACOSX_DEPLOYMENT_TARGET=${MACOSX_DEPLOYMENT_TARGET} && \
    export CFLAGS=${CFLAGS} && \
    export LDFLAGS=${LDFLAGS} && \
    export FFLAGS=${FFLAGS} && \
    ${PYTHON} setup.py build && \
    ${PYTHON} setup.py install && \
    cd .. && \
    ${PYTHON} -c "import scipy; print 'Installed SciPy', scipy.__version__"

# Extract, build and install Matplotlib
matplotlib:
    cd ${TMP_DIR} && \
    rm -rf matplotlib-${MATPLOTLIBVERSION} && \
    tar -xf matplotlib-${MATPLOTLIBVERSION}.tar.gz && \
    cd matplotlib-${MATPLOTLIBVERSION} && \
    export MACOSX_DEPLOYMENT_TARGET=${MACOSX_DEPLOYMENT_TARGET} && \
    export CFLAGS=${CFLAGS} && \
    export LDFLAGS=${LDFLAGS} && \
    ${PYTHON} setup.py build && \
    ${PYTHON} setup.py install && \
    cd .. && \
    ${PYTHON} -c "import matplotlib; print 'Installed Matplotlib', matplotlib.__version__"

# Fetch and build NumPy, SciPy and Matplotlib
all: fetch numpy scipy matplotlib
    echo "all done"

# Execute tests for NumPy and SciPy
test: _tmp_dir
    export MACOSX_DEPLOYMENT_TARGET=${MACOSX_DEPLOYMENT_TARGET} && \
    export CFLAGS=${CFLAGS} && \
    export LDFLAGS=${LDFLAGS} && \
    export FFLAGS=${FFLAGS} && \
    cd ${TMP_DIR} && \
    ${PYTHON} -c "import numpy; numpy.test('1', '10')" && \
    ${PYTHON} -c "import scipy; scipy.test('1', '10')"

I have only tested it on my local machine yet. Hope, it helps and you don’t run into any troubles.

* If you don’t rename the makefile, you must pass it’s name to make: make -f Makefile_nsm <target>.

Building SciPy on Snow Leopard with Python 2.7

I recently had some struggle to build and install SciPy 0.8.0 on Mac OS X 10.6 Snow Leopard, but actually it’s quite easy.

I used the instructions on scipy.org and the HJBlog as source. Since there are builds of NumPy 1.5.0 for Python 2.7 available, you don’t need to install fftw and umfpack manually.

You only need to install gfortran from this site. Pick the latest build for Snow Leopard (e.g. this one).

Next, install NumPy with the disk image from SourceForge or with pip:

$ pip install numpy

To build and install SciPy, download the latest version from SourceForge and do the following:

$ tar -xf scipy-0.8.0.tar.gz
$ cd scipy-0.8.0
$ export CFLAGS="-arch i386 -arch x86_64"
$ export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch i386 -arch x86_64"
$ export FFLAGS="-arch i386 -arch x86_64"
$ export MACOSX_DEPLOYMENT_TARGET=10.6
$ python setup.py build
$ python setup.py install

Everything in one command:

$ CFLAGS="-arch i386 -arch x86_64" LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch i386 -arch x86_64" FFLAGS="-arch i386 -arch x86_64" MACOSX_DEPLOYMENT_TARGET=10.6 python setup.py build install

I hope this works for you as well as it did for me.

Collectors 1.0

It took me nearly three months to fix five small issues with the documentation. But now I finally released Collectors v1.0. :-)

You can read everything important in the RC1 posting.

Collectors 1.0-rc1

Collectors made a huge jump von v0.1 to v1.0 over the last weeks, since we added lots of changes and consider what we’ve done as stable. If we don’t find any bugs, we’ll release the final v1.0 at the end of next week. The changes are:

  • [NEW] Documentation!
  • [NEW] Tests!
  • [NEW] Collectors can use different storage backends now
  • [NEW] Storage backend for PyTables
  • [NEW] Storage backend for MS Excel
  • [NEW] Collector.collect() as alias to Collector.__call__()
  • [CHANGE] Monitor is now called Collector
  • [CHANGE] Shortcut functions moved to shortcuts package

You can download and install Collectors via PyPI. If you find any bugs or have ideas for further enhancement, please let us know.

The Documentation can be found here.