Packaging the agent-based model Python code for friends and colleagues

In this post I will walk through packaging the agent-based model Python code with Anaconda’s conda build scripts. The big idea is to make the simulation apparatus accessible to folks with some Python chops and reasonable facility with the Jupyter Notebook, IPython, or command line environment. I will put all of the relevant links at the end of the post. There are many variations on how to build conda packages. I will discuss how I did it and leave the remaining variations to the documentation. For example, I will build from a GitHub repository; but there are other ways to do the build without a GitHub account.

Step 1: Tag a release in GitHub

I tagged the master branch with version 0.0.1b. You can see the result here.

Step 2: Write a setup.py file and put it in the root.

You can see the directory structure on my GitHub page for pyziabm. The file makes use of setup, which is imported from setuptools. Many of the keywords are optional, but Conda requires the “packages” keyword to do the build. Rummaging around the setup documentation could prove useful if you plan to do more complicated builds – with Cython, for instance.

from setuptools import setup

setup(
   name='pyziabm',
   version='0.1.0b',
   description='Zero Intelligence Agent-Based Model of Modern Limit Order Book',
   author='Charles Collver',
   author_email='jackbenny@always39.com',
   packages=['pyziabm'],
   url='https://github.com/JackBenny39/pyziabm',
   license='BSD3'
)

 

Step 3: Create a local build directory with three files: meta.yaml, build.sh, bld.bat.

I named the build directory with the package name (pyziabm), but it doesn’t matter. The meta.yaml file specifies the information Conda requires to run the build. I chose to ensure that the user has all of the necessary packages to create hdf5 files by adding them to the requirements: run: specs. The only test is to ensure that the package imports upon completion. You can run other tests as well. I chose to run them (manually) in Eclipse/PyDev before pushing to GitHub, merging and tagging.

package:

  name: pyziabm
  version: "0.1.0b"

source:

  git_rev: v0.1.0b
  git_url: https://github.com/JackBenny39/pyziabm.git

requirements:

  build:

    - python

  run:

    - python
    - numpy
    - h5py
    - pytables
    - pandas

test:

  imports:

    - pyziabm

about:

  home: https://github.com/JackBenny39/pyziabm
  license: BSD3
  license_file: LICENSE

 

The build scripts, bld.bat and build.sh, run setup.py install with some flags. See the install flags link below for more details.

build.sh and bld.bat

$PYTHON setup.py install --single-version-externally-managed --record=record.txt

%PYTHON% setup.py install --single-version-externally-managed --record=record.txt
if errorlevel 1 exit 1

 

Step 4: Navigate to the local build directory and run the build

This step typically produces a lot of output to the screen. While it is an interesting read, the important thing to check is if the build was actually successful! If it was, the output will contain the full path to the package. The output package file is usually buried somewhere in your anaconda directory structure. Copy that path, you will need it later.

pyziabm:~$ conda-build .

The dot is not a period. You can run conda build from another (enclosing) directory by specifying the full path.

Step 5: Install the new package and run it through its paces

somepath:~$ conda install --use-local pyziabm

 

Step 6: Push the package up to the conda cloud

This step requires you to have an account with Anaconda. It is free and painless. It is also unnecessary. You can just as easily distribute the package (via email attachment, for example) and have your friends conda install from local. Assuming you are determined to make your package available in the conda cloud and have created an account, then:

~$ anaconda login

~$ anaconda upload that/long/path/you/copied/pyziabm-0.1.0b-py36_0.tar.bz2

 

Step 7: Repeat steps 3 – 6 for other operating systems

I built from Windows then repeated from my Mac and an EC2-Ubuntu instance on AWS. The documentation claims you can use the “conda convert” command to generate the package for other OS, but it didn’t work for me.

The packages I built and uploaded to the conda cloud are available at: https://anaconda.org/dennycrane/pyziabm

As always, feel free to leave a comment or ask questions.

Useful links:

Tagging a Release in GitHub: https://help.github.com/articles/creating-releases/

distutils.setup keywords: https://setuptools.readthedocs.io/en/latest/setuptools.html#basic-use

install flags: https://setuptools.readthedocs.io/en/latest/setuptools.html#command-reference

Conda build: https://conda.io/docs/user-guide/tasks/build-packages/index.html