Getting A Cryptalgo Running

A Shareable Algo - Give it a Try

A Running Algo in a Shareable Container

Jesse is a framework by Saleh Mirnezami that implements an algo trader on some large crypto markets. You can run it with past data in a backtest - live trading is not yet implemented. Mirnezami wrote a blog post detailing how to get backtesting working in Jesse, which I followed.

I got backtesting in Jesse running reliably in this Gitpod - have a try yourself. If you like, you can take this Gitpod and develop further with it: add exchange data (it currently has candle data from Bifinex and Binance for their Bitcoin contracts), add coins, change parameters, extend or change Mirnezami’s code - it’s a full development environment in a functioning Ubuntu 20.04.

This blog works you through how do create such a Gitpod. What a Gitpod is an why I used it I explain here.

Why Do It?

I wanted to create a algo that I could share with others for Webinars and similar, so participants could follow along if they wished, but without wasting time laboriously working them through the setup.

The Initial Gitpod

Following Mirnezami’s blog post How to write a profitable strategy for algotrading Bitcoin, he first directs you to install Jesse. Doing this in Gitpod leads to a problem later when you realize it requires Postgresql - not something that can be installed by a non-root user easily. Luckily Gitpod provides some base Docker containers, including one for PostgreSQL. See the Gitpod Prebuilds documentation.

Here’s one way to set up a Gitpod for Jesse. First you must sign up to GitHub (free). Start Chrome and install the Gitpod Chrome Extension then go to Gitpod’s basic image. Click the “Gitpod” button the Chrome app just created (top right, next to “Clone or download”). This will bring up a new Gitpod.

In the bottom right after initialization completes, a popup will appear:

Setup Project

Click “Setup Project”. This will produce a right side-panel with a list of 5 steps, the first being “Create .gitpod.yml” - click the button for this, then click “Update Docker Configuration”:

Update Docker Configuration

Choose “Default + PostgreSQL” and “Yes, generate a .gitpod.Dockerfile”. At this point the image will contain a functioning PostgreSQL installation when it is re-started. Note that you can get back to the Project Setup by doing F1 and searching for Toggle Project Setup View.

Continuing the sequence of steps, if you wish, click “Update Readme”. Click “Test Drive Configuration” and then “Push to Branch & Start Workspace”. A popup will appear in the top bar: choose “Fork to my account”

Fork To My Account

This will create a new repo in your GitHub which will be your-github-name/gitpod. You can close the current Gitpod now - even delete it.

Go to github.com, find that repo and click on it to open it. Under Branch: change from master to your-github-name/gitpod-setup. Click the “Gitpod” button - creating another Gitpod instance - this time Postgresql will be installed (test it by opening a Terminal and entering psql - it should go to a PSQL prompt - then \q to quit).

Install Jesse

The actuall Jesse install can now - with some adjustments for the lack of root user - be followed. Python 3 (>3.6) is required, which is available in the image as python3 - mine is 3.8.2. If necessary upgrade pip with

python3 -m pip install --upgrade pip

Jesse relies on talib, a Technical Analysis library. This is a C++ library, exposed via a CPython wrapper. Getting it in the Gitpod is slightly tricky: SourceForge’s downloads don’t directly expose the filename. Prior investigation shows that it is something like https://phoenixnap.dl.sourceforge.net/project/ta-lib/ta-lib/0.4.0/ta-lib-0.4.0-src.tar.gz. Using a version number in this address the same as the one on the download page, download and unpack it:

wget https://phoenixnap.dl.sourceforge.net/project/ta-lib/ta-lib/0.4.0/ta-lib-0.4.0-src.tar.gz
tar xf ta-lib-0.4.0-src.tar.gz

That will create a directory called ta-lib.

So that that the installation persists between Gitpod restarts, this C++ library needs to be installed locally to the workspace, so I suggest first do

mkdir .local
cd .local
mkdir bin lib include man share
export PREFIX=/workspace/gitpod/.local

Now it is time to build the library:

cd /workspace/gitpod/ta-lib
./configure --prefix=$PREFIX
make
make install

This will put all requrired files within the .local hierarchy.

Now for the Python layer. First you must direct it to the ta-lib installation:

export TA_LIBRARY_PATH=$PREFIX/lib
export TA_INCLUDE_PATH=$PREFIX/include
export LD_LIBRARY_PATH=$PREFIX/lib

numpy is required for the Python talib layer - it should be installed with the Scipy packages - useful for other work with Jesse:

python3 -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose

The Python library ZIP file is available from https://github.com/mrjbq7/ta-lib.

cd /workspace/gitpod
wget https://github.com/mrjbq7/ta-lib/archive/master.zip
unzip master.zip
cd ta-lib-master
python3 setup.py install

Because this install must be written to the Python installation which is in the gitpod home directory, it is lost during a Gitpod Stop/Start cycle. Therefore it needs to be repeated by addition to the .gitpod.yml file:

tasks:
  - init: echo "Not doing anything with 'init' right now!"
    command: | 
      cd /workspace/gitpod/ta-lib-master && python3 setup.py install && cd /workspace/gitpod
      export LD_LIBRARY_PATH=/workspace/gitpod/.local/lib
image:
  file: .gitpod.Dockerfile

Finally install Jesse’s Requirements (pip install from the documentation doesn’t work between Gitpod restarts):

python3 -m pip install -r https://raw.githubusercontent.com/jesse-ai/jesse/master/requirements.txt

Now install Jesse:

python3 -m pip install --user jesse

Create The Algo

With Jesse installed, first you create a project:

jesse make-project your-project

which will create the directory your-project (choose an appropriate name for your project).

Now you can follow the process described in Mirnezami’s blog post following from the jesse install.

Share it Around

You should have a functioning Algo in a Gitpod. One of the beauties of the technology is the ease by which you can share workspaces: just right-click on the top-right-hand corner user icon and click “Share Workspace Snapshot”.

Workspace Snapshot

The link produced can be passed to anyone: they will then see the functioning IDE in their browser and can re-run the backtests of Mirnezami’s blog.

For example, my image contains candle data for both Bitfinex and Binance for Bitcoin/USD contracts and is set up to backtest on Binance - easily changed via the Jesse config.py and routes.py.

To see this working, click the “my image” link above, give it a few minutes to start, and in the then run

jesse backtest 2019-01-01 2020-01-01

Subscribe to our Newsletter