# on cointbot, my cointegration trader

By [bt3gl's symposium](https://paragraph.com/@go-outside) · 2023-01-05

---

tl; dr
------

today i go over a CLI tool and a set of **trading bots** that i’ve written to detect **profitable cryptocurrency pairs** to be shorted or longed on trading exchanges.

these statistical algorithmic strategies are named [**cointegration**](https://www.wallstreetmojo.com/cointegration), which has been around for a long time, for either traditional or decentralized finances.

* * *

🎶 today’s mood
---------------

[https://open.spotify.com/track/1tDWVeCR9oWGX8d5J9rswk?si=8ba65b4175ac4e1f](https://open.spotify.com/track/1tDWVeCR9oWGX8d5J9rswk?si=8ba65b4175ac4e1f)

* * *

🧘🏻‍♀️✨ cointegration strategy for pair trading
------------------------------------------------

pair trading is a classic example of a strategy based on mathematical analysis.

put it simply, when two or more **non-stationary series** can be **combined** to make a **stationary series**, they are said to be **cointegrated**.

in other words, this strategy allows you to find evidence of an underlying economic link for a **pair of securities** (say, A and B) within a **timeframe**. it also allows you to mathematically model this link, so that you can make trades on it.

> 💡 a **_series_** are said to be **stationary** when the parameters of the data-generating process do not change over time.

* * *

### modeling a pair of securities with math

let’s take two random crypto assets, say, **A and B** **futures**. let’s model each of their returns by drawing their **normal distributions** (aka the [**bell curve**](https://www.investopedia.com/terms/b/bell-curve.asp)).

> 💡 **_crypto derivatives_** _are financial contracts that derive their values from underlying assets._ **_futures_** _are financial contracts that_ **_bet_** _on a cryptocurrency's_ **_future price_**_, allowing_ **_exposure without purchasing_**_._

if these two series are cointegrated, there exists some **linear combination** between them varying around a **mean**. in other words, their combination should be related to the same **probability distribution**.

![cointegration of FLOWUSDT and 1INCHUSDT, generated by cointbot](https://storage.googleapis.com/papyrus_images/d0f700477d90216aaa4e3bd997e1f35c261b0a8c046f8f51dc32081e83880580.png)

cointegration of FLOWUSDT and 1INCHUSDT, generated by cointbot

* * *

### the beauty of p-values

**correlation and cointegration are similar but not the same**. for example, correlated series could just diverge together without being cointegrated.

how do we infer cointegration? _we do like the scientists do._

a **p-value** is the probability of obtaining results at least as extreme as the results of a **hypothesis test**, assuming that the [**null hypothesis is correct**](https://www.investopedia.com/terms/n/null_hypothesis.asp).

a p-value of **0.05 or lower** is generally considered statistically **significant**.

cointegrated series can show very **small** p-values but still not be correlated.

* * *

### the trick of pair trading

the coefficients that define stationary combinations of two series are called **hedge ratios**. in practical terms, the hedge ratio describes the suggested **amount of B to buy or sell for every of A**.

because both securities drift towards and apart from each other, sometimes the distance is high, and sometimes the distance is low.

the magick comes from maintaining a **hedged position across A and B**. if both go down or up, you neither make nor lose money. **profit comes from the spread** of them **reverting to the mean**:

*   when A and B are far apart, you short B and long A: when the spread is small, you expect it to become larger.
    
*   when A and B are close, you long B and short A: when the spread is large, you expect it to become larger.
    

* * *

### spread and z-score

we apply a linear regression to calculate the [**spread**](https://www.investopedia.com/terms/s/spread.asp) of these two series, which is simply defined by:

    spread = first series - (hedge ratio * second series)  
    

this gives us that linear combination coefficient, the hedge ratio (this is known as the [**engle-granger method**](https://www.statisticshowto.com/engle-granger-test/)).

however, the spread does not give you an immediate signal for trading. **the signal still needs to be normalized** so it can be treated as a **z-score**, which is the number of [**standard deviations**](https://www.investopedia.com/terms/s/standarddeviation.asp) separating the **current price** from the **mean price**.

traders can look at the **momentum of the average z-score** and takes a **contrarian approach** to trade, to generate **buy and sell signals**. graphically, **positive z-scores lie to the right** of the mean, and **negative z-scores lie to the left** of the mean.

here is an example of a strategy:

*   whenever the z-score < -1, you long the spread.
    
*   whenever the z-score > 1, you short the spread.
    
*   exit positions when the z-score ~ 0.
    

* * *

### “there are three types of lies: lies, damn lies, and statistics”

math is awesome, but…

obviously, any trading strategy comes with advantages and shortcomings (pretty much like [**_any flavor of text editor_**](https://en.wikipedia.org/wiki/Editor_war), you know the drill).

here is a simple picture of cointegration:

![.you are the master of your own life.](https://storage.googleapis.com/papyrus_images/ffdf7b431ac3a21353b71cf733725312d950915fd21f885a52ea8024261a02f6.png)

.you are the master of your own life.

* * *

🧘🏾✨ the cointbot package
--------------------------

the `cointbot` package consists of a CLI and a set of libraries for cointegration pair trading, with support for different market types, parameters, and bots designs:

![](https://storage.googleapis.com/papyrus_images/86637b9ce479c46483aefc897d571d37dfb008c098922251637e5a984736d58a.png)

for example, `Bot1` has the following strategy:

1️⃣ search for **all possible crypto perpetual derivative contracts** in a cex that can be longed or shorted

2️⃣ retrieve their **price history** for a given timeframe

3️⃣ calculate **all the pairs that cointegrated** by looking at **p-values** smaller than a certain threshold

4️⃣ calculate their **spread** and their **latest z-score** signal

5️⃣ backtest to **long** when **z-score < 0**

6️⃣ if the asset is hot, confirm **tokens to be longed and shorted**, within the initial capital

7️⃣ with these close signals, average in **limit orders** or place **market orders**

* * *

### setting up cointbot

to test cointbot, you will need [**a testnet account from bybit**](https://testnet.bybit.com/en-US/). if you want to use any other cex, the code is free (or wait until i have time to implement them).

[**after cloning cointbot**](https://github.com/go-outside-labs/blockchain-science-py), add all the necessary system and trading settings to a `.env` file, and then install the python package:

![](https://storage.googleapis.com/papyrus_images/98b5da6cd9906cc70acc496e5e674ecec114fc79679dabb24d50e604795d2d45.png)

* * *

### ✅ you are now all set to explore cointbot:

![cointbot CLI ](https://storage.googleapis.com/papyrus_images/ea77c0b9199b08de9719ce098d132d23c9a0ead39a4e88a6aca12fb9686566de.png)

cointbot CLI

* * *

🧘🏿‍♀️✨ fetching a perpetual currency’s data
---------------------------------------------

> 💡 _a_ **_perpetual contract_** _is a contract that can be held in perpetuity, i.e., indefinitely until the trader closes their position._

let’s start testing cointbot by running the simplest option, which is simply calling bybit’s API to query the market data for all derivatives (symbols) for a given currency (_e.g.,_ `USDT`):

![](https://storage.googleapis.com/papyrus_images/bb1ea966183e9f3a212ca97835149f806a4896153046a5edc333483e2e3fe926.png)

here is an example of the output:

![.fetching all available derivative's data for USDT.](https://storage.googleapis.com/papyrus_images/7880d9fec981ff8c5f1a90f1def1002c7c90e35653a64f891611fc0bbc4e40aa.png)

.fetching all available derivative's data for USDT.

* * *

🧘🏼✨ fetching price history for a derivative currency
------------------------------------------------------

cute. now let’s get to business and start our cointegration analysis.

the second menu option queries the market price k-lines for all symbols above in a given `TIMEFRAME` and `KLINE-LIMIT`, not only printing them to `STDOUT` but also saving them as `JSON` to `OUTPUTDIR/PRICE_HISTORY_FILE`:

![](https://storage.googleapis.com/papyrus_images/7c846e045e4af42bc03ec3f671a4a5be0cb430346bf2ea3ef5797a4eea9d4cff.png)

> 💡 _in the context of trading, a_ **_k-line_** _represents the_ **_fluctuation_** _of asset prices in a given time frame. it shows the_ **_close price_**_,_ **_open price_**_,_ **_high price_**_, and_ **_low price_**_. if the_ **_close price > open price_**_, the k-line has a_ **_positive line_**_. otherwise, it is a_ **_negative line_**_._

here is an example of output:

![.fetching price history for USDT.](https://storage.googleapis.com/papyrus_images/74f5cdbda1a43dff9a5e37e3e0e2993e8d91b35bb789d1f225fd361b3cd03f30.png)

.fetching price history for USDT.

> 💡 _bybit employs a_ [**_dual-price mechanism_**](https://www.bybit.com/en-US/help-center/bybitHC_Article?id=360039261074&language=en_US) **_to prevent market manipulations_** _(when the market price on a futures exchange deviates from the spot price, causing mass liquidation of traders' positions). the dual-price mechanism consists of_ **_mark price_** _and_ **_last traded price_**_. "mark price" refers to a_ **_global spot price index plus a decaying funding basis rate_**_, and it's used as a trigger for liquidation and to measure_ **_unrealized profit and loss_**_. "last traded price" is the_ **_current market price_**_, anchored to the_ **_spot price_** _using the funding mechanism._

* * *

🧘🏽‍♀️✨ calculating cointegration for the history data
-------------------------------------------------------

with the price history data from the previous step, we can now calculate cointegration for each symbol (for the desired `PLIMIT` , the chosen p-value that defines a "hot" pair:

![](https://storage.googleapis.com/papyrus_images/2da34122e8b3f2b77aec87490c3664a7dbce9e154ac012f6b7c8a6caf0e364d1.png)

here is an example of output:

![](https://storage.googleapis.com/papyrus_images/70b5633a2d9f8d5738557468285268ff98cf69a406a1f8389e642deb533ae8cb.png)

this step will also calculate **p-values**, **hedge ratios**, and **zero crossings**. the resulting Pandas' `DataFrame` is then saved at `OUTPUTDIR/COINTEGRATION_FILE`, sorted by `zero_crossing`.

> 💡 _in statistics,_ **_zero crossing_** _is a point where the sign of function changes. In the context of trading, it determines an_ **_entry point_** _(using the price in relation to the moving average as a_ **_direction confirmation_**_)._

* * *

🧘🏼‍♀️✨ backtesting a cointegrated pair
----------------------------------------

_did it work?_

> 💡 _in the context of crypto trading,_ **_backtesting_** _is accomplished by_ **_reconstructing_**_, with_ **_historical data_**_, trades that would have occurred in the past using rules defined by a given strategy, gauging the_ **_effectiveness of the strategy_**_._

select your favorite asset pair from the previous step, and let’s **backtest** their cointegration by testing the success of the hypothesis (and making some cool plots for their series’ spreads and z-score).

![](https://storage.googleapis.com/papyrus_images/ea8cb35b4490c53db277324c7098b72333bb207174b8e09421aec367852eb145.png)

example of output for `BNBUSDT` vs. `ALGOUSDT`:

![](https://storage.googleapis.com/papyrus_images/94b16d7d356a1f22968ba159359ea36b7263928cf5e926ca61a73972a06902c5.png)

by the way, this command also generates their cointegration plots and backtest data, and saves them at `OUTPUTDIR/`.

> 💡\* **lil tip**: if you are **starting an entirely new run**, **clean up** the current setup with \*`make clean_data`.\*

* * *

🧘🏽‍♂️✨ looking at the top cointegrated pairs
----------------------------------------------

once we have all data from the previous step, we can look at the top cointegrated securities for the given `TIMEFRAME` and `NUMBER`:

![](https://storage.googleapis.com/papyrus_images/0b21c2937fdfb49fa754f2231c7e7146a5c25a3e2257b6ce6db83e8704eb195f.png)

example of output:

![](https://storage.googleapis.com/papyrus_images/6ec1c6ea9abd19c23729a1bd085cee5911484215d719d0d08e90c93c4db6df9e.png)

note that this command automatically generates the backtesting data and plots (similar to the previous option).

### ✅ congrats, you now understand cointegration pair trading. it’s time to move to our trading bots.

* * *

🧘🏾‍♀️✨ testing orderbooks websockets
--------------------------------------

our bot will be connecting to bybit’s through both REST APIs and websockets endpoints. let’s start by testing the last one.

to open a websocket subscribed to a cointegration pair (either for **spot**, **linear**, or **inverse** markets), run:

![](https://storage.googleapis.com/papyrus_images/3343a4a71c0ef115db04fa7c105df9bc4135016948241e0f9a0af1dd8874d373.png)

* * *

### topics for spot market

**spot market** topics are implemented by the `trade_v1_stream()` method, which pushes raw data for each trade ([**API docs here**](https://bybit-exchange.github.io/docs/spot/v1/#t-websocket)).

after a successful subscription message, the first data message (`f: true`), consists of the last 60 trades.

after (`f: false`), only new trades are pushed (at a frequency of 300ms, where the message received has a maximum delay of 400ms).

example of output:

![websockets connection for spot](https://storage.googleapis.com/papyrus_images/d6afb508c0c3c9e51b6a7ca96ab77ba1d4547bfaedc08073e9e8d41283362a70.png)

websockets connection for spot

* * *

### topics for inverse perpetual/futures market

**inverse market** topics are implemented with `orderbook_25_stream()`, which fetches the orderbook with a depth of 25 orders per side ([**API docs here**](https://bybit-exchange.github.io/docs/futuresV2/inverse/#t-websocketresponse)).

![](https://storage.googleapis.com/papyrus_images/bfa56b0508c8610902dd8feaf078161760f10ff210d33de5710fafe68450174d.png)

after the subscription response, the first response will be the snapshot response, showing the entire orderbook.

the data is ordered by price (starting with the lowest buys). push frequency is 20ms.

example of output:

![websockets connection for inverse](https://storage.googleapis.com/papyrus_images/5c5157f1d9fb17d428e53d34286f8ad504f30180a78a9838b9e1702f385f847c.png)

websockets connection for inverse

* * *

### topics for USDT linear perpetual

finally, `USDT` **linear market** topics are implemented with `orderbook_25_stream()`, which fetches the orderbook with a depth of 25 orders per side ([**API docs here**](https://bybit-exchange.github.io/docs/futuresV2/linear/#t-websocketresponse)).

the first response is the snapshot response, showing the entire orderbook.

the data is ordered by price, starting with the lowest buys and ending with the highest sells. push frequency is 20ms.

example of output:

![.websockets connection for linear.](https://storage.googleapis.com/papyrus_images/7c98987a9b4e57fef572caabb8b18a57c557639de3aadfd240177592ae7661c7.png)

.websockets connection for linear.

* * *

🧎🏻‍♀️✨ deploying a cointegrated trading bots
----------------------------------------------

all right, we made it. let’s deploy those cuties.

several bots with different strategies are found inside `src/bots/`. in this article, we will go over the strategy and deployment of `Bot1`. feel free to explore the other bots there, and if you would like to keep up to date with the new ones i am continuously adding, just star the repo, dunno 🤷🏻‍♀️.

by the way, each bot has a different number and configuration settings in the .`env` file (_e.g.,_ `BOT_COINS`, `BOT_MARKET_TYPE`, `BOT_ORDER_TYPE`, `BOT_STOP_LOSS`, `BOT_TRADEABLE_CAPITAL`, and others). before the next step, you should check them out (and understand their effects).

* * *

### high-level strategy for Bot1

this is how `Bot1` gets set up:

![](https://storage.googleapis.com/papyrus_images/c0769c9bd494b5ef5338086b5b6a260d0f433f50541a9a900fbc803e42c82c98.png)

and this is how `Bot1` executes, inside a `while True` loop:

![](https://storage.googleapis.com/papyrus_images/d4076192c6f4692c13500b3839a3e5937f0d4b935f14c4a5063042f565402d79.png)

you should check the code (the main class is called `BbBotOne`), and then spin it up:

![](https://storage.googleapis.com/papyrus_images/a9c5db41ca468953599e01b2e2ea4c89b3bfc441fcf5d5676ff41386b878e4a1.png)

for more details on what happens next, check out cointbot repo 😉.

by the way, you can also have `Bot1` running inside a docker container with:

![](https://storage.googleapis.com/papyrus_images/ce15dbd2a54bba6e12d22542b9ed6bcdcfde968d35a8a19560eb8fd5ed5f5459.png)

* * *

**◻️ motherofbots.eth**
-----------------------

---

*Originally published on [bt3gl's symposium](https://paragraph.com/@go-outside/on-cointbot-my-cointegration-trader)*