Today I’ll be writing about the database technology behind the application and I will reveal some really clever tricks I had to do to get around the very limited functionality of KV. I’ll start with “the why” though.

The first question I am getting from my web3 friends is “why not IPFS/Arweave/put your favorite decentralized solution here”. Don’t get me wrong, IPFS is a great idea. And there are a lot of promising projects in this space, I believe that one day we will be getting a serious contender from this space. But if you ever tried Zitrone, you may have noticed that the app is instantly ready and it shows your favorite events in a manner of milliseconds. That’s the UX I was aiming for and while there are some clever tricks in data loading pattern in the Remix itself, this is beyond any optimization that exist today, including SSR.

Therefore, you need a fast database. KV is one of “edge ready” data sources, the functionality compared to regular databases is super simplified, but your data is distributed across the globe providing ultra low latency. Now, I am not affiliated with Cloudflare in any way - there are other similar solutions like FaunaDB I love equally or Supabase that would fit your needs better for really tabular data. They even solve most of the problems I am going to describe for you. So, why did I pick KV? The Cloudflare pages. They allow you to deploy your Remix application to the very same edge, sitting close to your data. The result is that when you open the browser to open the app, it gets all it needs in a first application layer request. And fetching the rest of the data is super optimized due to the data routing logic of Remix. This is a relatively new and not well known technology stack, but it beats most of the currently used solutions by miles.

Again, I can imagine that Deno Deploy with Supabase would be close, Vercel is working hard on edge technologies as well and SvelteKit is primarily built for this future - it is hard to test all of the options without rewriting huge parts of the code. I am not comparing or benchmarking those.

With “the why” out of the way, we can continue on some obstacles I solved when using KV. Important to understand is that if you distribute the database across the earth, you have to solve the consistency somehow. Or not, which is sort of what Cloudflare decided. They throw away anything you’d expect from database in order to get pure key-value performance. So, just to be sure you understand: if you write the same key/value pair in San Francisco and Prague, one of writes will win and you can’t manage which. This is crazy, right?

Fortunately, there is life saving “listing keys” functionality. It allows you to query all keys based on the beginning of the key part. So what I did is that I am writing keys as granular as possible. In our case, I have an event record that looks like `getbydao:D_D:ID:timeinmilliseconds` and the attendance for this event is a series of other keys that have a prefix `getattendance` - this way I am able to get all attendance keys relevant for an event key. I am also using a metadata functionality - in this case the participant ETH address is saved as metadata in order to get it right from the listing query.

You can say that this is not databases anymore and you’ll be partially right. One can say that what Cloudflare did is more similar to Elasticsearch or Redis stripped from most of the functionality. But if it fits into your use-case and you can figure out how to build application logic around the limitations, KV is hard to beat.

Next time we will return to Remix and I’ll explain how I work with resource routes and how they fit into nested routing when you use same component in different paths.

Today I’ll be writing about the database technology behind the application and I will reveal some really clever tricks I had to do to get around the very limited functionality of KV. I’ll start with “the why” though.

The first question I am getting from my web3 friends is “why not IPFS/Arweave/put your favorite decentralized solution here”. Don’t get me wrong, IPFS is a great idea. And there are a lot of promising projects in this space, I believe that one day we will be getting a serious contender from this space. But if you ever tried Zitrone, you may have noticed that the app is instantly ready and it shows your favorite events in a manner of milliseconds. That’s the UX I was aiming for and while there are some clever tricks in data loading pattern in the Remix itself, this is beyond any optimization that exist today, including SSR.

Therefore, you need a fast database. KV is one of “edge ready” data sources, the functionality compared to regular databases is super simplified, but your data is distributed across the globe providing ultra low latency. Now, I am not affiliated with Cloudflare in any way - there are other similar solutions like FaunaDB I love equally or Supabase that would fit your needs better for really tabular data. They even solve most of the problems I am going to describe for you. So, why did I pick KV? The Cloudflare pages. They allow you to deploy your Remix application to the very same edge, sitting close to your data. The result is that when you open the browser to open the app, it gets all it needs in a first application layer request. And fetching the rest of the data is super optimized due to the data routing logic of Remix. This is a relatively new and not well known technology stack, but it beats most of the currently used solutions by miles.

Again, I can imagine that Deno Deploy with Supabase would be close, Vercel is working hard on edge technologies as well and SvelteKit is primarily built for this future - it is hard to test all of the options without rewriting huge parts of the code. I am not comparing or benchmarking those.

With “the why” out of the way, we can continue on some obstacles I solved when using KV. Important to understand is that if you distribute the database across the earth, you have to solve the consistency somehow. Or not, which is sort of what Cloudflare decided. They throw away anything you’d expect from database in order to get pure key-value performance. So, just to be sure you understand: if you write the same key/value pair in San Francisco and Prague, one of writes will win and you can’t manage which. This is crazy, right?

Fortunately, there is life saving “listing keys” functionality. It allows you to query all keys based on the beginning of the key part. So what I did is that I am writing keys as granular as possible. In our case, I have an event record that looks like `getbydao:D_D:ID:timeinmilliseconds` and the attendance for this event is a series of other keys that have a prefix `getattendance` - this way I am able to get all attendance keys relevant for an event key. I am also using a metadata functionality - in this case the participant ETH address is saved as metadata in order to get it right from the listing query.

You can say that this is not databases anymore and you’ll be partially right. One can say that what Cloudflare did is more similar to Elasticsearch or Redis stripped from most of the functionality. But if it fits into your use-case and you can figure out how to build application logic around the limitations, KV is hard to beat.

Next time we will return to Remix and I’ll explain how I work with resource routes and how they fit into nested routing when you use same component in different paths.