I'm a big fan of cheap things that just work. Servers are one of them.

Not very cash money of you

According to industry reports, 33% of organizations are spending over $12 million annually on public cloud services, with small to medium businesses averaging over $1.2 million yearly. A good chunk of this is going to serverless or overpriced servers.

Take a new trendy web app (meow?) that handles steady traffic throughout the day. I'll do some quick calculations based on Vercel Pro's pricing to compare costs.

Let's say you:

• have 4 other team members ($20/seat)

• get around 200,000 function invocations per day, or 6,000,000 per month ($0.60/M reqs minus 1M included reqs)

• use 256MB RAM and 150ms on average per request, 90% of which is idle time ($0.18/GB-hr minus 1,000 GB-hr included)

• get 100GB in and 1.5TB out ($0.06/GB Fast Origin Transfer minus 100GB included, $0.15/GB Fast Data Transfer minus 1TB included)

For simplicity's sake, let's say we have enough scale for Fluid Compute to fill up all the idle time with other requests.

That's about $20 (plan) + $80 (seats) + $3 (invocations) + $832.50 (CPU time) + $30 (Fast Origin Transfer) + $75 (Fast Data Transfer), which sums to just over 1 thousand US dollars per month.

Meanwhile, let's compare that pricing to a more server-based stack.

A 12GB Budget VM ($11.99/mo) from Altivox, which comes with 4 vCPU cores and 1TB included bandwidth can handle 200,000 rpd (2.3 rps) very very comfortably.

Extra bandwidth is $4/TB, so that's an extra $2.40 in transfer costs. Total server cost is $14.39, a large drop compared to the more 'modern', 'efficient' serverless stack.

What about latency / being close to my users?

Spin up more servers in different regions. Altivox has limited scale, but there are various other inexpensive hosting providers like Hetzner and OVH.

To be within 200ms of your users, you only really need five servers: Europe, Tokyo, US East, US West and Australia. Even without calculations, you can already tell that it beats Vercel easily.

Server management is not hard

Serverless companies actually make money off telling developers that the alternatives are difficult to manage, rather than providing high cost efficiency, as shown by the estimates above. This is also false due to modern tools like Docker and Kubernetes which make managing your apps easy.

Deploying an application today looks like this:

docker build -t myapp .
docker run -d -p 80:3000 myapp

That's it. No dependency issues, no environment mismatches, no configuration drift. Want zero-downtime deployments? Docker Compose with a load balancer handles it fine. Need even more scale? Get another VM, or maybe even bare metal.

Compare that to debugging why your serverless function works locally but fails in production because of Node.js version mismatches between your development environment and Vercel's runtime. These environment inconsistencies literally cannot exist when you control the entire stack, especially when combined with devcontainers or other reproducible dev environments.

Complexity is transferred

Generally, serverless doesn't eliminate complexity, rather, it shifts it to a different place. Instead of managing servers, you're now managing:

• Function timeout limits and memory configurations

• Cold start optimization strategies

• Vendor-specific deployment configurations

• Configuring efficient CDN and caching etc etc.

You've traded straightforward server administration for vendor-specific serverless cloud architecture which changes based on your choice. Though the learning curve for a totally new dev is slightly simpler than managing Linux servers, these are skills that are solely useful for proprietary platforms, which leads to...

Vendor lock-in

Traditional server applications, especially containerized ones, are portable. A Docker container running on DigitalOcean will run identically on Hetzner, OVH, or your laptop. Your nginx config, database setup, and deployment scripts work everywhere.

Serverless apps become strongly architecturally-coupled to their platforms. When you use AWS, you don't just use AWS Lambda, you use API Gateway's request/response format, CloudWatch's logging system, IAM's permission model, and tons of other AWS-only services. When you use Vercel, it's the same thing. Moving away ends up being a total rewrite of your code!

This lock-in has real costs beyond just switching friction. It limits your ability to negotiate pricing, constrains your architectural choices, and makes you dependent on a single vendor's roadmap and reliability.

Building Linux skills

Server management teaches fundamental concepts: how operating systems allocate resources, how networks route traffic, how databases store and retrieve data, how applications scale under load. These concepts remain relevant even in serverless development, because they're not perfectly abstracted away!

The skills learned by developing with servers are transferable and valuable throughout a dev career. If you understand Linux, networking, and system administration, you become a better developer overall, regardless of your deployment target.

A practical alternative

Serverless is not always a bad option. For very event-driven workloads or applications with extremely variable traffic patterns, serverless can make sense. Do the numbers before making important decisions!

But for the majority of web apps: APIs, web services, background workers; traditional deployment models offer better pricing and more control.

Modern server management isn't the nightmare it was 10 years ago. With Docker, excellent VPS providers, and mature tooling, running your own infrastructure is more accessible than ever. You'll likely save money, gain flexibility, and build valuable skills.

Major cloud providers profit from your dependencies and won't teach you these fundamentals. But for most applications, well-managed servers outperform serverless on cost, perf, and maintainability while keeping you in control of your own technology stack.

Ready to try it out for yourself? Coolify offers an excellent self-hosted PaaS experience combined with a solid server like those provided by Altivox.

For example, USD Coin and Wrapped Bitcoin are smart contracts (a program running on the Ethereum network). They store user balances in their own internal state and expose a standard API called ERC-20 for modifying it. Other smart contracts can talk with USDC, WBTC, and every other token using the same ERC-20 API.

DeFi (Decentralized Finance) has been called 'money legos' because of this composable nature. An example of a basic DeFi protocol (here in DeFi, protocol means something like a stateful system rather than a way of communicating) is Uniswap, which is an exchange. It holds balances of tokens and lets people swap between them according to its pricing formula.

When someone wants to buy WBTC, they first approve an allowance to Uniswap, then Uniswap pulls the funds and returns the corresponding amount of WBTC according to the pricing formula. It works for every token, because all tokens implement the same common APIs which Uniswap uses too.

Another example of a basic DeFi protocol is Aave. It's a lending platform that lets people take out loans against their crypto collateral. This means they don't need to sell if they need to pay bills, for example. They can simply borrow against it.

A more complex example is Yearn, which is a yield optimizer which takes your money and rotates it between different strategies to get you the best rates on your savings. It talks with Aave, and Morpho, and Euler, and all those different DeFi protocols using their APIs.

Not only is it composable, Ethereum DeFi is permissionless. This means that no one can stop you from building your own smart contract to interface with others for your own purposes.

And not only can you build finance, but you can also build governance onchain. DAOs are smart contracts that can manage money by vote of your organization. Arbitration protocols like Kleros allow people to make escrow deals. Etc.

Pretty much, Ethereum is the New Internet: a world of relationships between autonomous, open source code. Traditional finance is sort of like this, except instead of code, paperwork is used to compose businesses together. + Ethereum is much newer than traditional finance, so lacks a lot of the tech debt it has (clearinghouses, settlement, all those hacky solutions, ancient regulations)

This is not only limited to primarily financial assets. As the world goes onchain, more and more things will be 'tokenized', i.e. making it usable through these APIs.

An instance right now of a somewhat non-financial asset is ENS Domains. The fact they are tokenized means they can be held by DAOs or multisigs (smart contracts that require multiple different signatures to confirm a transaction) natively, allowing groups of people to manage these assets collectively.

Decentralization wins because decentralization lets more people build together. The more we build, the stronger we become. No need to for permission, just open APIs that anyone can write smart contracts to use.

Now wouldn't that be something great?

"Just because you're paranoid doesn't mean they aren't after you."

This is an incomplete, definitely non-exhaustive set of security guidelines for securing DeFi protocols. Every protocol is different and even if you follow every single rule in here to the letter, you may still get hacked. The job of this guide is to reduce the chance of getting hacked by North Korea.

This guide is meant to be used in combination with other guidance.

This guide is a work in progress and will be updated as I learn more.

(If you'd like to see more of what we do, check out our website.)

1. Formally verify your code. It's not that hard unless your devs suck.

2. Get audited by a good auditing firm that isn't CertiK.

3. Test rigorously. Aim for 99% code coverage, with fuzz, integration and invariant tests.

4. Use circuit breakers to slow down or pause any suspiciously large outflows.

Development process, bug bounties, war rooms, multisigs, et cetera.

1. Use VS Code dev containers to sandbox your development environment.

   • DevPod or any homebrew Docker solution is acceptable. Just ensure it has no way to escape the sandbox.

   • Another benefit of this is that tooling across all developers is the same, so no more "works on my machine"

   • Or use a separate machine entirely for your dev work and connect to it via SSH. Maybe even GitHub Codespaces (ew).

2. Fucking use Linux.

   • An immutable distribution of Linux is preferred. MacOS is acceptable.

   • They don't usually build malware for Linux.

   • It's just generally a better experience for development.

3. Force all your devs to stay up to date with all system updates.

4. Always set ownership in your contract initializers to a protocol DAO or multisig.

5. Never reuse a deployment key for a different purpose.

6. Never put deployment keys in environment variables.

7. Or you could use Truffle Dashboard.

   • Unmaintained

   • From my experience, has some compatibility issues with Foundry

8. Ensure all developer git commits are signed. I prefer SSH keys, but you can use GPG.

9. Open-source all code running in production.

10. Pay out your bug bounties.

    • It's not rational at all to refuse to pay or be stingy, it's just retarded. You're setting up incentives for grayhats or whitehats to become blackhats.

11. Always do the Kim Jong-Un test.

    

    Pop Punk

    @PopPunkOnChain

    

    This is how you identify a North Korean engineer.
    
    I asked him to say something negative about North Korea and Kim Jong Un and he immediately deleted the chat.

    

    9,003

    6:34 PM • Aug 26, 2024

12. Keep your development machine as minimal as possible. Browser, VSC, Docker, that's it.

    • If you must install apps, use Flatpak.

13. Make an incident response plan for what exactly to do in case of a hack.

14. Add your contact details to crytic/blockchain-security-contacts.

You can't treat building a financial app in the same way you build any other web app. NORTH KOREA IS YOUR ADVERSARY. Therefore.

1. Set up an IPFS instance of your dapp.

   • Recommend not to set it up using Fleek or any managed provider

2. Test every frontend update before it goes live using multiple wallets and browsers.

   • You want to induce malicious behavior if it's present before it gets shipped out.

   • Watch for network requests to any domains you don't recognize.

   • Watch for any behavior you don't recognize, such as asking for infinite approvals to suspicious contracts.

   • Don't test using dev wallets.

   • Make yourself look like a new user every test.

3. Nudge your users away from MetaMask.

   • MetaMask promotes harmful practices such as blind signing

   • Wallets such as Rabby or Rainbow which do transaction simulation are preferred

4. Don't block VPNs. Please.

   • This isn't technically a security concern, it's just very harmful to us anons

5. Make your web app immutable

   • This reduces the impact of a successful domain hijack only to new users or to those who have cleared cache

   • Ensure there is a way to update the frontend from inside

6. Onchain is better than offchain. Hold your domains in a multisig using 3DNS.

   • Gives additional protection against having to trust 1 dev with domain ownership

7. Follow Cloudflare's guide to preventing domain hijacking.

TL;DR: You can use Unicode characters that mimic colons in markdown links to create very realistic looking links. I use the Hebrew Punctuation Sof Pasuq character:

Then just use it in a Markdown link in the text. For example, [https׃ //google.com](https://youtube.com) - the first colon is not actually colon.

1. "Oh cool Discord made Markdown links"

2. "How should I rickroll people with this...?"

3. "Nope anything starting with http:// or https:// as text doesn't work"

4. "Hey wait what if I replaced one of the characters at the start"

5. GG

I then proceeded to abuse this by trolling people with fake links

As a paranoid cypherpunk unfortunately living under an authoritarian government, I have some privacy requirements: only Windscribe, my own devices, and some very select services are allowed to see my IP; and connections between the Pi and my other devices must always be encrypted.

This ruled out Tailscale (although that could be useful especially with the integration with Mullvad as exit nodes) as that poses the risk of leaking my IP address to things linked to my digital identity.

It took me quite a bit of finagling and research to get this setup to work just right. I hope my experiences can save you time even if you don't have an extremely niche use case.

I have two VPNs active at once on my devices: Windscribe and Nebula. Windscribe routes my outgoing traffic through their network, while Nebula connects the devices together, somewhat like Tailscale.

On the Windscribe page, I used the config generator to generate me a WireGuard config for a certain location. I simply copied this config to my Raspberry Pi and started the connection with wg-quick (I'm not gonna go into details here, should be trivial enough with a Perplexity search)

Nebula requires one always-online server called a lighthouse with a static IP (I believe it may work with dynamic DNS, but untested) in order to direct devices to reach each other. I set it up on my Raspberry Pi and opened up a port on my router to make it reachable from the Internet.

Following the Nebula docs worked out pretty well, got nothing extra to say there.

As Nebula doesn't come with systemd services, I had to write them myself. On my Raspberry Pi, I created a new system user account called nebula, gave it ownership of /etc/nebula/ and set it up as a service:

[Unit]
Description=Nebula Overlay Networking
After=network.target

[Service]
Type=simple
ExecStart=/bin/nebula -config /etc/nebula/config.yaml
Restart=always
RestartSec=10
User=nebula
Group=nebula
AmbientCapabilities=CAP_NET_ADMIN

[Install]
WantedBy=multi-user.target

I did a similar one for my laptop, setting up Nebula as a service that runs after Windscribe starts and making sure to create the necessary user accounts:

[Unit]
Description=Nebula Overlay Networking
After=network.target

[Service]
Type=simple
ExecStart=/bin/nebula -config /etc/nebula/config.yaml
Restart=always
RestartSec=10
User=nebula
Group=nebula
AmbientCapabilities=CAP_NET_ADMIN

[Install]
WantedBy=multi-user.target

Additionally, I had to add my chosen range to Windscribe's split tunneling on my laptop in order to allow connections to go through Nebula. (Why did I use TEST-NET-2 and not the designated CGNAT range? Scroll further)

When I change my Windscribe location, I have to also restart the Nebula service, but aside from that it's perfect.

On my Android, I use a work profile to run Nebula while using the main profile with Windscribe. This bypasses the Android restriction of only one VPN active at a time, so now I can have most of my apps behind Windscribe and the selfhosted apps on Nebula. Nice!

It took me a while to get it done, but:

sudo ufw allow ssh
sudo ufw allow 
sudo ufw default deny outgoing

On my first try, I decided to set Nebula up on the CGNAT range. Too bad Windscribe uses it.

I had to redo my Nebula network to use TEST-NET-2, which is 198.51.100.0/24. So yeah, double check your ranges.

Windscribe doesn't support split tunneling IPv6 ranges unfortunately. (and after I finished setting up Yggdrasil!) That was one of the things that stopped me from using Yggdrasil, the other being the high latency.

https://github.com

Support IPv6 ranges in split tunneling · Issue #176 · Windscribe/Desktop-App

Is your feature request related to a problem? Please describe. I'm trying to use Yggdrasil, which is a mesh network that uses an IPv6 range (0200::/7). I don't want to have to turn off the Windscribe firewall to access it. Describe the s...

Now I've got the basic networking out of the way, I am finally free to finally get some selfhosted services up.

This journey taught me CIDR notation and a little more about being a sysadmin. It would never have been possible (or at least, would take 10 times more time) without the help of my beloved Claude and Perplexity.

During my research, I came across a list of different projects. I opted for Nebula, but here's a small inexhaustive list of the stuff I came across during my search:

• Innernet, a mesh network based on CIDRs

• Headscale, an open source Tailscale control server implementation

• tinc-boot, a wrapper over tinc, which is a full-mesh, auto-healing VPN system supporting many many OSes

• A blog post from fmac titled 'Nested WireGuard tunnels to my home'

• gluetun, a tool for running a VPN (the privacy tool kind, not the access tool kind) inside Docker

• wireguard-go, a userspace implementation of WireGuard

• NetBird, a mesh net with a full GUI and centralized access control like Tailscale (actually now that I'm writing this post I think I should've just used this but whatever i've set it up already)

Originally thought about in the TLX Discord.

Currently, TLX leveraged tokens work by rebalancing to maintain a leverage within a range to avoid liquidations, which is what causes "volatility decay". It only rebalances after hitting the leverage bound (or on deposit and withdrawal) and also incurs swap fees.

Volatility decay would not be a factor if there were perfect rebalancing, i.e. maintaining a perfect constant leverage and having no swap fees.

My scheme: The synth aims to track a perfect constant-leverage index. People can borrow against collateral to mint these synths (let's say ETH2L) and short it by selling ETH2L. On the other side, people can long by buying ETH2L off the market.

Shorters get liquidation risk while longers don't, which could be an issue for getting people to mint in the first place, but they could be compensated by trading fees and fees on the token itself.