Running rl-swarm on Windows is not very well tested and may require some debugging to set up properly. You first need to install WSL and Linux on your Windows machine.

1. Open PowerShell or Command Prompt (run as administrator)

2. Execute the following command to install WSL and Ubuntu distribution:

   wsl --install
   

3. Restart your computer

4. When starting WSL for the first time, you'll need to set up a Linux username and password

1. Open the WSL terminal (through the Start menu or by typing wsl in Command Prompt)

2. Update the package list:

   sudo apt update && sudo apt upgrade -y
   

3. Install Python and other necessary tools:

   sudo apt install python3 python3-pip python3-venv git curl -y
   

Since the Ubuntu system has Node.js 12.22.9 by default, which is outdated and incompatible with js-cookie@3.0.5 used by rl-swarm, you need to manually upgrade Node.js to v14+:

1. Remove the old Node.js (recommended but optional):

   sudo apt remove nodejs npm
   sudo apt autoremove
   

2. Add the NodeSource repository (for Node.js 22.x LTS):

   curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
   

3. Install Node.js:

   sudo apt-get install -y nodejs
   

4. Verify the installation:

   node -v  # Should show v22.x.x
   npm -v
   

In the WSL terminal, execute:

git clone https://github.com/gensyn-ai/rl-swarm.git
cd rl-swarm

rl-swarm uses an interactive installation script that will guide you through the installation process.

In the project directory, execute:

./run_rl_swarm.sh

The installation script will:

• Verify if your environment meets the requirements

• Install necessary Python dependencies

• Set up and configure rl-swarm

During the installation, when asked for confirmation, press Enter or input 'Y' to continue.

After installation, rl-swarm will start a local web interface. By default, you need to manually navigate to http://localhost:3000/ (if you're running on a VM, you'll need to manually navigate to this URL).

If you're running the model on CPU and encounter a Python RuntimeError, or if your training progress seems to have stopped, you can try the following solutions:

1. Set this experimental fix:

   export PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 && ./run_rl_swarm.sh
   

2. Add the following training argument in your device configuration file: Edit ./hivemind_exp/configs/<directory_relevant_to_your_device>/grpo-qwen-2.5-0.5b-deepseek-r1.yaml Add parameter: max_grad_norm=0.5

3. Use floating point 32 instead of bfloat16 to train your model. This can be changed in the device configuration file.

rl-swarm is fully open and permissionless, meaning you can run it on a basic consumer laptop at home or on a powerful GPU in the cloud. After running a node, you can launch a new swarm or join an existing one.

rl-swarm uses Hivemind for distributed training, allowing models on different nodes to collaboratively learn and improve.

• Windows support for rl-swarm is not as robust as Linux, and may require some debugging.

• If you have multiple GPUs and want to run multiple nodes on one machine, you'll need to manually adjust the configuration.

• Different training speeds between machines are normal.