Cover photo

Building a Voice-Controlled Cooking Companion

Pairing OpenClaw with Embedded Gadgets

How It Ended

post image
where it sits now in my kitchen

My Cooking Companion currently resides plugged in on my kitchen countertop. When I need to retrieve a recipe, I simply switch on the power, click the top "talk" button, and say, "get so and so recipe." After an admittedly long 5 to 12 seconds, it responds with the required ingredients, then the first step. To continue, I tap the side "next" button, and my sous chef moves on to the next step for me.

While relatively simple, and completely replaceable by a phone or Google Home, this device was quite fun to make. It started as an OpenClaw cooking chat bot about a month ago, and that OpenClaw instance gradually built a fork of itself with API endpoints for requesting recipes by voice, and a three.js frontend to simulate the hardware that would eventually use these endpoints. This laid the groundwork for an actual embedded device spec, physical prototype, and eventually an encased assembly.

While not the first DIY device I've ever 3D printed or assembled, this is the first I've designed from scratch. Because of this, I figured I would go through the process, key learnings, and pitfalls here. I'll keep it relatively short in this post, and I'll link to more detailed specs for anyone who wants to actually follow along.

How It Started

post image
my first attempt at simulating the hardware

As mentioned above, this project started as someone else's OpenClaw cooking assistant agent, which they built as part of RaidGuild's April cohort. That original agent is here, but I should note that because of a cohort sponsorship, this agent is meant to be deployed through Pinata's Agent Templates.

For my own cohort project, I wanted to make something hardware-oriented, but within the constraints of the sponsorship. So I deployed the cooking assistant agent through Pinata, and began prompting it to fork its own original repo, and build off of itself. To do this I created a GitHub user for the bot, generated a temporary PAT, then had it cherry-pick all the relevant template code into a new template repo.

I began with the following features:

  • SQLite-backed recipe, food-event, and voice-session persistence

  • /query for JSON text / next-step requests

  • /query-audio for multipart spoken queries and JSON next-step requests

  • generated MP3 serving via /app/api/audio/[name]

  • generated-audio cleanup script under workspace/scripts/prune-generated-audio.sh

I used query for initial text-based Postman testing, as well as what was to be hit when the "next" button was pushed. /query-audio was reserved for the "talk" button. The flow for /query-audio went like this:

  • Client hits the endpoint with multipart data primarily containing some common type of audio file

  • OpenAI's whisper-1 model is used to transcribe the audio

  • OpenAI's gpt-4.1-nano is used to figure out intent, and general routing to retrieve the correct data

  • OpenAI's gpt-4o-mini-tts is used to generate a response audio file

  • The response audio file is stored temporarily in the agent workspace

  • When the response gets to the client, the client finds the URL for the temporary audio file, downloads it, and plays it to the user

  • All audio files are pruned daily

Once I got a bare-minimum working, I decided to create a simulated hardware page to test the UX. This may seem like overkill, but it actually did help quite a lot in improving the quality of responses, and for getting a feel for how I'd actually use the device. The first version (see image above) was completely made by OpenClaw, and to be honest, it was very time consuming and expensive. Remarkably, it was actually faster to learn how to use Tinkercad, and make my own 3D model from scratch.

post image
my second attempt at simulating the hardware

The second attempt turned out much better, and as I realized midway through 3D modeling, it could be used as the file for the 3D printed case too. Something I realized through this process is that simulating things accurately early on saves you a lot of time later on. Planning for this to be the first of many of these types of projects, which means much more attention will be put on fast and accurate simulations in the future.

The Build

Warning: soldering is requiring to properly follow along

For those who don't want to use a Pinata Agent for the backend, I've since created a bare-minimum API on its own, which can be hosted anywhere.

Picking the Parts

Before I (OpenClaw) wrote my endpoints, case, and overall simulation, I actually already had all my parts picked out (important before simulating, of course). Three sets of parts were need: 3D printed pieces, electronic components, and miscellaneous things to connect everything together.

My 3D printed pieces were:

  • Top enclosure

  • Bottom enclosure

  • "Talk" button parts

  • "Next" button parts

My electronic components were:

  • Microcontroller

  • Mic

  • Amp

  • Speaker

  • 2 buttons

  • 1 LED light

  • USB power cable

And miscellaneous parts were:

  • Wires

  • Breadboard for prototyping

  • Perfboard for final assembly

You can find a reference for the exact electronic components I used here. Not including negligible parts that you can bulk-buy like wires, buttons, perfboard, and the LED, this only cost me $23.54.

Breadboarding

post image
first working version of the cooking companion via breadboard

This project was my first time using a breadboard, and as I'm sure anyone who does maker stuff will point it, it's an absolutely must-have for anything you are designing from scratch. Using a breadboard is relatively intuitive, but I found this video useful for filling in my mental gaps. I also took a 1-week detour to go through all tutorials in the starter kit my breadboard came with (the ELEGOO UNO Project Super Starter Kit).

Some extra bits I did and learned from this process:

  • You can flash your program to the ESP32 using most common languages. I used Arduino's IDE, and so flashed .ino files. You can see the exact code I used here

  • This may seem obvious, but you should solder on all the component headers before using the breadboard; I spent well over an hour wondering why my speaker wasn't working due to my loose-fitting amp headers

  • Make sure you wire the GPIO and GND on opposite sides (diagonally) and across the breadboard gap when connecting the buttons

Printing the Case

post image
first attempt at 3D printing my own design

3D modeling and printing the enclosure turned out to be surprisingly laborious. In the first place, I had to freshen up on how to actually use 3D modeling software. I used Tinkercad, which seems to have handled my simple needs, but I can also see moving on to Blender for any future designs that are relatively complicated.

post image
enclosure print in action

Once I got the handle of the software, a lot of things came down to trial and error. I had the original design that I made of the hardware simulator, but the measurements, hole placements, and whole sizes turned out to not be accurate. I also realized a bottom piece of the enclosure was needed to hold the ESP32 in place.

post image
a mere fraction of my failed attempts

The final count for design iterations was 40. Another seemingly obvious thing I learned while going along is that I could design each enclosure component separately (speaker hole, buttons, mic holder/shelf, bottom enclosure, etc.), then piece them together at the end. Unfortunately, even at the end, fitting all the components together in the 3D modeling software took multiple iterations to get right as well.

For those building along, I've hosted my final enclosure 3D model here, at the cost of $1 (charging mainly as an experiment with x402).

Final Assembly

At long last, I reached the final stage of assembly. This meant upgrading from a breadboard to a perfboard (a perforated board board meant for prototyping or simple assembly; basically, the middle-ground between breadboard and custom printed circuit board), soldering numerous wires between board and components, and shoving everything into the newly minted enclosure.

post image
working device in soldered form

Incidentally, this was the easiest bit of the entire project. The only false starts came from trying to balance a few of the components in place while also shoving everything into the case. As you can see below, I ended up using Scotch Tape to hold the buttons and LED in place, while I shoved the wires in and clicked the bottom piece into position.

post image
shoving wires into the case and hoping nothing breaks

Biggest Takeaways

To wrap things up, I've compiled a short list of biggest takeaways from this project:

  • Simulations are important; not only for ensuring your endpoints and design work together, but also for ensuring the entire UX actually makes sense in the way you want to use the device in day-to-day life

  • For large 3D models, break it down into small parts. And once they all fit correctly composite them together

  • Before you start on any kind of soldering, breadboard your prototype first. You want to make sure that specific parts and wiring work together, as well as your firmware software, before you commit to soldering

  • Test at all stages as often as possible. You just added the amp to your breadboard? Make sure it is receiving current. You added the speaker? Write a quick timed beep program first before you move on to a button trigger. Testing often saves you a lot of time and confusion when you inevitably make a mistake