Everyday we see announcements for the new things our models and agents can do. We applaud them in public and give them cheers for their good benchmark scores. But some of us.. we scold them for the things they can’t do yet.

“Why can’t you do basic math??”

“Did you really just make up that research report?”

“What do you mean your knowledge cutoff is in 2024??”

I’m guilty of it too, but I remember I have a toddler. I get amazed every time she remembers the random promise I made to give her a muffin at home since we didn’t have the esoteric berry she wanted at the playground. The image comes crashing back down when you realize she hallucinates a new friend at daycare that joined her class. She learns words and new skills at break neck pace, but can’t seem to hold two tasks in memory.

It’s all really the same. Sometimes I need to meet her where her abilities are at whether that’s:

• breaking tasks down one at a time

• checking facts with her recollection of events with her teachers

• bring the step stool over for helping us make dinner in the kitchen

Our models may be stupid now.. but they’ll be grown and ready for college before you can say “think step by step”.