When the original GPT models came out, they were wild, crazy, and even scary. Yes they hallucinated and weren’t practical for real applications. But they were unhinged in a very interesting and creative way. Model companies quickly realized their potential and prioritized making them useful. To do this, they started RLHF’ing the models — feeding in human examples of what is “correct”, and fine tuning them to get closer and closer to these responses. They stopped releasing raw, pre-trained ope...
