Musk’s artificial intelligence company xAI announced the opening of the weights and architecture of its 314 billion parameter hybrid expert model Grok-1. Grok-1 is a large-scale language model trained from scratch by xAI, using a hybrid expert model architecture where 25% of the weights are active on a given token. The model's basic model weights and network architecture are now available under the Apache 2.0 license. The pre-training phase of Grok-1 ended in October 2023, and what is cu...
