Have you ever noticed that your AI chatbot get lost in the middle of a conversation, or it simply says it cannot handle prompts that are too long? Well, that is because each model has a limitation in its processing capabilities, and starts to suffer once it goes over that limit —pretty much like they suffered from some kind of a digital attention deficit disorder. But this could soon change thanks to a new method for supercharging LLM capabilities.

Current LLMs have limited context capacities. For example, ChatGPT taps just 8,000 tokens of context, while Claude handles 100,000. Tokens are the basic units of text or code used by an LLM AI to process and generate language This restricts how much background information they can harness when formulating replies. Abacus AI has developed a method that allegedly doubles the usable context length for open-source LLMs like Meta’s Llama without compromising the model's accuracy in practical application.

Their technique involves "scaling" the position embeddings that track word locations in input texts. According to their Github page, Abacus AI claims that its scaling method drastically increases the number of tokens that a model can handle.

The researchers evaluated two scaled LlaMA variants on tasks like substring location and open-book QA. The scale 16 model maintained accuracy on real-world examples up to 16,000-word contexts, versus only 2,000 words in baseline Llama. It even showed some coherence at 20,000+ words, something that was not possible to achieve with just fine-tuning techniques.

The significance of context extension cannot be overstated. A narrow context window makes the model accurate but not really usable in complex tasks that require some background. Conversely, with an expanded context, LLMs can process and generate better responses but either take more time to do so or return sup-par results. Handling longer contexts efficiently could enable LLMs to absorb whole documents or multiple documents as background when generating text. This may lead to outputs that are more knowledge-grounded and consistent across long conversations.

However, the gains are not perfectly proportional to the scale factors.

It’s still necessary to fine tune strategies because scaling alone doesn’t guarantee high quality outputs. The Abacus team is also exploring advanced position encoding schemes from recent papers to further extend context capacity.

Their work suggests that scaling up existing LLMs is a viable path to expanding usable context length. This could democratize access to Large Language Models capable of handling lots of context at once.

However, the gains are not perfectly proportional to the scale factors.

It’s still necessary to fine tune strategies because scaling alone doesn’t guarantee high quality outputs. The Abacus team is also exploring advanced position encoding schemes from recent papers to further extend context capacity.

Their work suggests that scaling up existing LLMs is a viable path to expanding usable context length. This could democratize access to Large Language Models capable of handling lots of context at once.

In brief:

• A Colombian senator said that the country should look to El Salvador for inspiration.

• Gustavo Petro said that the South American country could use renewable energy to mine Bitcoin instead of making cocaine.

A high-profile Colombian politician has suggested the country look to El Salvador for inspiration and mine Bitcoin using renewable energy—instead of producing cocaine. 

Gustavo Petro, a leftist senator retweeted a story about El Salvador President Nayib Bukele announcing that the country had started mining Bitcoin using volcanic energy and commented: “What if the Pacific coast took advantage of the steep falls of the rivers of the western mountains to produce all the energy of the coast and replace cocaine with energy for cryptocurrencies?” 

President Bukele, who made Bitcoin legal tender in El Salvador last month, claims the Central American nation will mine Bitcoin using geothermal power from its volcanoes.

Bitcoin mining is the process of using powerful computers to verify transactions on the blockchain and produce digital assets. It uses lots of energy, though, so mining companies are now looking for clean energy to produce the asset. 

Petro, a frontrunner for Colombia’s presidential election next year, added via Twitter that “virtual currency is pure information and therefore energy.”

It wasn’t clear what Petro meant but Latin America energy policy analyst Wesley Tomaselli told Decrypt that the senator was referring to the potential of Colombia’s Pacific coast to use renewable energy. 

“He's right in that Colombia has great potential for Bitcoin mining because about three-fourths of its electricity generation comes from hydroelectric power,” he said. “The problem is, Petro appears to be selling Bitcoin mining as an alternative model for development to coca growing and cocaine shipments. It's not. Unless Petro has a magic wand.”

Colombia is the world’s largest producer of cocaine, according to the United Nations, and its Pacific coast is where a lot of the coca—the drug’s base ingredient—is grown. 

The Pacific coast is also a potential renewable energy hotspot. “Colombia's severe Andean mountain geography and river systems make it prime for hydroelectric power generation,” noted Tomaselli. 

But he added that mega projects in the area “have grown less popular here because of resistance from environmentalists who see them as damaging to local communities and ecosystems.”

El Salvador’s government has said that it will use only renewable and clean energy to produce the currency, an idea that has been both praised by Bitcoin believers and slammed by Bukele’s critics. 

Could Colombia be next?