Source: Wall Street

What is the ultimate winner of the AI war?

Perhaps it is neither the hardest talk of OpenAI, nor the scientific and technological giants of Microsoft, Google, which may occupy highlands in the future.

On Saturday, according to media semianalysis, Google researchers confessed in a leak paper, and the Google did not protect the urban river, as did OpenAI, which would be difficult to take advantage of competing with the source AI.

As mentioned in the document, open-source model training is faster, more customized, private and more sophisticated than the same product capacity. They are doing something with US$ 100 million and US$ 13 billion in terms of the parameters of “wise 10 million and 5.4 billion are difficult to meet” and will be able to do so in just weeks, not months.

For users, who also pays for Google products if there is no restricted, free and high-quality alternatives?

Below are Google leak documents:

Google does not take care of the urban river, as does OpenAI.

We have a lot of thought and thought about OpenAI, who will cross the next milestone? What next steps would be?

But it is disturbing that we do not have the capacity to win this arms race, as do OpenAI. As we struggle, the third faction has been strangling our meals.

I am referring to open-source AI, in short, they are taking away our market share. We believe that the “major open issues” have now been resolved and have reached users. Only a few examples are given:

LLMs on mobile phones: people operate the basic model at Pixel 6.

Expanded personal manual intelligence: you can fine-tune your personalized AI assistants at a night with your laptops.

Responsible publication: the issue is not “solution”, but “avoidance”. The entire web site is full of art models without any restrictions, while the text is not behind.

Multimodality: The current multi-model ScienceQA SOTA is being trained within one hour.

While our models still have a qualitative advantage, the gap is rapidly closing at a surprising pace. Open-source model training is faster, more customized, private and more sophisticated than the same product capacity. They are doing something with US$ 100 million and US$ 13 billion in terms of the parameters of the “wise of US$ 10 million and 54 billion is difficult” and will be able to do so in a few weeks, not months. This has profound implications for us:

We are not embarrassed. Our greatest hope is to learn from and cooperate with others outside the Google. We should give priority to achieving the 3P integration.

When free and unrestricted alternatives are of comparable quality, one cannot pay for a restricted model. Where should we consider our added value?

The big model is dragging us, and in the long run, the best model is those that can be quickly replaced.

What has happened?

In early March, the MaaMA language model was leaked and the open-source community received the first truly capable base model. It has no directive or dialogue adjustments or RLHF. Nevertheless, the community immediately understood what they had received.

Subsequently, enormous innovations have emerged, with development taking place only a few days. Less than a month now, there have been changes in order, quantification, quality improvement, human evaluation, modelling, RLHF, many of which are interrelated.

Most importantly, they have addressed the issue of deregulation (scaling) to the extent that anyone can adjust. Many new ideas come from the general population, and the threshold has been reduced from one major research institution to one person, one night and a powerful laptop computer.

In many respects, this is less surprising for anyone. The revitalization of the current open-source model is closely linked to the heat generated by the image model, which has not been forgotten by the originating communities, many of whom have referred to as the “Stable Diffusion” moment in LLMs.

In combination with major breakthroughs on scale (e.g., Grand Model Chinchilla), the public can be involved at lower cost through the low-lying matrices (LoRA); in both cases, access to a sufficiently high-quality model can trigger the ideas and duplicative booms of individuals and institutions around the world and will soon go beyond large enterprises.

These contributions are critical in the area of image generation, which puts Stable Diffusion on a different path from Dall-E. Product integration, markets, user interfaces and innovations resulting from an open model are not available in Dall-E.

Its effects are understandable: in terms of cultural impact, it quickly dominates and becomes more interdependent than the OpenAI solution. Whether the same thing will occur on LLM is still to be seen, but the broad structural elements are the same.

What are we mistaken?

Recent successful innovations have directly addressed the problems that we are still struggling, and their work can help us to avoid repeating.

LoRA is a very powerful technology, and we should pay more attention to the fact that the work of LoRA is based on the expression of a model update as a low-causuality, which will reduce the size of the updated matrix by several thousand times. This leaves only a small amount of cost and time required for fine-tuning of models. The individualization of language models in consumer hardware within a few hours is a major undertaking, especially for knowledge that involves the inclusion of new and diversified knowledge in near real time. The presence of this technology has not been fully utilized within Google, although it directly affects some of our most ambitious projects.

The beginning of the re-training model is a difficult road, and LoRA is so effective, partly because — as in other forms of fine-tuning — is piled, and improvements such as directive adjustments can be applied and then as other contributors increase.

Source: Wall Street

What is the ultimate winner of the AI war?

Perhaps it is neither the hardest talk of OpenAI, nor the scientific and technological giants of Microsoft, Google, which may occupy highlands in the future.

On Saturday, according to media semianalysis, Google researchers confessed in a leak paper, and the Google did not protect the urban river, as did OpenAI, which would be difficult to take advantage of competing with the source AI.

As mentioned in the document, open-source model training is faster, more customized, private and more sophisticated than the same product capacity. They are doing something with US$ 100 million and US$ 13 billion in terms of the parameters of “wise 10 million and 5.4 billion are difficult to meet” and will be able to do so in just weeks, not months.

For users, who also pays for Google products if there is no restricted, free and high-quality alternatives?

Below are Google leak documents:

Google does not take care of the urban river, as does OpenAI.

We have a lot of thought and thought about OpenAI, who will cross the next milestone? What next steps would be?

But it is disturbing that we do not have the capacity to win this arms race, as do OpenAI. As we struggle, the third faction has been strangling our meals.

I am referring to open-source AI, in short, they are taking away our market share. We believe that the “major open issues” have now been resolved and have reached users. Only a few examples are given:

LLMs on mobile phones: people operate the basic model at Pixel 6.

Expanded personal manual intelligence: you can fine-tune your personalized AI assistants at a night with your laptops.

Responsible publication: the issue is not “solution”, but “avoidance”. The entire web site is full of art models without any restrictions, while the text is not behind.

Multimodality: The current multi-model ScienceQA SOTA is being trained within one hour.

While our models still have a qualitative advantage, the gap is rapidly closing at a surprising pace. Open-source model training is faster, more customized, private and more sophisticated than the same product capacity. They are doing something with US$ 100 million and US$ 13 billion in terms of the parameters of the “wise of US$ 10 million and 54 billion is difficult” and will be able to do so in a few weeks, not months. This has profound implications for us:

We are not embarrassed. Our greatest hope is to learn from and cooperate with others outside the Google. We should give priority to achieving the 3P integration.

When free and unrestricted alternatives are of comparable quality, one cannot pay for a restricted model. Where should we consider our added value?

The big model is dragging us, and in the long run, the best model is those that can be quickly replaced.

What has happened?

In early March, the MaaMA language model was leaked and the open-source community received the first truly capable base model. It has no directive or dialogue adjustments or RLHF. Nevertheless, the community immediately understood what they had received.

Subsequently, enormous innovations have emerged, with development taking place only a few days. Less than a month now, there have been changes in order, quantification, quality improvement, human evaluation, modelling, RLHF, many of which are interrelated.

Most importantly, they have addressed the issue of deregulation (scaling) to the extent that anyone can adjust. Many new ideas come from the general population, and the threshold has been reduced from one major research institution to one person, one night and a powerful laptop computer.

In many respects, this is less surprising for anyone. The revitalization of the current open-source model is closely linked to the heat generated by the image model, which has not been forgotten by the originating communities, many of whom have referred to as the “Stable Diffusion” moment in LLMs.

In combination with major breakthroughs on scale (e.g., Grand Model Chinchilla), the public can be involved at lower cost through the low-lying matrices (LoRA); in both cases, access to a sufficiently high-quality model can trigger the ideas and duplicative booms of individuals and institutions around the world and will soon go beyond large enterprises.

These contributions are critical in the area of image generation, which puts Stable Diffusion on a different path from Dall-E. Product integration, markets, user interfaces and innovations resulting from an open model are not available in Dall-E.

Its effects are understandable: in terms of cultural impact, it quickly dominates and becomes more interdependent than the OpenAI solution. Whether the same thing will occur on LLM is still to be seen, but the broad structural elements are the same.

What are we mistaken?

Recent successful innovations have directly addressed the problems that we are still struggling, and their work can help us to avoid repeating.

LoRA is a very powerful technology, and we should pay more attention to the fact that the work of LoRA is based on the expression of a model update as a low-causuality, which will reduce the size of the updated matrix by several thousand times. This leaves only a small amount of cost and time required for fine-tuning of models. The individualization of language models in consumer hardware within a few hours is a major undertaking, especially for knowledge that involves the inclusion of new and diversified knowledge in near real time. The presence of this technology has not been fully utilized within Google, although it directly affects some of our most ambitious projects.

The beginning of the re-training model is a difficult road, and LoRA is so effective, partly because — as in other forms of fine-tuning — is piled, and improvements such as directive adjustments can be applied and then as other contributors increase.