In fact, in less intuitive places, the data is quietly changing: Stanford University's "AI Index 2023 Report" shows that the percentage of companies adopting AI in 2022 has more than doubled since 2017. These companies report significant cost reductions and revenue increases as a result of AI adoption.

Although the figures for 2023 are not yet available, it is easy to assume that the above figures will reach a new inflection point this year just by the AIGC field, which has been brought on fire by ChatGPT, and AIGC is poised to set off the fourth industrial revolution.

But at the same time, these companies also have a new challenge in building AI infrastructure.

First of all, even if the power, AI field computing power demand surges and the contradiction formed by the lack of supply in this year has become particularly intense, even OpenAI CEO Sam Altman admitted that his company is plagued by the shortage of computing power, its API reliability and speed problems are repeatedly complained. In addition, a large number of companies are also facing the rising cost of computing power due to this wave of high demand.

Second, in the choice of models, many companies found that the most hotly discussed large models do not have a mature business model, and there are still problems with their security and other aspects. Take Samsung's device solutions division as an example, they had three data leaks in less than a month after enabling ChatGPT, which made companies who originally intended to call the OpenAI API directly back off. In addition, training and deploying mega-models themselves is equally dissuasive: imagine that simply sending a request to a large model once may require an expensive GPU card for exclusive computing, something that many enterprises can ill afford.

But then again, is a "know-it-all" mega-model like ChatGPT really necessary for businesses? Does running AI models to support your business mean scaling GPUs like crazy? What are companies that are already leveraging AI to improve their bottom line doing? After analyzing some of their best practices, we've found some reference answers.

Companies that are already using AI: The tough choice between performance and cost

The Internet is one of the first industries to use AI to improve efficiency, and its typical workloads -- recommendation systems, visual processing, natural language processing, etc. -- cannot be optimized without AI. They also face different challenges at the levels of performance and cost.

Let's look at recommendation systems first. Recommender systems are widely used in e-commerce, social media, audio and video streaming, and many other areas. Take e-commerce as an example, in the annual 618 and Double 11 shopping peak, Alibaba and other leading e-commerce companies face hundreds of millions of real-time requests from a huge global customer base, so they want to meet the requirements of AI inference in terms of throughput and latency, while ensuring AI inference accuracy and guaranteeing recommendation quality.

The next look at the visual processing, only Meituan, we can find intelligent image processing, merchant registration license recognition, sweep code open bicycle, sweep medicine box to buy drugs and many other application scenarios. AI has become an important part of its business map. However, with the rapid growth of Meituan's business and user volume, more and more applications need to build intelligent processes through visual AI. Meituan needs to improve the throughput rate of visual AI reasoning while ensuring the accuracy of visual AI reasoning to support more intelligent businesses.

Finally, look at natural language processing. Thanks to the buzz generated by ChatGPT, natural language processing is gaining unprecedented market attention and technology tracking. As a pioneer in NLP technology research in China, Baidu has built a complete product system and technology portfolio in this field. ERNIE 3.0, as an important part of its large NLP model, has also demonstrated excellent performance in various NLP application scenarios, especially in Chinese natural language understanding and generation tasks. However, as NLP becomes commercially available in more industries, users are demanding more from ERNIE 3.0, such as higher processing efficiency and a wider range of deployment scenarios.

All of these issues can only be solved with large-scale infrastructure investment, but the common problem that plagues these companies is that standalone GPUs can meet the performance needs, but the cost pressure is high, so expanding the GPU size is not an optimal option.

The Cost-Effective Solution: Intel® 4th Generation Xeon® Scalable Processors

There is a stereotype in the AI community that CPUs are not suited to carry AI tasks. But a demonstration by Julien Simon, Chief Communications Officer at Hugging Face, shattered that stereotype. His company and Intel have partnered to build a generative AI application called Q8-Chat that delivers a ChatGPT-like chat experience, but runs on a single 32-core Intel® Xeon® processor.

As this example shows, using CPUs to host AI tasks (especially inference tasks) is actually very common in industry, with Alibaba, Meituan, and Baidu all using related solutions to alleviate the arithmetic problem.