The new generation of flagship smartphones released in the recent quarter exudes a brand-new flavor of AI.
On October 29, the Xiaomi 15/15 Pro, the first to be equipped with the brand-new Snapdragon 8 Ultra Edition, was released. The voice assistant Xiaoai Tongxue has transformed into AI Super Xiaoai.
Shortly after that, the flagship smartphones equipped with the Snapdragon 8 Ultra Edition were released one after another. The YOYO assistant of the Honor Magic 7 series has turned into the YOYO Agent. The Breeno assistant of the OnePlus 13 has evolved into the Super Breeno assistant.
Just as the birth of the mouse changed the interaction mode of the computer interface, and the birth of the smartphone brought a brand-new touch interaction interface, nowadays, generative AI is also bringing about a new terminal interaction revolution.
“I would like to show you this vision: Generative AI will fundamentally change applications and user experiences, and this is one of the core focuses of Snapdragon now and in the future.” Cristiano Amon, President and CEO of Qualcomm, shared at the 2024 Qualcomm Snapdragon Summit, “The experiences we are familiar with are about to change and will be transformed into a way that is easier to accept and understand. This is not about a single killer application, but the experience of every application will change.”
At the Web Summit, a major technology event in November, Amon also emphasized Qualcomm’s view on the future of generative AI again. He emphasized that generative AI is the new UI, and it will serve as a new interaction mode for all computing terminals around us. AI-powered agents will be the key for the industry to respond to and seize opportunities. Just as the change in the touch interface made mobile phones a new computing space, in the revolution of generative AI, more edge terminals such as automobiles and XR will also become new computing spaces. Embracing the era of generative AI is also the clear strategic direction that Qualcomm has already determined. “AI is the new UI, and the app-centered experience is changing,” Amon said.
By delving deeper into the powerful AI engine of the Snapdragon 8 Ultra Edition and the Snapdragon Digital Chassis, which is revolutionizing the terminal experience from the bottom up, from hardware to software, we will look forward to the future of hybrid AI even more.
01
AI Makes Mobile Phones “More Human-like”
The collective upgrade of intelligent assistants on the new flagship mobile phones may be regarded as a marketing gimmick. However, by specifically looking at the new upgrades of common functions, we can savor the “AI flavor” of mobile phones in the era of generative AI.
Let’s first look at the changes of Xiaomi’s AI Super Xiaoai. The new AI caption function supports real-time translation globally. When watching foreign movies, there are real-time captions, and when having overseas meetings, there is simultaneous interpretation. There is also AI voice recognition that supports global voice transcription, which can distinguish different speakers and generate summaries. AI writing is a global text tool that supports one-step text polishing and rewriting.
Common functions have new ways of use and are available globally, all thanks to the support of the AI at the mobile phone system level.
The Honor YOYO Agent is also a platform-level AI. For example, just by saying “Turn off application notifications” or “Order a cup of milk tea”, operations that previously required several manual steps can now be completed by the mobile phone automatically mobilizing different applications from the bottom of the system with just one sentence.
The AI functions of mobile phones are getting smarter and have a wider range of applications, making mobile phones increasingly have the flavor of generative AI.
“People are now accustomed to the app-centered experience,” Amon said. “Now every application and use case may face changes. The emergence of AI enables machines to understand human language, so the usage experiences we are familiar with will be completely redefined.”
Then, what kind of killer application of generative AI can subvert the mobile phone experience?
Amon said in response to this, “Many applications with killer potential have not yet emerged, but the current transformation is taking place. I firmly believe that generative AI is the future, but no one can give the specific timing.”
During the process of mobile phones evolving towards agents, killer applications may emerge.
“Among the currently released smartphones, true agent processing has not yet been achieved. The existing applications on the market are relatively excellent intelligent voice assistants,” Durga Malladi, Senior Vice President and General Manager of the Technology Planning and Edge Solutions Business of Qualcomm Technologies, said when the Snapdragon 8 Ultra Edition was just released. “The difference between them is that agents can perform more complex tasks, especially those tasks with unclear instructions. For example, agents need to choose appropriate models on their own to perform processing tasks.”
02
The Core of Mobile Phone Agent Implementation – Multimodal and Heterogeneous Computing
The implementation of mobile phone agents requires a comprehensive innovation of mobile phones from the bottom up, from hardware to software.
Multimodal large models are the key to realizing agents.
Malladi shared an example. Pick up the mobile phone and say, “Help me book a Mexican restaurant tonight. It shouldn’t be too far away, with a high rating, and the estimated arrival time at the restaurant is between 7:30 and 8:30 pm, and it should be a restaurant that I haven’t been to in the past three months.”
To answer this question, the agent will need to call various different models for processing. First, understand the question raised. Second, if you want to choose a restaurant that is not too far away, you need to know the current location information. Finally, to meet the requirement of “a restaurant that has not been visited in the past three months”, it is necessary to use the user’s personal knowledge graph, that is, the user context information stored on the terminal side (such as favorite diets, exercise habits, past preferences, etc.) as the personal knowledge graph to help the mobile phone better understand the user’s intention.
“The essence of an agent is a planner that can handle very complex tasks. The way to interact with an agent should combine voice, text, images, and so on,” Malladi further said. “To conduct agent planning, multimodal capabilities will be of great help, so agents must have multimodal support.”
As early as February this year, Qualcomm AI Research demonstrated the world’s first multimodal large model running on Android mobile phones. At the same time, it also demonstrated a multimodal large model with over 7 billion parameters running on Windows PCs equipped with the Snapdragon X Elite, which can accept text and ambient audio inputs (such as music, traffic ambient audio, etc.) and generate multi-round dialogues based on the audio content.
In October, Qualcomm launched the Qualcomm AI Planner. By understanding personal preferences integrated with local context information, the Qualcomm AI Planner can integrate this knowledge into generative AI assistants to provide users with meaningful personalized replies and actions.
Of course, with the help of multimodal models, the Qualcomm AI Planner supports multiple terminal interaction modalities, including text, vision, and voice.
Alex Katouzian, General Manager of the Mobile Phone, Computing, and XR Business Group of Qualcomm Technologies, said, “Smartphone assistants that can support multimodal and agent functions mean that they can see what you see, hear what you hear, and understand the world around you. This ensures that they can handle personal needs that meet users’ requirements without prompting other applications, run completely on the terminal to ensure data security and privacy, and respond to user requests immediately.”
The Qualcomm AI Planner will become an important part of the Qualcomm AI software stack. The evolution of software also requires a comprehensive hardware upgrade to achieve better support.
The Qualcomm AI Engine with Significantly Upgraded Computing Efficiency and Memory
The Snapdragon 820 launched in 2015 integrated the first Qualcomm AI Engine, which is a heterogeneous computing engine. Since then, Qualcomm’s AI Engine has continued to be upgraded in line with the development of AI technology and market demands.
In 2018, the Qualcomm Hexagon DSP was upgraded to the Hexagon NPU. Earlier than the industry’s attention to the NPU, Qualcomm’s heterogeneous AI engine was also upgraded to a combination of CPU, GPU, and NPU.
This year’s Snapdragon 8 Ultra Edition has seen another all-round upgrade of the Qualcomm AI Engine.
It is particularly worth noting that the Qualcomm Oryon CPU, known as the last piece of the puzzle for Qualcomm’s self-developed chips, has been integrated into the mobile platform for the first time.
The first-generation self-developed Oryon CPU of Qualcomm was carried on the PC platform Snapdragon X Elite. After the announcement of the first batch of Windows 11 AI+ PCs equipped with the Qualcomm Oryon CPU at Microsoft Build this year, Qualcomm quickly became a dark horse and a benchmark in the AI PC field. The Oryon CPU has helped Qualcomm achieve a further leap in its transformation from a wireless connection company to a connection + computing company.
The second-generation self-developed Oryon CPU on the Snapdragon 8 Ultra Edition has achieved a significant performance improvement of up to 45% in both single-threaded and multi-threaded benchmark tests compared with the third-generation Snapdragon 8 in a room temperature environment, and the energy efficiency has also been improved by as much as 44%. The cross-generational performance improvement of the second-generation Oryon CPU has significantly enhanced multi-task processing, latency-critical tasks, productivity applications, and initial AI workloads.
Introducing the second-generation Oryon CPU into the Qualcomm AI Engine is like adding an outstanding conductor to a symphony orchestra, responsible for starting all AI workloads and handling heavy load tasks, allowing other components of the AI Engine to focus on specific AI tasks.
The comprehensively upgraded Hexagon NPU is like an amazing violinist in a symphony orchestra. The performance of the newly architected Hexagon NPU has been improved by 45%, and the energy efficiency has also been improved by 45%.
To support generative AI and multimodal models and achieve faster inference, the Hexagon NPU has added more cores. More scalar cores can support longer content, reaching a 4k context window, and more vector cores can strengthen the support for large language models and traditional AI.
A significant improvement in processing efficiency is not enough. Memory is also a major factor restricting the operation of generative AI on terminals.
“To directly run a 100-billion-parameter model on a mobile phone without any compression, at least 20GB of memory is required. So model compression and quantization are very important,” Malladi said. “We now support quantization technologies with different data precisions. Through various quantization technologies, models can be shrunk. Last year, running large models on the terminal side was still a challenge. Today, it has become relatively simple for us, and we are still continuously improving.”
The significantly improved memory architecture of the second-generation Oryon CPU also enables the Snapdragon 8 Ultra Edition to meet the needs of edge AI.
The new Oryon CPU has 12MB of level-2 cache for each cluster, and a total of 24MB of tightly coupled dedicated cache, which is the largest CPU cache in the field of smartphones. Meanwhile, Qualcomm has also significantly increased the level-1 cache in each super core and performance core. In addition, considering that generative AI is highly dependent on memory, the Snapdragon 8 Ultra Edition is equipped with the currently highest-speed 10.7Gbps LPDDR5x memory.
The memory concurrency of the Qualcomm AI Engine allows AI and computer vision workloads to access a memory space simultaneously, further enhancing the performance and possibilities of terminal-side AI.
There is also the pioneering sliced architecture Adreno GPU, with its performance increased by 40% and power consumption reduced by 40%, which is also an important component of the new AI engine.
The all-round upgrade of the Snapdragon 8 Ultra Edition’s AI Engine has enabled the large model processing performance of the Snapdragon 8 Ultra Edition to comprehensively surpass that of the third-generation Snapdragon 8. The token generation rate of the basic large language model has been increased by up to 100%. On some of the currently popular large language models in the industry, the processing speed of the Snapdragon 8 Ultra Edition reaches over 70 tokens/s, which is at the leading level in the industry.
Of course, combining the comprehensively upgraded Qualcomm AI Engine with the newly designed AI-ISP and GPU of the Snapdragon 8 Ultra Edition has also enabled the new generation of flagship mobile phones to break through the upper limits of imaging and gaming performance and bring more possibilities for smartphone AI applications.
For example, on the Honor Magic 7 series, there are magic photo editing functions such as AI erasure, AI image enlargement, AI style, and face restoration. On the Xiaomi 15 series, you can tell the shooting content and the function mode or parameters will be automatically matched, and tell the photo editing direction and the image will be automatically adjusted. On the OnePlus 13, there are AI functions to remove reflections, AI erasure, and AI to remove motion blur to “save” bad photos.
The AI God Assistant 2.0 function on the realme GT7 Pro, which is also equipped with the Snapdragon 8 Ultra Edition, makes AI the best assistant for the whole team in games.
03
Automobiles Are New Computing Spaces
Generative AI is a major revolution in the computing field. It will change the layout of the smartphone era, change the roles of operating systems and app stores, and change the role of computing.
Amon said, “The automotive industry is experiencing an unprecedented major revolution. Currently, automobiles have become brand-new computing spaces. Besides bringing new screens and digital cockpits to automobiles, generative AI also brings changes to the automotive industry.”
The Snapdragon Digital Chassis solution has already become one of the important assets in the automotive industry. Qualcomm has cooperative relationships with almost all automobile manufacturers.
The newly released Snapdragon Cockpit Ultra Edition platform and Snapdragon Ride Ultra Edition have also comprehensively upgraded the performance of the Snapdragon Digital Chassis.
The NPU integrated in the Snapdragon Cockpit Ultra Edition has a performance that is up to 12 times higher than that of the previous-generation flagship Snapdragon Cockpit SoC. It is designed to support transformer acceleration and end-to-end network architectures and can support large basic models.
Using one of the basic models, through Retrieval-Augmented Generation (RAG), use cases such as vehicle preventive maintenance can be realized – the AI assistant can use the model trained on the automobile manual to retrieve answers for unknown icons on the screen.
“We will witness a revolution driven by AI,” as Amon said. This is also why many enterprises are also focusing on and firmly believing in this future full of changes and are cooperating with Qualcomm based on the Snapdragon platform.
Qualcomm has been always committed to promoting the openness of the platform’s AI capabilities, ensuring that all developers can build AI applications and capabilities for different terminals based on Qualcomm and the Snapdragon platform. At the beginning of this year, Qualcomm launched the Qualcomm AI Hub. There are currently hundreds of cutting-edge open-source AI models optimized for Qualcomm and the Snapdragon platform on the Qualcomm AI Hub, which have been recognized by the leading developer ecosystem.
During the Snapdragon Summit, Qualcomm also announced further cooperation with a series of leading AI enterprises in the industry, such as OpenAI, Meta, Tencent Hunyuan, Zhipu, Mistral AI, and Tech Mahindra, to jointly build a prosperous edge-side AI ecosystem and accelerate the experience changes brought by generative AI to a wide range of terminals.
Honor and Lenovo are talking a lot about agents, while Qualcomm has more forward-looking terminal-side AI approaches.
A new pattern in the PC market is taking shape. How does the AI PC of Qualcomm + Windows 11 lead Apple?
The rare appearance of Qualcomm’s CEO on the same stage with Intel and AMD, AI may reshape the PC market.