A recently published research paper reveals that Apple Inc. (NASDAQ:AAPL) might be planning to introduce on-device generative AI capabilities on iPhones.
What Happened: Apple’s research paper suggests a strategy for running large language models (LLMs) on devices with limited RAM. In this case, it would help iPhones immensely since they have relatively less RAM than their Android counterparts.
Instead of loading the complete model at once, Apple’s approach involves storing model parameters and pulling portions into the device’s RAM as needed.
The paper suggests that this method could facilitate the operation of models up to twice the size of the available memory, leading to 4-5x and 20-25x increases in inference speed in CPU and GPU, respectively.
The potential introduction of on-device generative AI, which typically benefits from abundant RAM for faster read/write speeds, could mean shorter wait times for users and a conversationally paced AI assistant.
Apple’s proposal, however, indicates that ample RAM may not be essential for responsive on-device AI.
Apple’s strategy could bring on-device generative AI features to both old and new iPhones, despite iPhones traditionally having less RAM than many top-tier Android phones.
The paper also indicates that Apple isn’t the only company striving to reduce the footprint of LLMs. Both Qualcomm Inc. (NASDAQ:QCOM) and MediaTek‘s latest flagship processors support INT4 precision to shrink these models.
Why It Matters: This research paper comes on the heels of Apple’s earlier revelation about its innovative AI technology, “HUGS”, which can create a digital human avatar from a short video in approximately 30 minutes.
HUGS, or Human Gaussian Splats, utilizes machine learning and computer vision strategies to develop digital human avatars. The research paper on Apple’s Machine Learning Research page clarified that only minimal video footage of the subject in motion is required for the system to operate.
This should also relieve iPhone users and Apple fans alike, given that Apple was relatively silent about its AI efforts. At the same time, its rivals Alphabet Inc.’s (NASDAQ:GOOG) (NASDAQ:GOOGL) Google and Microsoft Corp. (NASDAQ:MSFT) have been vocal about their efforts.
