Posted by Apple has launched OpenELM (Open-source Efficient Language Models), a set of large language models (LLMs) intended to operate directly on devices instead of relying on cloud servers. Apple has yet to introduce AI capabilities powered by such models to its devices. However, rumours indicate that iOS 18 could feature several new AI features as Apple intends to deploy LLMs directly on devices for privacy reasons.
Apple’s OpenELM
As detailed in a white paper, titled– OpenELM: An efficient language model family with open-source training and inference framework, OpenELM outperforms comparable-sized existing LLMs pre-trained on publicly available datasets. These open-source models are now accessible on the Hugging Face Hub, a platform for sharing AI code within the community.
“Notably, OpenELM outperforms the recent open LLM, OLMo, by 2.36 per cent while requiring 2× fewer pre-training tokens. The average accuracy is calculated across multiple tasks listed, which are also part of the OpenLLM leaderboard,” the paper claims.
There are eight OpenELMs in total, four of which are pre-trained using the CoreNet library, whereas the other four are instruction-tuned models. Apple says that it employs a layer-wise scaling approach to enhance its accuracy and efficiency.
Instead of just releasing the final trained version of the model, Apple has made available the code, training logs, and various versions of the models as well. The researchers involved anticipate that this approach will accelerate advancements and yield more reliable outcomes in the natural language AI domain.
Apple affirms that the launch of OpenELMs will bolster and enrich the open research community by providing advanced LLMs. Notably, open sharing of information not only helps the community but also lets researchers find associated risks. In the case of open sharing of OpenELMS, developers and companies utilise them unchanged or customise them as per their needs, while allowing researchers to explore data and model biases in addition to any potential risk.