Microsoft has introduced Mu, its innovative language model designed to enhance interaction with the Settings agent in Windows. This model is characterized by its ability to operate efficiently on local devices, especially those equipped with a Neural Processing Unit (NPU), allowing it to infer complex relationships between inputs and outputs with remarkable performance.
Mu is capable of processing over 100 tokens per second, making it an ideal tool for situations that demand quick and accurate responses. With its optimized transformer architecture, which utilizes an encoder-decoder design, the model reduces latency and improves performance compared to models that use only decoding.
During its development, Microsoft implemented various optimizations to ensure Mu’s effectiveness on NPUs. This included adjustments to the model’s architecture and an optimized arrangement of parameters, maximizing efficiency for specific hardware operations. Additionally, the model incorporates advanced quantization techniques that enable optimal performance without compromising accuracy.
To create the agent in Settings, Microsoft trained Mu with millions of examples, focusing on facilitating natural language inquiries that allow users to make changes to system settings. This process aimed to reduce response times to under 500 milliseconds, a goal that Mu has successfully achieved.
The development of Mu not only enhances the user experience by enabling more natural interactions with the operating system but also marks a significant advancement in the capabilities of language models to operate on lower-powered devices. Collaboration with applied science teams was essential in refining and optimizing the model, focusing on user feedback to continue improving the experience.
Source: MiMub in Spanish