Meta announces SeamlessM4T: universal real-time translation
Meta unveiled on Tuesday (22) SeamlessM4T, an artificial intelligence model capable of translating speech and text in real time between 101 languages. The technology, made available as open source, promises to break down language barriers more naturally, without relying on text intermediaries.
According to the company, SeamlessM4T can perform direct audio-to-audio translation, meaning the user speaks in their language and hears the translation in another without any text conversion in between. This reduces latency and preserves speech nuances such as tone and emotion.
The model covers 101 languages for text input and 96 for speech input. Meta released the weights, code, and training data as open source, allowing researchers and developers to use and improve it.
How does it work?
SeamlessM4T is a multimodal model that combines speech and text processing. It can:
- Translate speech to speech
- Translate speech to text
- Translate text to speech
- Translate text to text
The innovation lies in its ability to perform audio-to-audio translation without a textual stage, something previous models could not do as fluently.
Implications and availability
Meta stated that the technology can be used in applications such as automatic subtitles, multilingual virtual assistants, and real-time communication between people who speak different languages. Being open source, it is expected that the community will contribute to improve the model and expand its use.
The company also highlighted ethical and safety concerns, including filters to prevent malicious uses such as audio deepfakes. The model was trained with speech and text data from various public sources.
The announcement comes amid growing competition in the AI sector, with giants like Google and OpenAI also investing in machine translation. SeamlessM4T stands out for its language coverage and open-source approach.