Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

VibeVoice

Open-source speech recognition and real-time text-to-speech models for voice AI research

VibeVoice is a family of open-source voice AI models for speech recognition and text-to-speech. VibeVoice transcribes long recordings with speaker labels and timestamps, supports streaming audio recognition, and generates speech from streaming text. VibeVoice serves researchers and developers through publicly available code, model weights, and demos.

Updated 9 Nov 20266 sections

VibeVoice website screenshot

What is VibeVoice?

VibeVoice is an open-source voice AI research framework from Microsoft that includes automatic speech recognition and text-to-speech models. VibeVoice is intended for researchers and developers working on speech transcription, synthesis, and conversational audio. The GitHub repository links to model weights, technical reports, documentation, and demonstrations for the available model variants.

VibeVoice uses continuous acoustic and semantic speech tokenizers to process long sequences. VibeVoice also describes a next-token diffusion framework that combines a large language model for textual context and dialogue flow with a diffusion head for acoustic generation. The repository lists an MIT license and provides contribution guidelines.

What VibeVoice can do

VibeVoice covers both recognizing recorded or incoming speech and generating spoken audio from text. VibeVoice speech recognition combines transcription, speaker diarization, and timestamping in structured output. Available variants address long recordings, streaming input, and CPU inference, while the real-time speech generation model accepts text as it arrives.

  • VibeVoice processes up to 60 minutes of audio in a single recognition pass.
  • VibeVoice identifies who spoke, what was said, and when.
  • VibeVoice accepts customized hotwords such as names, technical terms, and background information.
  • VibeVoice emits streaming recognition text once per audio chunk.
  • VibeVoice supports over 50 languages in its long-form recognition model.
  • VibeVoice generates speech from streaming text input.
  • VibeVoice provides a CPU inference engine that does not require a GPU.

How VibeVoice works

VibeVoice provides separate documentation and access paths for its model variants. VibeVoice speech recognition can be explored through a playground, used through Hugging Face Transformers, or served with vLLM. The repository also links to ASR finetuning resources, streaming recognition documentation, and a CPU inference implementation.

VibeVoice real-time text-to-speech is linked to model weights, documentation, and a Colab demonstration. VibeVoice accepts streaming text and reports approximately 300 milliseconds to the first audible output, with long-form generation of approximately 10 minutes. Experimental speaker options include multilingual voices and distinct English speaking styles.

Who VibeVoice is for

VibeVoice is intended for speech AI research and development, including work on long-form recognition, streaming transcription, and speech synthesis. VibeVoice provides structured transcription for recordings where speaker identity and timing matter, while customized hotwords support recognition involving specialized vocabulary. The available finetuning code also supports experimentation with the recognition model.

  • VibeVoice supports researchers studying speech recognition and synthesis.
  • VibeVoice supports developers testing transcription with speaker labels and timestamps.
  • VibeVoice supports experiments with incoming audio transcription and streaming text-to-speech.
  • VibeVoice supports recognition workflows involving specific names or technical terms.

VibeVoice pricing and access

VibeVoice is distributed as an open-source project through its public GitHub repository, which lists an MIT license. VibeVoice links to downloadable model weights on Hugging Face and provides playground or Colab access for selected models. The source does not describe a VibeVoice subscription, paid tier, or hosted-service pricing schedule.

VibeVoice strengths and limitations

VibeVoice combines long-form speech recognition, structured speaker-aware transcripts, streaming variants, and accessible research resources within one model family. VibeVoice also offers a CPU recognition option and integration with established inference libraries. However, the repository explicitly frames the work as research and development and warns that generated outputs can contain errors or bias.

  • VibeVoice has removed the original long-form TTS code from the repository.
  • VibeVoice may produce unexpected, biased, or inaccurate outputs.
  • VibeVoice is not recommended for commercial or real-world applications without further testing and development.
  • VibeVoice synthetic speech requires responsible use to avoid impersonation, fraud, and disinformation.

VibeVoice FAQ

What is VibeVoice?

VibeVoice is an open-source family of voice AI models from Microsoft covering automatic speech recognition and text-to-speech. VibeVoice includes long-form transcription, streaming recognition, and real-time speech generation, with documentation, model weights, and demonstrations linked from its repository.

Is VibeVoice free?

VibeVoice is available as an open-source project through a public GitHub repository that lists an MIT license. VibeVoice provides links to model weights and demonstrations, but the source does not state a subscription price or hosted-service pricing.

Can VibeVoice identify speakers and add timestamps?

VibeVoice speech recognition produces structured transcripts containing speaker identity, timestamps, and spoken content. VibeVoice jointly performs transcription, diarization, and timestamping, and its long-form recognition model accepts up to 60 minutes of continuous audio in a single pass.

Does VibeVoice support streaming speech?

VibeVoice supports streaming speech recognition and real-time text-to-speech through separate model variants. VibeVoice streaming recognition emits text once per audio chunk, while VibeVoice real-time speech generation accepts streaming text input and supports long-form spoken output.

What integrations does VibeVoice support?

VibeVoice speech recognition is available through Hugging Face Transformers and supports vLLM inference. VibeVoice speech recognition is also integrated into Azure AI Foundry Labs, where users can explore and test its speech-to-text capabilities through Microsoft Foundry.

Can VibeVoice be used in commercial applications?

VibeVoice is intended for research and development, and the repository does not recommend commercial or real-world applications without further testing and development. VibeVoice outputs may be inaccurate or biased, and users are expected to verify content and deploy models lawfully.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like VibeVoice

Deepgram ASR

www.deepgram.com

Speech-to-text APIs, text-to-speech, and voice agents for real-time voice applications.

Freemium

NeMo

www.github.com

Open-source framework for creating, customizing, and deploying speech recognition, text-to-speech, and speech LLMs.

Free

Cockatoo

www.cockatoo.com

AI audio and video transcription with speaker labels, timestamps, transcript editing, and file exports.

Freemium

Suno AI Bark

www.github.com

Open-source text-to-audio model for multilingual speech, music, sound effects and nonverbal sounds.

Free