OpenMOSS-Team/MOSS-Transcribe-Diarize

Overview

MOSS-Transcribe-Diarize is a 0.9B end-to-end speech-to-text model for long-form multi-speaker transcription. It uses a Whisper-style audio encoder and a Qwen3-style causal decoder to generate structured transcripts with text, timestamps, and speaker labels in a single pass.

Example output:

[0.11][S01] Good morning! [1.03]
[1.11][S02] Morning, let's start the meeting. [2.42]

Key capabilities:

• End-to-end multi-speaker transcription without a separate ASR, diarization, or alignment pipeline.
• Structured text output with speaker IDs and timestamps.
• Long-audio transcription for meetings, interviews, podcasts, and customer support recordings.
• Prompt-based hotwords for names, organizations, product names, project codes, and domain vocabulary.

Prerequisites

Install vLLM with audio dependencies. If your installed vLLM release does not include MOSS-Transcribe-Diarize yet, pull the latest main branch or install the latest nightly build.

uv venv
source .venv/bin/activate

uv pip install -U "vllm[audio]" --torch-backend auto

For the latest source build:

git clone https://github.com/vllm-project/vllm.git
cd vllm
uv pip install -e ".[audio]"

For the latest nightly wheel, see the vLLM installation guide.

Launching with vLLM

vllm serve OpenMOSS-Team/MOSS-Transcribe-Diarize \
  --trust-remote-code \
  --served-model-name moss-mtd

The model uses custom Hugging Face processor code, so --trust-remote-code is required.

Client Usage

OpenAI Transcription API

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

with open("meeting.wav", "rb") as audio_file:
    transcription = client.audio.transcriptions.create(
        model="moss-mtd",
        file=audio_file,
        response_format="json",
        temperature=0,
    )

print(transcription.text)

cURL

curl http://localhost:8000/v1/audio/transcriptions \
  -F "model=moss-mtd" \
  -F "file=@meeting.wav" \
  -F "response_format=json" \
  -F "temperature=0"

Output Format

MOSS-Transcribe-Diarize returns a single text field with timestamped speaker turns:

[0.41][S01] Let's review the deployment plan. [2.80]
[3.12][S02] The vLLM server is already running. [5.10]

Speaker IDs such as S01, S02, and S03 are generated by the model and are intended to remain consistent within the audio.

The current vLLM integration returns this timestamped transcript as text. If you need a segment JSON schema, parse the text into {start, end, speaker, text} objects on the client side.

References

• Model card
• GitHub repository
• vLLM audio support