Melia rt on prem docs - preview

docs/deployments/container/accessing-images.mdx

@@ -131,7 +131,7 @@ This image contains both models. To load only one of them, see [Running only one
 
 ### Melia 1 model
 
-Melia 1 is a multilingual, GPU-only model, currently available in Batch. For an overview, see [Models](/speech-to-text/models#melia-1).
+Melia 1 is a multilingual, GPU-only model, generally available in Batch, with Real-time (streaming) available as an early preview. For an overview, see [Models](/speech-to-text/models#melia-1).
 
 <CodeBlock language="bash">
   {`# pulling the Melia 1 Batch transcriber with the ${smVariables.latestMelia1ContainerVersion} tag:

docs/deployments/container/gpu-speech-to-text-melia-1.mdx

@@ -10,8 +10,7 @@ import melia1GpuSttDockerCompose from './assets/gpu-stt-melia-1-docker-compose.y
 
 # GPU Speech to text container (Melia 1)
 
-Melia 1 is a multilingual, GPU-only model, currently available in Batch. For an overview of the model, see [Models](/speech-to-text/models#melia-1).
-
+Melia 1 is a multilingual, GPU-only model, currently available in Batch. For an overview, see [Models](/speech-to-text/models#melia-1).
 ## Prerequisites
 
 - [A license file or a license token](/deployments/container/licensing)
@@ -87,14 +86,15 @@ I0705 08:12:55.598672 1 http_server.cc:400] "Started Metrics Service at 0.0.0.0:
 
 The Melia 1 inference server currently runs in _batch_ mode only, processing whole files and returning the transcript at the end.
 
-Once the inference server is running, run the Melia 1 transcriber with the `SM_INFERENCE_ENDPOINT` environment variable set to the gRPC endpoint of the inference server. The transcriber offloads inference to the server, so it does not require a GPU:
+Once the inference server is running, run the Melia 1 transcriber with the `SM_INFERENCE_ENDPOINT` environment variable set to the gRPC endpoint of the inference server. The transcriber offloads inference to the server, so it does not require a GPU. The transcriber is a single unified image for both batch and real-time, so the `SM_ASR_MODE` environment variable must also be set, to `batch` or `rt`:
 
 ```bash
 docker run --rm -it \
   -e SM_INFERENCE_ENDPOINT=<server>:<port> \
+  -e SM_ASR_MODE=batch \
   -v $PWD/license.json:/license.json \
   -v $PWD/example.wav:/input.audio \
-  <speech_container_image_name>
+  speechmaticspublic.azurecr.io/sm-asr-transcriber-melia-1:${smVariables.latestMelia1ContainerVersion}
 ```
 
 To accept multiple jobs over an HTTP API without restarting the container between jobs, run the transcriber as a [batch persistent worker](/deployments/container/batch-persistent-worker).

docs/private/assets/gpu-stt-melia-1-rt-docker-compose.yml

@@ -0,0 +1,36 @@
+version: "3.8"
+
+networks:
+  transcriber:
+    driver: bridge
+
+services:
+  triton:
+    image: speechmaticspublic.azurecr.io/sm-asr-inference-server-melia-1:{{ CONTAINER_VERSION }}
+    deploy:
+      resources:
+        reservations:
+          devices:
+            - driver: nvidia
+              ### Limit to N GPUs
+              # count: 1
+              ### Pick specific GPUs by device ID
+              # device_ids:
+              #   - 0
+              #   - 3
+              capabilities:
+                - gpu
+    container_name: triton
+    networks:
+      - transcriber
+    expose:
+      - 8000/tcp
+      - 8001/tcp
+      - 8002/tcp
+    environment:
+      - NVIDIA_DRIVER_CAPABILITIES=all
+      - NVIDIA_VISIBLE_DEVICES=all
+      - CUDA_VISIBLE_DEVICES=0
+      - SM_STREAMING=true
+    volumes:
+      - $PWD/license.json:/license.json:ro

docs/private/melia-1-realtime.mdx

@@ -7,6 +7,7 @@ import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 import meliaRealtimePythonExample from "./assets/melia-1-rt-example.py?raw";
 import meliaRealtimeJsExample from "./assets/melia-1-rt-example.js?raw";
+import melia1GpuSttRtDockerCompose from './assets/gpu-stt-melia-1-rt-docker-compose.yml?raw';
 
 # Melia 1 Realtime Preview - Multilingual transcription
 
@@ -35,6 +36,82 @@ The preview runs on a dedicated endpoint, separate from the production Realtime
 
 The preview is deployed in France (EU) and Oregon (USA), and is available on SaaS only.
 
+### Real-time inference on prem
+
+:::warning
+Real-time (streaming) inference on prem for Melia 1 is an early preview for evaluation only, and is not intended for production use.
+:::
+
+The Melia 1 inference server can run in two modes: _batch_ or _real-time_. The default mode is batch. To configure the server for real-time, set the `SM_STREAMING` environment variable to `true` when starting the inference server:
+
+<CodeBlock language="bash">
+  {`docker run --rm -it \\
+    -v $PWD/license.json:/license.json \\
+    --gpus '"device=0"' \\
+    -e CUDA_VISIBLE_DEVICES \\
+    -e SM_STREAMING=true \\
+    -p 8001:8001 \\
+    speechmaticspublic.azurecr.io/sm-asr-inference-server-melia-1:1.7.0
+`}
+</CodeBlock>
+
+When the Container starts you should see output similar to this,
+indicating that the server has started and is ready to serve requests.
+
+```
+I0911 11:04:02.321449 1 server.cc:709] 
++--------------------+---------+--------+
+| Model              | Version | Status |
++--------------------+---------+--------+
+| aed                | 1       | READY  |
+| body               | 1       | READY  |
+| preprocessor       | 1       | READY  |
+| streaming          | 1       | READY  |
+| streaming_ensemble | 1       | READY  |
++--------------------+---------+--------+
+...
+I0911 11:04:02.449550 1 grpc_server.cc:2579] "Started GRPCInferenceService at 0.0.0.0:8001"
+I0911 11:04:02.449744 1 http_server.cc:4961] "Started HTTPService at 0.0.0.0:8000"
+I0911 11:04:02.491582 1 http_server.cc:400] "Started Metrics Service at 0.0.0.0:8002"
+```
+
+The server can only support one of these modes at once.
+
+Once the inference server is running in real-time mode, run the Melia 1 transcriber as normal, with the `SM_INFERENCE_ENDPOINT` environment variable set to the gRPC endpoint of the inference server. Unlike the [Standard and Enhanced GPU container](/deployments/container/gpu-speech-to-text#batch-and-realtime-inference), Melia 1 uses the same transcriber image for both batch and real-time — there are no separate `rt-` and `batch-` prefixed images. Set `SM_ASR_MODE=rt` to run the transcriber in real-time mode:
+
+```bash
+docker run --rm -it \
+  -e SM_INFERENCE_ENDPOINT=<server>:<port> \
+  -e SM_ASR_MODE=rt \
+  -v $PWD/license.json:/license.json \
+  -p 9000:9000 \
+  speechmaticspublic.azurecr.io/sm-asr-transcriber-melia-1:1.7.0
+```
+
+Connect over WebSocket as described in [Realtime transcription](/deployments/container/cpu-speech-to-text#realtime-transcription).
+
+### Multi-session containers
+
+By default the real-time container will accept only one websocket connection at a time. To enable multiple connections, set the environment variable `SM_MAX_CONCURRENT_CONNECTIONS` to the maximum number of sessions to allow. When this is set, the `/ready` [health check endpoint](/deployments/container/cpu-speech-to-text#health-service) will return true if there is a free connection available.
+
+CPU usage scales with the number of active sessions, whereas most memory usage is shared between connections.
+
+### Monitoring the server
+
+The inference server is based on [Nvidia's Triton architecture](https://developer.nvidia.com/nvidia-triton-inference-server) and as such
+can be monitored using Triton's inbuilt Prometheus metrics, or the GRPC/HTTP APIs. To expose these, configure an external mapping for port
+8002(Prometheus) or 8000(HTTP).
+
+## Docker compose example
+
+This Docker Compose file will create a Speechmatics Melia 1 GPU Inference Server:
+
+(assumes your `license.json` file is in the current working directory)
+
+<CodeBlock language="yaml" title="docker-compose.yml">
+{melia1GpuSttRtDockerCompose.replace('{{ CONTAINER_VERSION }}', '1.7.0')}
+</CodeBlock>
+
 ### Minimal example
 Example minimal `StartRecognition` message:
 
@@ -127,8 +204,8 @@ Message format:
 | Word timings                                   | ✅                                                                         |                                             |
 | Language labeling                              | ✅ Word-level                                                              |                                             |
 | Alphanumeric smart formatting                  | ✅ Improved accuracy, e.g. recognition of email/web addresses transcription |                                            |
-| Language hints (guide predicted languages)     | ⚠️ Not yet                                                                | Targeting August 2026                        |
-| End of Utterance / Force end of Utterance      | ⚠️ Not yet                                                                | Targeting August 2026                        |
+| Language hints (guide predicted languages)     | ⚠️ Not yet                                                                | Targeting September 2026                        |
+| End of Utterance / Force end of Utterance      | ⚠️ Not yet                                                                | Targeting September 2026                        |
 | Speaker diarization                            | ⚠️ Not yet                                                                | Planned Q4 2026                              |
 | Speaker identification                         | ⚠️ Not yet                                                                | Prioritized according to customer need       |
 | `max_delay` / `max_delay_mode` latency control | ⚠️ Not yet. Finals latency averages around 4 seconds and varies           | Lower latency and control planned Q3/Q4 2026 |