Bemærk
Adgang til denne side kræver godkendelse. Du kan prøve at logge på eller ændre mapper.
Adgang til denne side kræver godkendelse. Du kan prøve at ændre mapper.
These examples run large-scale batch inference on AI Runtime serverless GPUs. They cover a range of modalities, such as LLM text generation and speech-to-text, using frameworks like Ray Data, vLLM, and Transformers.
| Example | Interface | Description |
|---|---|---|
| Qwen2.5-32B batch inference with Ray Data and vLLM | Notebook | Run multilingual batch inference with eight persistent vLLM replicas of Qwen2.5-32B-Instruct on 8 H100 GPUs and save the results to Unity Catalog. |
| Batch speech-to-text with Whisper | Notebook | Transcribe audio in batches with OpenAI Whisper large-v3-turbo on serverless GPUs, using libraries pre-installed in AI Runtime. |
| Batch inference with Qwen2.5-7B-Instruct and vLLM | CLI | Use Ray Data and vLLM to generate responses on an 8x H100 node and write the results to a Unity Catalog volume as Parquet. |
| Audio transcription and diarization with Whisper and pyannote | CLI | Use Ray Data to distribute speech transcription and speaker diarization across a GPU cluster and write the results to a Unity Catalog volume as Parquet. |
| Protein embeddings with BioNeMo ESM-2 | CLI | Use Ray Data and BioNeMo ESM-2 to embed proteins across an 8x H100 node, then train and evaluate a linear membrane-protein classifier. |
| Batch image understanding with Qwen3-VL-8B and vLLM | CLI | Use Ray Data and vLLM to caption and classify images on an 8x H100 node and write structured results to Parquet. |