Revert "Merge remote-tracking branch 'upstream/master' into prompt"

This reverts commit 6e42088656, reversing changes made to 4a59bb011d.
Merge remote-tracking branch 'upstream/master' into prompt
2024-09-12 00:49:31 +08:00 · 2024-09-04 17:48:06 +08:00 · 2024-07-27 09:02:58 +07:00 · 2024-07-24 09:00:32 +07:00 · 2024-07-18 16:48:52 +07:00 · 2024-07-10 10:16:35 +08:00
27 changed files with 2912 additions and 167 deletions
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -7,7 +7,7 @@ Contributions are welcome! Here are some pointers to help you install the librar
 We recommend installing the module in editable mode with the `dev` extra requirements:

 ```bash
-git clone https://github.com/guillaumekln/faster-whisper.git
+git clone https://github.com/SYSTRAN/faster-whisper.git
 cd faster-whisper/
 pip install -e .[dev]
 ```
--- a/2
+++ b/2
@@ -1,6 +1,6 @@
 MIT License

-Copyright (c) 2023 Guillaume Klein
+Copyright (c) 2023 SYSTRAN

 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-[![CI](https://github.com/guillaumekln/faster-whisper/workflows/CI/badge.svg)](https://github.com/guillaumekln/faster-whisper/actions?query=workflow%3ACI) [![PyPI version](https://badge.fury.io/py/faster-whisper.svg)](https://badge.fury.io/py/faster-whisper)
+[![CI](https://github.com/SYSTRAN/faster-whisper/workflows/CI/badge.svg)](https://github.com/SYSTRAN/faster-whisper/actions?query=workflow%3ACI) [![PyPI version](https://badge.fury.io/py/faster-whisper.svg)](https://badge.fury.io/py/faster-whisper)

 # Faster Whisper transcription with CTranslate2

@@ -8,11 +8,13 @@ This implementation is up to 4 times faster than [openai/whisper](https://github

 ## Benchmark

+### Whisper
+
 For reference, here's the time and memory usage that are required to transcribe [**13 minutes**](https://www.youtube.com/watch?v=0u7tTptBo9I) of audio using different implementations:

 * [openai/whisper](https://github.com/openai/whisper)@[6dea21fd](https://github.com/openai/whisper/commit/6dea21fd7f7253bfe450f1e2512a0fe47ee2d258)
 * [whisper.cpp](https://github.com/ggerganov/whisper.cpp)@[3b010f9](https://github.com/ggerganov/whisper.cpp/commit/3b010f9bed9a6068609e9faf52383aea792b0362)
-* [faster-whisper](https://github.com/guillaumekln/faster-whisper)@[cce6b53e](https://github.com/guillaumekln/faster-whisper/commit/cce6b53e4554f71172dad188c45f10fb100f6e3e)
+* [faster-whisper](https://github.com/SYSTRAN/faster-whisper)@[cce6b53e](https://github.com/SYSTRAN/faster-whisper/commit/cce6b53e4554f71172dad188c45f10fb100f6e3e)

 ### Large-v2 model on GPU

@@ -36,6 +38,78 @@ For reference, here's the time and memory usage that are required to transcribe

 *Executed with 8 threads on a Intel(R) Xeon(R) Gold 6226R.*

+
+### Distil-whisper
+
+| Implementation | Precision | Beam size | Time | Gigaspeech WER |
+| --- | --- | --- | --- | --- |
+| distil-whisper/distil-large-v2 | fp16 | 4 |- | 10.36 |
+| [faster-distil-large-v2](https://huggingface.co/Systran/faster-distil-whisper-large-v2) | fp16 | 5 | - | 10.28 |
+| distil-whisper/distil-medium.en | fp16 | 4 | - | 11.21 |
+| [faster-distil-medium.en](https://huggingface.co/Systran/faster-distil-whisper-medium.en) | fp16 | 5 | - | 11.21 |
+
+*Executed with CUDA 11.4 on a NVIDIA 3090.*
+
+<details>
+<summary>testing details (click to expand)</summary>
+
+For `distil-whisper/distil-large-v2`, the WER is tested with code sample from [link](https://huggingface.co/distil-whisper/distil-large-v2#evaluation). for `faster-distil-whisper`, the WER is tested with setting:
+```python
+from faster_whisper import WhisperModel
+
+model_size = "distil-large-v2"
+# model_size = "distil-medium.en"
+# Run on GPU with FP16
+model = WhisperModel(model_size, device="cuda", compute_type="float16")
+segments, info = model.transcribe("audio.mp3", beam_size=5, language="en")
+```
+</details>
+
+## Requirements
+
+* Python 3.8 or greater
+
+Unlike openai-whisper, FFmpeg does **not** need to be installed on the system. The audio is decoded with the Python library [PyAV](https://github.com/PyAV-Org/PyAV) which bundles the FFmpeg libraries in its package.
+
+### GPU
+
+GPU execution requires the following NVIDIA libraries to be installed:
+
+* [cuBLAS for CUDA 12](https://developer.nvidia.com/cublas)
+* [cuDNN 8 for CUDA 12](https://developer.nvidia.com/cudnn)
+
+**Note**: Latest versions of `ctranslate2` support CUDA 12 only. For CUDA 11, the current workaround is downgrading to the `3.24.0` version of `ctranslate2` (This can be done with `pip install --force-reinstall ctranslate2==3.24.0` or specifying the version in a `requirements.txt`).
+
+There are multiple ways to install the NVIDIA libraries mentioned above. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below. 
+
+<details>
+<summary>Other installation methods (click to expand)</summary>
+
+
+**Note:** For all these methods below, keep in mind the above note regarding CUDA versions. Depending on your setup, you may need to install the _CUDA 11_ versions of libraries that correspond to the CUDA 12 libraries listed in the instructions below.
+
+#### Use Docker
+
+The libraries (cuBLAS, cuDNN) are installed in these official NVIDIA CUDA Docker images: `nvidia/cuda:12.0.0-runtime-ubuntu20.04` or `nvidia/cuda:12.0.0-runtime-ubuntu22.04`.
+
+#### Install with `pip` (Linux only)
+
+On Linux these libraries can be installed with `pip`. Note that `LD_LIBRARY_PATH` must be set before launching Python.
+
+```bash
+pip install nvidia-cublas-cu12 nvidia-cudnn-cu12
+
+export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`
+```
+
+**Note**: Version 9+ of `nvidia-cudnn-cu12` appears to cause issues due its reliance on cuDNN 9 (Faster-Whisper does not currently support cuDNN 9). Ensure your version of the Python package is for cuDNN 8.
+
+#### Download the libraries from Purfview's repository (Windows & Linux)
+
+Purfview's [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) provides the required NVIDIA libraries for Windows & Linux in a [single archive](https://github.com/Purfview/whisper-standalone-win/releases/tag/libs). Decompress the archive and place the libraries in a directory included in the `PATH`.
+
+</details>
+
 ## Installation

 The module can be installed from [PyPI](https://pypi.org/project/faster-whisper/):
@@ -44,26 +118,31 @@ The module can be installed from [PyPI](https://pypi.org/project/faster-whisper/
 pip install faster-whisper
 ```

-**Other installation methods:**
+<details>
+<summary>Other installation methods (click to expand)</summary>
+
+### Install the master branch

 ```bash
-# Install the master branch:
-pip install --force-reinstall "faster-whisper @ https://github.com/guillaumekln/faster-whisper/archive/refs/heads/master.tar.gz"
-
-# Install a specific commit:
-pip install --force-reinstall "faster-whisper @ https://github.com/guillaumekln/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"
+pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/refs/heads/master.tar.gz"
 ```

-### GPU support
+### Install a specific commit

-GPU execution requires the NVIDIA libraries cuBLAS 11.x and cuDNN 8.x to be installed on the system. Please refer to the [CTranslate2 documentation](https://opennmt.net/CTranslate2/installation.html).
+```bash
+pip install --force-reinstall "faster-whisper @ https://github.com/SYSTRAN/faster-whisper/archive/a4f1cc8f11433e454c3934442b5e1a4ed5e865c3.tar.gz"
+```
+
+</details>

 ## Usage

+### Faster-whisper
+
 ```python
 from faster_whisper import WhisperModel

-model_size = "large-v2"
+model_size = "large-v3"

 # Run on GPU with FP16
 model = WhisperModel(model_size, device="cuda", compute_type="float16")
@@ -87,6 +166,25 @@ for segment in segments:
 segments, _ = model.transcribe("audio.mp3")
 segments = list(segments)  # The transcription will actually run here.
 ```
+### Faster Distil-Whisper
+
+The Distil-Whisper checkpoints are compatible with the Faster-Whisper package. In particular, the latest [distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3)
+checkpoint is intrinsically designed to work with the Faster-Whisper transcription algorithm. The following code snippet 
+demonstrates how to run inference with distil-large-v3 on a specified audio file:
+
+```python
+from faster_whisper import WhisperModel
+
+model_size = "distil-large-v3"
+
+model = WhisperModel(model_size, device="cuda", compute_type="float16")
+segments, info = model.transcribe("audio.mp3", beam_size=5, language="en", condition_on_previous_text=False)
+
+for segment in segments:
+    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
+```
+
+For more information about the distil-large-v3 model, refer to the original [model card](https://huggingface.co/distil-whisper/distil-large-v3).

 ### Word-level timestamps

@@ -106,7 +204,7 @@ The library integrates the [Silero VAD](https://github.com/snakers4/silero-vad)
 segments, _ = model.transcribe("audio.mp3", vad_filter=True)
 ```

-The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in the [source code](https://github.com/guillaumekln/faster-whisper/blob/master/faster_whisper/vad.py). They can be customized with the dictionary argument `vad_parameters`:
+The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in the [source code](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/vad.py). They can be customized with the dictionary argument `vad_parameters`:

 ```python
 segments, _ = model.transcribe(
@@ -129,31 +227,39 @@ logging.getLogger("faster_whisper").setLevel(logging.DEBUG)

 ### Going further

-See more model and transcription options in the [`WhisperModel`](https://github.com/guillaumekln/faster-whisper/blob/master/faster_whisper/transcribe.py) class implementation.
+See more model and transcription options in the [`WhisperModel`](https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/transcribe.py) class implementation.

 ## Community integrations

 Here is a non exhaustive list of open-source projects using faster-whisper. Feel free to add your project to the list!

+
+* [faster-whisper-server](https://github.com/fedirz/faster-whisper-server) is an OpenAI compatible server using `faster-whisper`. It's easily deployable with Docker, works with OpenAI SDKs/CLI, supports streaming, and live transcription.
+* [WhisperX](https://github.com/m-bain/whisperX) is an award-winning Python library that offers speaker diarization and accurate word-level timestamps using wav2vec2 alignment
 * [whisper-ctranslate2](https://github.com/Softcatala/whisper-ctranslate2) is a command line client based on faster-whisper and compatible with the original client from openai/whisper.
 * [whisper-diarize](https://github.com/MahmoudAshraf97/whisper-diarization) is a speaker diarization tool that is based on faster-whisper and NVIDIA NeMo.
-* [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) contains the portable ready to run binaries of faster-whisper for Windows.
+* [whisper-standalone-win](https://github.com/Purfview/whisper-standalone-win) Standalone CLI executables of faster-whisper for Windows, Linux & macOS. 
 * [asr-sd-pipeline](https://github.com/hedrergudene/asr-sd-pipeline) provides a scalable, modular, end to end multi-speaker speech to text solution implemented using AzureML pipelines.
 * [Open-Lyrics](https://github.com/zh-plus/Open-Lyrics) is a Python library that transcribes voice files using faster-whisper, and translates/polishes the resulting text into `.lrc` files in the desired language using OpenAI-GPT.
+* [wscribe](https://github.com/geekodour/wscribe) is a flexible transcript generation tool supporting faster-whisper, it can export word level transcript and the exported transcript then can be edited with [wscribe-editor](https://github.com/geekodour/wscribe-editor)
+* [aTrain](https://github.com/BANDAS-Center/aTrain) is a graphical user interface implementation of faster-whisper developed at the BANDAS-Center at the University of Graz for transcription and diarization in Windows ([Windows Store App](https://apps.microsoft.com/detail/atrain/9N15Q44SZNS2)) and Linux.
+* [Whisper-Streaming](https://github.com/ufal/whisper_streaming) implements real-time mode for offline Whisper-like speech-to-text models with faster-whisper as the most recommended back-end. It implements a streaming policy with self-adaptive latency based on the actual source complexity, and demonstrates the state of the art.
+* [WhisperLive](https://github.com/collabora/WhisperLive) is a nearly-live implementation of OpenAI's Whisper which uses faster-whisper as the backend to transcribe audio in real-time.
+* [Faster-Whisper-Transcriber](https://github.com/BBC-Esq/ctranslate2-faster-whisper-transcriber) is a simple but reliable voice transcriber that provides a user-friendly interface.

 ## Model conversion

-When loading a model from its size such as `WhisperModel("large-v2")`, the correspondig CTranslate2 model is automatically downloaded from the [Hugging Face Hub](https://huggingface.co/guillaumekln).
+When loading a model from its size such as `WhisperModel("large-v3")`, the corresponding CTranslate2 model is automatically downloaded from the [Hugging Face Hub](https://huggingface.co/Systran).

 We also provide a script to convert any Whisper models compatible with the Transformers library. They could be the original OpenAI models or user fine-tuned models.

-For example the command below converts the [original "large-v2" Whisper model](https://huggingface.co/openai/whisper-large-v2) and saves the weights in FP16:
+For example the command below converts the [original "large-v3" Whisper model](https://huggingface.co/openai/whisper-large-v3) and saves the weights in FP16:

 ```bash
 pip install transformers[torch]>=4.23

-ct2-transformers-converter --model openai/whisper-large-v2 --output_dir whisper-large-v2-ct2 \
-    --copy_files tokenizer.json --quantization float16
+ct2-transformers-converter --model openai/whisper-large-v3 --output_dir whisper-large-v3-ct2
+--copy_files tokenizer.json preprocessor_config.json --quantization float16
 ```

 * The option `--model` accepts a model name on the Hub or a path to a model directory.
@@ -165,12 +271,12 @@ Models can also be converted from the code. See the [conversion API](https://ope

 1. Directly load the model from a local directory:
 ```python
-model = faster_whisper.WhisperModel('whisper-large-v2-ct2')
+model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
 ```

 2. [Upload your model to the Hugging Face Hub](https://huggingface.co/docs/transformers/model_sharing#upload-with-the-web-interface) and load it from its name:
 ```python
-model = faster_whisper.WhisperModel('username/whisper-large-v2-ct2')
+model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")
 ```

 ## Comparing performance against other implementations
--- a/benchmark/benchmark.m4a
+++ b/benchmark/benchmark.m4a
--- a/benchmark/memory_benchmark.py
+++ b/benchmark/memory_benchmark.py
@@ -0,0 +1,94 @@
+import argparse
+import time
+
+from typing import Callable
+
+import py3nvml.py3nvml as nvml
+
+from memory_profiler import memory_usage
+from utils import MyThread, get_logger, inference
+
+logger = get_logger("faster-whisper")
+parser = argparse.ArgumentParser(description="Memory benchmark")
+parser.add_argument(
+    "--gpu_memory", action="store_true", help="Measure GPU memory usage"
+)
+parser.add_argument("--device-index", type=int, default=0, help="GPU device index")
+parser.add_argument(
+    "--interval",
+    type=float,
+    default=0.5,
+    help="Interval at which measurements are collected",
+)
+args = parser.parse_args()
+device_idx = args.device_index
+interval = args.interval
+
+
+def measure_memory(func: Callable[[], None]):
+    if args.gpu_memory:
+        logger.info(
+            "Measuring maximum GPU memory usage on GPU device."
+            " Make sure to not have additional processes running on the same GPU."
+        )
+        # init nvml
+        nvml.nvmlInit()
+        handle = nvml.nvmlDeviceGetHandleByIndex(device_idx)
+        gpu_name = nvml.nvmlDeviceGetName(handle)
+        gpu_memory_limit = nvml.nvmlDeviceGetMemoryInfo(handle).total >> 20
+        gpu_power_limit = nvml.nvmlDeviceGetPowerManagementLimit(handle) / 1000.0
+        info = {"gpu_memory_usage": [], "gpu_power_usage": []}
+
+        def _get_gpu_info():
+            while True:
+                info["gpu_memory_usage"].append(
+                    nvml.nvmlDeviceGetMemoryInfo(handle).used >> 20
+                )
+                info["gpu_power_usage"].append(
+                    nvml.nvmlDeviceGetPowerUsage(handle) / 1000
+                )
+                time.sleep(interval)
+
+                if stop:
+                    break
+
+            return info
+
+        stop = False
+        thread = MyThread(_get_gpu_info, params=())
+        thread.start()
+        func()
+        stop = True
+        thread.join()
+        result = thread.get_result()
+
+        # shutdown nvml
+        nvml.nvmlShutdown()
+        max_memory_usage = max(result["gpu_memory_usage"])
+        max_power_usage = max(result["gpu_power_usage"])
+        print("GPU name: %s" % gpu_name)
+        print("GPU device index: %s" % device_idx)
+        print(
+            "Maximum GPU memory usage: %dMiB / %dMiB (%.2f%%)"
+            % (
+                max_memory_usage,
+                gpu_memory_limit,
+                (max_memory_usage / gpu_memory_limit) * 100,
+            )
+        )
+        print(
+            "Maximum GPU power usage: %dW / %dW (%.2f%%)"
+            % (
+                max_power_usage,
+                gpu_power_limit,
+                (max_power_usage / gpu_power_limit) * 100,
+            )
+        )
+    else:
+        logger.info("Measuring maximum increase of memory usage.")
+        max_usage = memory_usage(func, max_usage=True, interval=interval)
+        print("Maximum increase of RAM memory usage: %d MiB" % max_usage)
+
+
+if __name__ == "__main__":
+    measure_memory(inference)
--- a/benchmark/normalizer.json
+++ b/benchmark/normalizer.json
--- a/benchmark/requirements.benchmark.txt
+++ b/benchmark/requirements.benchmark.txt
@@ -0,0 +1,6 @@
+transformers
+jiwer
+evaluate
+datasets
+memory_profiler
+py3nvml
--- a/benchmark/speed_benchmark.py
+++ b/benchmark/speed_benchmark.py
@@ -0,0 +1,31 @@
+import argparse
+import timeit
+
+from typing import Callable
+
+from utils import inference
+
+parser = argparse.ArgumentParser(description="Speed benchmark")
+parser.add_argument(
+    "--repeat",
+    type=int,
+    default=3,
+    help="Times an experiment will be run.",
+)
+args = parser.parse_args()
+
+
+def measure_speed(func: Callable[[], None]):
+    # as written in https://docs.python.org/3/library/timeit.html#timeit.Timer.repeat,
+    # min should be taken rather than the average
+    runtimes = timeit.repeat(
+        func,
+        repeat=args.repeat,
+        number=10,
+    )
+    print(runtimes)
+    print("Min execution time: %.3fs" % (min(runtimes) / 10.0))
+
+
+if __name__ == "__main__":
+    measure_speed(inference)
--- a/benchmark/utils.py
+++ b/benchmark/utils.py
@@ -0,0 +1,39 @@
+import logging
+
+from threading import Thread
+from typing import Optional
+
+from faster_whisper import WhisperModel
+
+model_path = "large-v3"
+model = WhisperModel(model_path, device="cuda")
+
+
+def inference():
+    segments, info = model.transcribe("benchmark.m4a", language="fr")
+    for segment in segments:
+        print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
+
+
+def get_logger(name: Optional[str] = None) -> logging.Logger:
+    formatter = logging.Formatter("%(levelname)s: %(message)s")
+    logger = logging.getLogger(name)
+    logger.setLevel(logging.DEBUG)
+    handler = logging.StreamHandler()
+    handler.setFormatter(formatter)
+    logger.addHandler(handler)
+    return logger
+
+
+class MyThread(Thread):
+    def __init__(self, func, params):
+        super(MyThread, self).__init__()
+        self.func = func
+        self.params = params
+        self.result = None
+
+    def run(self):
+        self.result = self.func(*self.params)
+
+    def get_result(self):
+        return self.result
--- a/benchmark/wer_benchmark.py
+++ b/benchmark/wer_benchmark.py
@@ -0,0 +1,61 @@
+import argparse
+import json
+
+from datasets import load_dataset
+from evaluate import load
+from tqdm import tqdm
+from transformers.models.whisper.english_normalizer import EnglishTextNormalizer
+
+from faster_whisper import WhisperModel
+
+parser = argparse.ArgumentParser(description="WER benchmark")
+parser.add_argument(
+    "--audio_numb",
+    type=int,
+    default=None,
+    help="Specify the number of validation audio files in the dataset."
+    " Set to None to retrieve all audio files.",
+)
+args = parser.parse_args()
+
+model_path = "large-v3"
+model = WhisperModel(model_path, device="cuda")
+
+# load the dataset with streaming mode
+dataset = load_dataset("librispeech_asr", "clean", split="validation", streaming=True)
+
+# define the evaluation metric
+wer_metric = load("wer")
+normalizer = EnglishTextNormalizer(json.load(open("normalizer.json")))
+
+
+def inference(batch):
+    batch["transcription"] = []
+    for sample in batch["audio"]:
+        segments, info = model.transcribe(sample["array"], language="en")
+        batch["transcription"].append("".join([segment.text for segment in segments]))
+    batch["reference"] = batch["text"]
+    return batch
+
+
+dataset = dataset.map(function=inference, batched=True, batch_size=16)
+
+all_transcriptions = []
+all_references = []
+
+# iterate over the dataset and run inference
+for i, result in tqdm(enumerate(dataset), desc="Evaluating..."):
+    all_transcriptions.append(result["transcription"])
+    all_references.append(result["reference"])
+    if args.audio_numb and i == (args.audio_numb - 1):
+        break
+
+# normalize predictions and references
+all_transcriptions = [normalizer(transcription) for transcription in all_transcriptions]
+all_references = [normalizer(reference) for reference in all_references]
+
+# compute the WER metric
+wer = 100 * wer_metric.compute(
+    predictions=all_transcriptions, references=all_references
+)
+print("WER: %.3f" % wer)
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -0,0 +1,6 @@
+FROM nvidia/cuda:12.2.2-cudnn8-runtime-ubuntu22.04
+WORKDIR /root
+RUN apt-get update -y && apt-get install -y python3-pip
+COPY infer.py jfk.flac ./
+RUN pip3 install faster-whisper
+CMD ["python3", "infer.py"]
--- a/docker/infer.py
+++ b/docker/infer.py
@@ -0,0 +1,7 @@
+from faster_whisper import WhisperModel
+
+jfk_path = "jfk.flac"
+model = WhisperModel("tiny", device="cuda")
+segments, info = model.transcribe(jfk_path, word_timestamps=True)
+for segment in segments:
+    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
--- a/docker/jfk.flac
+++ b/docker/jfk.flac
--- a/faster_whisper/init.py
+++ b/faster_whisper/init.py
@@ -1,9 +1,10 @@
 from faster_whisper.audio import decode_audio
 from faster_whisper.transcribe import WhisperModel
-from faster_whisper.utils import download_model, format_timestamp
+from faster_whisper.utils import available_models, download_model, format_timestamp
 from faster_whisper.version import __version__

 __all__ = [
+    "available_models",
    "decode_audio",
    "WhisperModel",
    "download_model",
--- a/faster_whisper/assets/init.py
+++ b/faster_whisper/assets/init.py
--- a/faster_whisper/assets/silero_vad.onnx
+++ b/faster_whisper/assets/silero_vad.onnx
--- a/faster_whisper/audio.py
+++ b/faster_whisper/audio.py
@@ -6,6 +6,7 @@ system dependencies. FFmpeg does not need to be installed on the system.
 However, the API is quite low-level so we need to manipulate audio frames directly.
 """

+import gc
 import io
 import itertools

@@ -42,7 +43,7 @@ def decode_audio(
    raw_buffer = io.BytesIO()
    dtype = None

-    with av.open(input_file, metadata_errors="ignore") as container:
+    with av.open(input_file, mode="r", metadata_errors="ignore") as container:
        frames = container.decode(audio=0)
        frames = _ignore_invalid_frames(frames)
        frames = _group_frames(frames, 500000)
@@ -53,6 +54,11 @@ def decode_audio(
            dtype = array.dtype
            raw_buffer.write(array)

+    # It appears that some objects related to the resampler are not freed
+    # unless the garbage collector is manually run.
+    del resampler
+    gc.collect()
+
    audio = np.frombuffer(raw_buffer.getbuffer(), dtype=dtype)

    # Convert s16 back to f32.
@@ -96,3 +102,18 @@ def _resample_frames(frames, resampler):
    # Add None to flush the resampler.
    for frame in itertools.chain(frames, [None]):
        yield from resampler.resample(frame)
+
+
+def pad_or_trim(array, length: int, *, axis: int = -1):
+    """
+    Pad or trim the audio array to N_SAMPLES, as expected by the encoder.
+    """
+    if array.shape[axis] > length:
+        array = array.take(indices=range(length), axis=axis)
+
+    if array.shape[axis] < length:
+        pad_widths = [(0, 0)] * array.ndim
+        pad_widths[axis] = (0, length - array.shape[axis])
+        array = np.pad(array, pad_widths)
+
+    return array
--- a/faster_whisper/feature_extractor.py
+++ b/faster_whisper/feature_extractor.py
@@ -142,11 +142,15 @@ class FeatureExtractor:
            data[f] = np.fft.fft(fft_signal, axis=0)[:num_fft_bins]
        return data.T

-    def __call__(self, waveform, padding=True):
+    def __call__(self, waveform, padding=True, chunk_length=None):
        """
        Compute the log-Mel spectrogram of the provided audio, gives similar results
        whisper's original torch implementation with 1e-5 tolerance.
        """
+        if chunk_length is not None:
+            self.n_samples = chunk_length * self.sampling_rate
+            self.nb_max_frames = self.n_samples // self.hop_length
+
        if padding:
            waveform = np.pad(waveform, [(0, self.n_samples)])

--- a/faster_whisper/tokenizer.py
+++ b/faster_whisper/tokenizer.py
@@ -19,15 +19,21 @@ class Tokenizer:
        self.tokenizer = tokenizer

        if multilingual:
+            if task not in _TASKS:
+                raise ValueError(
+                    "'%s' is not a valid task (accepted tasks: %s)"
+                    % (task, ", ".join(_TASKS))
+                )
+
+            if language not in _LANGUAGE_CODES:
+                raise ValueError(
+                    "'%s' is not a valid language code (accepted language codes: %s)"
+                    % (language, ", ".join(_LANGUAGE_CODES))
+                )
+
            self.task = self.tokenizer.token_to_id("<|%s|>" % task)
-            if self.task is None:
-                raise ValueError("%s is not a valid task" % task)
-
-            self.language_code = language
            self.language = self.tokenizer.token_to_id("<|%s|>" % language)
-            if self.language is None:
-                raise ValueError("%s is not a valid language code" % language)
-
+            self.language_code = language
        else:
            self.task = None
            self.language = None
@@ -99,10 +105,46 @@ class Tokenizer:
            [s if isinstance(s, str) else self.tokenizer.decode(s) for s in outputs]
        )

+    @cached_property
+    def non_speech_tokens(self) -> Tuple[int]:
+        """
+        Returns the list of tokens to suppress in order to avoid any speaker tags or non-speech
+        annotations, to prevent sampling texts that are not actually spoken in the audio, e.g.
+
+        - ♪♪♪
+        - ( SPEAKING FOREIGN LANGUAGE )
+        - [DAVID] Hey there,
+
+        keeping basic punctuations like commas, periods, question marks, exclamation points, etc.
+        """
+        symbols = list('"#()*+/:;<=>@[\\]^_`{|}~「」『』')
+        symbols += (
+            "<< >> <<< >>> -- --- -( -[ (' (\" (( )) ((( ))) [[ ]] {{ }} ♪♪ ♪♪♪".split()
+        )
+
+        # symbols that may be a single token or multiple tokens depending on the tokenizer.
+        # In case they're multiple tokens, suppress the first token, which is safe because:
+        # These are between U+2640 and U+267F miscellaneous symbols that are okay to suppress
+        # in generations, and in the 3-byte UTF-8 representation they share the first two bytes.
+        miscellaneous = set("♩♪♫♬♭♮♯")
+        assert all(0x2640 <= ord(c) <= 0x267F for c in miscellaneous)
+
+        # allow hyphens "-" and single quotes "'" between words, but not at the beginning of a word
+        result = {self.encode(" -")[0], self.encode(" '")[0]}
+        for symbol in symbols + list(miscellaneous):
+            for tokens in [
+                self.encode(symbol),
+                self.encode(" " + symbol),
+            ]:
+                if len(tokens) == 1 or symbol in miscellaneous:
+                    result.add(tokens[0])
+
+        return tuple(sorted(result))
+
    def split_to_word_tokens(
        self, tokens: List[int]
    ) -> Tuple[List[str], List[List[int]]]:
-        if self.language_code in {"zh", "ja", "th", "lo", "my"}:
+        if self.language_code in {"zh", "ja", "th", "lo", "my", "yue"}:
            # These languages don't typically use spaces, so it is difficult to split words
            # without morpheme analysis. Here, we instead split words at any
            # position where the tokens are decoded as valid unicode points
@@ -161,3 +203,112 @@ class Tokenizer:
                word_tokens[-1].extend(subword_tokens)

        return words, word_tokens
+
+
+_TASKS = (
+    "transcribe",
+    "translate",
+)
+
+_LANGUAGE_CODES = (
+    "af",
+    "am",
+    "ar",
+    "as",
+    "az",
+    "ba",
+    "be",
+    "bg",
+    "bn",
+    "bo",
+    "br",
+    "bs",
+    "ca",
+    "cs",
+    "cy",
+    "da",
+    "de",
+    "el",
+    "en",
+    "es",
+    "et",
+    "eu",
+    "fa",
+    "fi",
+    "fo",
+    "fr",
+    "gl",
+    "gu",
+    "ha",
+    "haw",
+    "he",
+    "hi",
+    "hr",
+    "ht",
+    "hu",
+    "hy",
+    "id",
+    "is",
+    "it",
+    "ja",
+    "jw",
+    "ka",
+    "kk",
+    "km",
+    "kn",
+    "ko",
+    "la",
+    "lb",
+    "ln",
+    "lo",
+    "lt",
+    "lv",
+    "mg",
+    "mi",
+    "mk",
+    "ml",
+    "mn",
+    "mr",
+    "ms",
+    "mt",
+    "my",
+    "ne",
+    "nl",
+    "nn",
+    "no",
+    "oc",
+    "pa",
+    "pl",
+    "ps",
+    "pt",
+    "ro",
+    "ru",
+    "sa",
+    "sd",
+    "si",
+    "sk",
+    "sl",
+    "sn",
+    "so",
+    "sq",
+    "sr",
+    "su",
+    "sv",
+    "sw",
+    "ta",
+    "te",
+    "tg",
+    "th",
+    "tk",
+    "tl",
+    "tr",
+    "tt",
+    "uk",
+    "ur",
+    "uz",
+    "vi",
+    "yi",
+    "yo",
+    "zh",
+    "yue",
+)
--- a/faster_whisper/transcribe.py
+++ b/faster_whisper/transcribe.py
@@ -1,18 +1,20 @@
 import itertools
+import json
 import logging
 import os
 import zlib

+from inspect import signature
 from typing import BinaryIO, Iterable, List, NamedTuple, Optional, Tuple, Union

 import ctranslate2
 import numpy as np
 import tokenizers

-from faster_whisper.audio import decode_audio
+from faster_whisper.audio import decode_audio, pad_or_trim
 from faster_whisper.feature_extractor import FeatureExtractor
-from faster_whisper.tokenizer import Tokenizer
-from faster_whisper.utils import download_model, format_timestamp, get_logger
+from faster_whisper.tokenizer import _LANGUAGE_CODES, Tokenizer
+from faster_whisper.utils import download_model, format_timestamp, get_end, get_logger
 from faster_whisper.vad import (
    SpeechTimestampsMap,
    VadOptions,
@@ -47,10 +49,13 @@ class TranscriptionOptions(NamedTuple):
    best_of: int
    patience: float
    length_penalty: float
+    repetition_penalty: float
+    no_repeat_ngram_size: int
    log_prob_threshold: Optional[float]
    no_speech_threshold: Optional[float]
    compression_ratio_threshold: Optional[float]
    condition_on_previous_text: bool
+    prompt_reset_on_temperature: float
    temperatures: List[float]
    initial_prompt: Optional[Union[str, Iterable[int]]]
    prefix: Optional[str]
@@ -61,12 +66,17 @@ class TranscriptionOptions(NamedTuple):
    word_timestamps: bool
    prepend_punctuations: str
    append_punctuations: str
+    max_new_tokens: Optional[int]
+    clip_timestamps: Union[str, List[float]]
+    hallucination_silence_threshold: Optional[float]
+    hotwords: Optional[str]


 class TranscriptionInfo(NamedTuple):
    language: str
    language_probability: float
    duration: float
+    duration_after_vad: float
    all_language_probs: Optional[List[Tuple[str, float]]]
    transcription_options: TranscriptionOptions
    vad_options: VadOptions
@@ -83,13 +93,16 @@ class WhisperModel:
        num_workers: int = 1,
        download_root: Optional[str] = None,
        local_files_only: bool = False,
+        files: dict = None,
+        **model_kwargs,
    ):
        """Initializes the Whisper model.

        Args:
          model_size_or_path: Size of the model to use (tiny, tiny.en, base, base.en,
-            small, small.en, medium, medium.en, large-v1, or large-v2), a path to a converted
-            model directory, or a CTranslate2-converted Whisper model ID from the Hugging Face Hub.
+            small, small.en, distil-small.en, medium, medium.en, distil-medium.en, large-v1,
+            large-v2, large-v3, large, distil-large-v2 or distil-large-v3), a path to a
+            converted model directory, or a CTranslate2-converted Whisper model ID from the HF Hub.
            When a size or a model ID is configured, the converted model is downloaded
            from the Hugging Face Hub.
          device: Device to use for computation ("cpu", "cuda", "auto").
@@ -109,10 +122,18 @@ class WhisperModel:
            are saved in the standard Hugging Face cache directory.
          local_files_only:  If True, avoid downloading the file and return the path to the
            local cached file if it exists.
+          files: Load model files from the memory. This argument is a dictionary mapping file names
+            to file contents as file-like or bytes objects. If this is set, model_path acts as an
+            identifier for this model.
        """
        self.logger = get_logger()

-        if os.path.isdir(model_size_or_path):
+        tokenizer_bytes, preprocessor_bytes = None, None
+        if files:
+            model_path = model_size_or_path
+            tokenizer_bytes = files.pop("tokenizer.json", None)
+            preprocessor_bytes = files.pop("preprocessor_config.json", None)
+        elif os.path.isdir(model_size_or_path):
            model_path = model_size_or_path
        else:
            model_path = download_model(
@@ -128,17 +149,21 @@ class WhisperModel:
            compute_type=compute_type,
            intra_threads=cpu_threads,
            inter_threads=num_workers,
+            files=files,
+            **model_kwargs,
        )

        tokenizer_file = os.path.join(model_path, "tokenizer.json")
-        if os.path.isfile(tokenizer_file):
+        if tokenizer_bytes:
+            self.hf_tokenizer = tokenizers.Tokenizer.from_buffer(tokenizer_bytes)
+        elif os.path.isfile(tokenizer_file):
            self.hf_tokenizer = tokenizers.Tokenizer.from_file(tokenizer_file)
        else:
            self.hf_tokenizer = tokenizers.Tokenizer.from_pretrained(
                "openai/whisper-tiny" + ("" if self.model.is_multilingual else ".en")
            )
-
-        self.feature_extractor = FeatureExtractor()
+        self.feat_kwargs = self._get_feature_kwargs(model_path, preprocessor_bytes)
+        self.feature_extractor = FeatureExtractor(**self.feat_kwargs)
        self.num_samples_per_token = self.feature_extractor.hop_length * 2
        self.frames_per_second = (
            self.feature_extractor.sampling_rate // self.feature_extractor.hop_length
@@ -150,6 +175,29 @@ class WhisperModel:
        self.time_precision = 0.02
        self.max_length = 448

+    @property
+    def supported_languages(self) -> List[str]:
+        """The languages supported by the model."""
+        return list(_LANGUAGE_CODES) if self.model.is_multilingual else ["en"]
+
+    def _get_feature_kwargs(self, model_path, preprocessor_bytes=None) -> dict:
+        config = {}
+        try:
+            config_path = os.path.join(model_path, "preprocessor_config.json")
+            if preprocessor_bytes:
+                config = json.loads(preprocessor_bytes)
+            elif os.path.isfile(config_path):
+                with open(config_path, "r", encoding="utf-8") as file:
+                    config = json.load(file)
+            else:
+                return config
+            valid_keys = signature(FeatureExtractor.__init__).parameters.keys()
+            return {k: v for k, v in config.items() if k in valid_keys}
+        except json.JSONDecodeError as e:
+            self.logger.warning("Could not load preprocessor config: %s", e)
+
+        return config
+
    def transcribe(
        self,
        audio: Union[str, BinaryIO, np.ndarray],
@@ -159,6 +207,8 @@ class WhisperModel:
        best_of: int = 5,
        patience: float = 1,
        length_penalty: float = 1,
+        repetition_penalty: float = 1,
+        no_repeat_ngram_size: int = 0,
        temperature: Union[float, List[float], Tuple[float, ...]] = [
            0.0,
            0.2,
@@ -171,6 +221,7 @@ class WhisperModel:
        log_prob_threshold: Optional[float] = -1.0,
        no_speech_threshold: Optional[float] = 0.6,
        condition_on_previous_text: bool = True,
+        prompt_reset_on_temperature: float = 0.5,
        initial_prompt: Optional[Union[str, Iterable[int]]] = None,
        prefix: Optional[str] = None,
        suppress_blank: bool = True,
@@ -182,6 +233,13 @@ class WhisperModel:
        append_punctuations: str = "\"'.。,，!！?？:：”)]}、",
        vad_filter: bool = False,
        vad_parameters: Optional[Union[dict, VadOptions]] = None,
+        max_new_tokens: Optional[int] = None,
+        chunk_length: Optional[int] = None,
+        clip_timestamps: Union[str, List[float]] = "0",
+        hallucination_silence_threshold: Optional[float] = None,
+        hotwords: Optional[str] = None,
+        language_detection_threshold: Optional[float] = None,
+        language_detection_segments: int = 1,
    ) -> Tuple[Iterable[Segment], TranscriptionInfo]:
        """Transcribes an input file.

@@ -195,6 +253,9 @@ class WhisperModel:
          best_of: Number of candidates when sampling with non-zero temperature.
          patience: Beam search patience factor.
          length_penalty: Exponential length penalty constant.
+          repetition_penalty: Penalty applied to the score of previously generated tokens
+            (set > 1 to penalize).
+          no_repeat_ngram_size: Prevent repetitions of ngrams with this size (set 0 to disable).
          temperature: Temperature for sampling. It can be a tuple of temperatures,
            which will be successively used upon failures according to either
            `compression_ratio_threshold` or `log_prob_threshold`.
@@ -209,12 +270,14 @@ class WhisperModel:
            as a prompt for the next window; disabling may make the text inconsistent across
            windows, but the model becomes less prone to getting stuck in a failure loop,
            such as repetition looping or timestamps going out of sync.
+          prompt_reset_on_temperature: Resets prompt if temperature is above this value.
+            Arg has effect only if condition_on_previous_text is True.
          initial_prompt: Optional text string or iterable of token ids to provide as a
            prompt for the first window.
          prefix: Optional text to provide as a prefix for the first window.
          suppress_blank: Suppress blank outputs at the beginning of the sampling.
          suppress_tokens: List of token IDs to suppress. -1 will suppress a default set
-            of symbols as defined in the model config.json file.
+            of symbols as defined in `tokenizer.non_speech_tokens()`
          without_timestamps: Only sample text tokens.
          max_initial_timestamp: The initial timestamp cannot be later than this.
          word_timestamps: Extract word-level timestamps using the cross-attention pattern
@@ -228,7 +291,22 @@ class WhisperModel:
            https://github.com/snakers4/silero-vad.
          vad_parameters: Dictionary of Silero VAD parameters or VadOptions class (see available
            parameters and default values in the class `VadOptions`).
-
+          max_new_tokens: Maximum number of new tokens to generate per-chunk. If not set,
+            the maximum will be set by the default max_length.
+          chunk_length: The length of audio segments. If it is not None, it will overwrite the
+            default chunk_length of the FeatureExtractor.
+          clip_timestamps:
+            Comma-separated list start,end,start,end,... timestamps (in seconds) of clips to
+             process. The last end timestamp defaults to the end of the file.
+             vad_filter will be ignored if clip_timestamps is used.
+          hallucination_silence_threshold:
+            When word_timestamps is True, skip silent periods longer than this threshold
+             (in seconds) when a possible hallucination is detected
+          hotwords:
+            Hotwords/hint phrases to provide the model with. Has no effect if prefix is not None.
+          language_detection_threshold: If the maximum probability of the language tokens is higher
+           than this value, the language is detected.
+          language_detection_segments: Number of segments to consider for the language detection.
        Returns:
          A tuple with:

@@ -241,22 +319,24 @@ class WhisperModel:
            audio = decode_audio(audio, sampling_rate=sampling_rate)

        duration = audio.shape[0] / sampling_rate
+        duration_after_vad = duration

        self.logger.info(
            "Processing audio with duration %s", format_timestamp(duration)
        )

-        if vad_filter:
+        if vad_filter and clip_timestamps == "0":
            if vad_parameters is None:
                vad_parameters = VadOptions()
            elif isinstance(vad_parameters, dict):
                vad_parameters = VadOptions(**vad_parameters)
            speech_chunks = get_speech_timestamps(audio, vad_parameters)
            audio = collect_chunks(audio, speech_chunks)
+            duration_after_vad = audio.shape[0] / sampling_rate

            self.logger.info(
                "VAD filter removed %s of audio",
-                format_timestamp(duration - (audio.shape[0] / sampling_rate)),
+                format_timestamp(duration - duration_after_vad),
            )

            if self.logger.isEnabledFor(logging.DEBUG):
@@ -275,7 +355,7 @@ class WhisperModel:
        else:
            speech_chunks = None

-        features = self.feature_extractor(audio)
+        features = self.feature_extractor(audio, chunk_length=chunk_length)

        encoder_output = None
        all_language_probs = None
@@ -285,15 +365,62 @@ class WhisperModel:
                language = "en"
                language_probability = 1
            else:
-                segment = features[:, : self.feature_extractor.nb_max_frames]
-                encoder_output = self.encode(segment)
-                # results is a list of tuple[str, float] with language names and
-                # probabilities.
-                results = self.model.detect_language(encoder_output)[0]
-                # Parse language names to strip out markers
-                all_language_probs = [(token[2:-2], prob) for (token, prob) in results]
-                # Get top language token and probability
-                language, language_probability = all_language_probs[0]
+                if (
+                    language_detection_segments is None
+                    or language_detection_segments < 1
+                ):
+                    language_detection_segments = 1
+                start_timestamp = (
+                    float(clip_timestamps.split(",")[0])
+                    if isinstance(clip_timestamps, str)
+                    else clip_timestamps[0]
+                )
+                content_frames = (
+                    features.shape[-1] - self.feature_extractor.nb_max_frames
+                )
+                seek = (
+                    int(start_timestamp * self.frames_per_second)
+                    if start_timestamp * self.frames_per_second < content_frames
+                    else 0
+                )
+                end_frames = min(
+                    seek
+                    + self.feature_extractor.nb_max_frames
+                    * language_detection_segments,
+                    content_frames,
+                )
+                detected_language_info = {}
+                while seek <= end_frames:
+                    segment = features[
+                        :, seek : seek + self.feature_extractor.nb_max_frames
+                    ]
+                    encoder_output = self.encode(segment)
+                    # results is a list of tuple[str, float] with language names and
+                    # probabilities.
+                    results = self.model.detect_language(encoder_output)[0]
+                    # Parse language names to strip out markers
+                    all_language_probs = [
+                        (token[2:-2], prob) for (token, prob) in results
+                    ]
+                    # Get top language token and probability
+                    language, language_probability = all_language_probs[0]
+                    if (
+                        language_detection_threshold is None
+                        or language_probability > language_detection_threshold
+                    ):
+                        break
+                    detected_language_info.setdefault(language, []).append(
+                        language_probability
+                    )
+                    seek += segment.shape[-1]
+                else:
+                    # If no language detected for all segments, the majority vote of the highest
+                    # projected languages for all segments is used to determine the language.
+                    language = max(
+                        detected_language_info,
+                        key=lambda lang: len(detected_language_info[lang]),
+                    )
+                    language_probability = max(detected_language_info[language])

                self.logger.info(
                    "Detected language '%s' with probability %.2f",
@@ -301,6 +428,13 @@ class WhisperModel:
                    language_probability,
                )
        else:
+            if not self.model.is_multilingual and language != "en":
+                self.logger.warning(
+                    "The current model is English-only but the language parameter is set to '%s'; "
+                    "using 'en' instead." % language
+                )
+                language = "en"
+
            language_probability = 1

        tokenizer = Tokenizer(
@@ -315,22 +449,33 @@ class WhisperModel:
            best_of=best_of,
            patience=patience,
            length_penalty=length_penalty,
+            repetition_penalty=repetition_penalty,
+            no_repeat_ngram_size=no_repeat_ngram_size,
            log_prob_threshold=log_prob_threshold,
            no_speech_threshold=no_speech_threshold,
            compression_ratio_threshold=compression_ratio_threshold,
            condition_on_previous_text=condition_on_previous_text,
+            prompt_reset_on_temperature=prompt_reset_on_temperature,
            temperatures=(
                temperature if isinstance(temperature, (list, tuple)) else [temperature]
            ),
            initial_prompt=initial_prompt,
            prefix=prefix,
            suppress_blank=suppress_blank,
-            suppress_tokens=get_suppressed_tokens(tokenizer, suppress_tokens),
+            suppress_tokens=(
+                get_suppressed_tokens(tokenizer, suppress_tokens)
+                if suppress_tokens
+                else suppress_tokens
+            ),
            without_timestamps=without_timestamps,
            max_initial_timestamp=max_initial_timestamp,
            word_timestamps=word_timestamps,
            prepend_punctuations=prepend_punctuations,
            append_punctuations=append_punctuations,
+            max_new_tokens=max_new_tokens,
+            clip_timestamps=clip_timestamps,
+            hallucination_silence_threshold=hallucination_silence_threshold,
+            hotwords=hotwords,
        )

        segments = self.generate_segments(features, tokenizer, options, encoder_output)
@@ -342,11 +487,11 @@ class WhisperModel:
            language=language,
            language_probability=language_probability,
            duration=duration,
+            duration_after_vad=duration_after_vad,
            transcription_options=options,
            vad_options=vad_parameters,
            all_language_probs=all_language_probs,
        )
-
        return segments, info

    def generate_segments(
@@ -357,9 +502,37 @@ class WhisperModel:
        encoder_output: Optional[ctranslate2.StorageView] = None,
    ) -> Iterable[Segment]:
        content_frames = features.shape[-1] - self.feature_extractor.nb_max_frames
+        content_duration = float(content_frames * self.feature_extractor.time_per_frame)
+
+        if isinstance(options.clip_timestamps, str):
+            options = options._replace(
+                clip_timestamps=[
+                    float(ts)
+                    for ts in (
+                        options.clip_timestamps.split(",")
+                        if options.clip_timestamps
+                        else []
+                    )
+                ]
+            )
+        seek_points: List[int] = [
+            round(ts * self.frames_per_second) for ts in options.clip_timestamps
+        ]
+        if len(seek_points) == 0:
+            seek_points.append(0)
+        if len(seek_points) % 2 == 1:
+            seek_points.append(content_frames)
+        seek_clips: List[Tuple[int, int]] = list(
+            zip(seek_points[::2], seek_points[1::2])
+        )
+
+        punctuation = "\"'“¿([{-\"'.。,，!！?？:：”)]}、"
+
        idx = 0
-        seek = 0
+        clip_idx = 0
+        seek = seek_clips[clip_idx][0]
        all_tokens = []
+        all_prompt_text = []
        prompt_reset_since = 0

        if options.initial_prompt is not None:
@@ -371,13 +544,34 @@ class WhisperModel:
                all_tokens.extend(options.initial_prompt)

        last_speech_timestamp = 0.0
-        while seek < content_frames:
+        # NOTE: This loop is obscurely flattened to make the diff readable.
+        # A later commit should turn this into a simpler nested loop.
+        # for seek_clip_start, seek_clip_end in seek_clips:
+        #     while seek < seek_clip_end
+        while clip_idx < len(seek_clips):
+            seek_clip_start, seek_clip_end = seek_clips[clip_idx]
+            if seek_clip_end > content_frames:
+                seek_clip_end = content_frames
+            if seek < seek_clip_start:
+                seek = seek_clip_start
+            if seek >= seek_clip_end:
+                clip_idx += 1
+                if clip_idx < len(seek_clips):
+                    seek = seek_clips[clip_idx][0]
+                continue
            time_offset = seek * self.feature_extractor.time_per_frame
-            segment = features[:, seek : seek + self.feature_extractor.nb_max_frames]
-            segment_size = min(
-                self.feature_extractor.nb_max_frames, content_frames - seek
+            window_end_time = float(
+                (seek + self.feature_extractor.nb_max_frames)
+                * self.feature_extractor.time_per_frame
            )
+            segment_size = min(
+                self.feature_extractor.nb_max_frames,
+                content_frames - seek,
+                seek_clip_end - seek,
+            )
+            segment = features[:, seek : seek + segment_size]
            segment_duration = segment_size * self.feature_extractor.time_per_frame
+            segment = pad_or_trim(segment, self.feature_extractor.nb_max_frames)

            if self.logger.isEnabledFor(logging.DEBUG):
                self.logger.debug(
@@ -390,9 +584,10 @@ class WhisperModel:
                previous_tokens,
                without_timestamps=options.without_timestamps,
                prefix=options.prefix if seek == 0 else None,
+                hotwords=options.hotwords,
            )

-            if encoder_output is None:
+            if seek > 0 or encoder_output is None:
                encoder_output = self.encode(segment)

            (
@@ -429,10 +624,33 @@ class WhisperModel:
            previous_seek = seek
            current_segments = []

+            # anomalous words are very long/short/improbable
+            def word_anomaly_score(word: dict) -> float:
+                probability = word.get("probability", 0.0)
+                duration = word["end"] - word["start"]
+                score = 0.0
+                if probability < 0.15:
+                    score += 1.0
+                if duration < 0.133:
+                    score += (0.133 - duration) * 15
+                if duration > 2.0:
+                    score += duration - 2.0
+                return score
+
+            def is_segment_anomaly(segment: Optional[dict]) -> bool:
+                if segment is None or not segment["words"]:
+                    return False
+                words = [w for w in segment["words"] if w["word"] not in punctuation]
+                words = words[:8]
+                score = sum(word_anomaly_score(w) for w in words)
+                return score >= 3 or score + 0.01 >= len(words)
+
+            def next_words_segment(segments: List[dict]) -> Optional[dict]:
+                return next((s for s in segments if s["words"]), None)
+
            single_timestamp_ending = (
                len(tokens) >= 2
-                and tokens[-2] < tokenizer.timestamp_begin
-                and tokens[-1] >= tokenizer.timestamp_begin
+                and tokens[-2] < tokenizer.timestamp_begin <= tokens[-1]
            )

            consecutive_timestamps = [
@@ -515,20 +733,62 @@ class WhisperModel:
                    last_speech_timestamp=last_speech_timestamp,
                )

-                word_end_timestamps = [
-                    w["end"] for s in current_segments for w in s["words"]
-                ]
-                if len(word_end_timestamps) > 0:
-                    last_speech_timestamp = word_end_timestamps[-1]
-                if not single_timestamp_ending and len(word_end_timestamps) > 0:
-                    seek_shift = round(
-                        (word_end_timestamps[-1] - time_offset) * self.frames_per_second
-                    )
+                if not single_timestamp_ending:
+                    last_word_end = get_end(current_segments)
+                    if last_word_end is not None and last_word_end > time_offset:
+                        seek = round(last_word_end * self.frames_per_second)

-                    if seek_shift > 0:
-                        seek = previous_seek + seek_shift
+                # skip silence before possible hallucinations
+                if options.hallucination_silence_threshold is not None:
+                    threshold = options.hallucination_silence_threshold

-            encoder_output = None
+                    # if first segment might be a hallucination, skip leading silence
+                    first_segment = next_words_segment(current_segments)
+                    if first_segment is not None and is_segment_anomaly(first_segment):
+                        gap = first_segment["start"] - time_offset
+                        if gap > threshold:
+                            seek = previous_seek + round(gap * self.frames_per_second)
+                            continue
+
+                    # skip silence before any possible hallucination that is surrounded
+                    # by silence or more hallucinations
+                    hal_last_end = last_speech_timestamp
+                    for si in range(len(current_segments)):
+                        segment = current_segments[si]
+                        if not segment["words"]:
+                            continue
+                        if is_segment_anomaly(segment):
+                            next_segment = next_words_segment(
+                                current_segments[si + 1 :]
+                            )
+                            if next_segment is not None:
+                                hal_next_start = next_segment["words"][0]["start"]
+                            else:
+                                hal_next_start = time_offset + segment_duration
+                            silence_before = (
+                                segment["start"] - hal_last_end > threshold
+                                or segment["start"] < threshold
+                                or segment["start"] - time_offset < 2.0
+                            )
+                            silence_after = (
+                                hal_next_start - segment["end"] > threshold
+                                or is_segment_anomaly(next_segment)
+                                or window_end_time - segment["end"] < 2.0
+                            )
+                            if silence_before and silence_after:
+                                seek = round(
+                                    max(time_offset + 1, segment["start"])
+                                    * self.frames_per_second
+                                )
+                                if content_duration - segment["end"] < threshold:
+                                    seek = content_frames
+                                current_segments[si:] = []
+                                break
+                        hal_last_end = segment["end"]
+
+                last_word_end = get_end(current_segments)
+                if last_word_end is not None:
+                    last_speech_timestamp = last_word_end

            for segment in current_segments:
                tokens = segment["tokens"]
@@ -537,8 +797,16 @@ class WhisperModel:
                if segment["start"] == segment["end"] or not text.strip():
                    continue

-                all_tokens.extend(tokens)
-                idx += 1
+                check_prompt_num = 1
+                if all(
+                    [
+                        text.strip() != i.strip()
+                        for i in all_prompt_text[-check_prompt_num:]
+                    ]
+                ):
+                    all_tokens.extend(tokens)
+                    all_prompt_text.append(text)
+                    idx += 1

                yield Segment(
                    id=idx,
@@ -558,7 +826,17 @@ class WhisperModel:
                    ),
                )

-            if not options.condition_on_previous_text or temperature > 0.5:
+            if (
+                not options.condition_on_previous_text
+                or temperature > options.prompt_reset_on_temperature
+            ):
+                if options.condition_on_previous_text:
+                    self.logger.debug(
+                        "Reset prompt. prompt_reset_on_temperature threshold is met %f > %f",
+                        temperature,
+                        options.prompt_reset_on_temperature,
+                    )
+
                prompt_reset_since = len(all_tokens)

    def encode(self, features: np.ndarray) -> ctranslate2.StorageView:
@@ -578,14 +856,28 @@ class WhisperModel:
        tokenizer: Tokenizer,
        options: TranscriptionOptions,
    ) -> Tuple[ctranslate2.models.WhisperGenerationResult, float, float, float]:
-        result = None
-        avg_logprob = None
-        final_temperature = None
-        compression_ratio = None
+        decode_result = None
+        all_results = []
+        below_cr_threshold_results = []

        max_initial_timestamp_index = int(
            round(options.max_initial_timestamp / self.time_precision)
        )
+        if options.max_new_tokens is not None:
+            max_length = len(prompt) + options.max_new_tokens
+        else:
+            max_length = self.max_length
+
+        if max_length > self.max_length:
+            raise ValueError(
+                f"The length of the prompt is {len(prompt)}, and the `max_new_tokens` "
+                f"{max_length - len(prompt)}. Thus, the combined length of the prompt "
+                f"and `max_new_tokens` is: {max_length}. This exceeds the "
+                f"`max_length` of the Whisper model: {self.max_length}. "
+                "You should either reduce the length of your prompt, or "
+                "reduce the value of `max_new_tokens`, "
+                f"so that their combined length is less that {self.max_length}."
+            )

        for temperature in options.temperatures:
            if temperature > 0:
@@ -601,12 +893,13 @@ class WhisperModel:
                    "patience": options.patience,
                }

-            final_temperature = temperature
            result = self.model.generate(
                encoder_output,
                [prompt],
                length_penalty=options.length_penalty,
-                max_length=self.max_length,
+                repetition_penalty=options.repetition_penalty,
+                no_repeat_ngram_size=options.no_repeat_ngram_size,
+                max_length=max_length,
                return_scores=True,
                return_no_speech_prob=True,
                suppress_blank=options.suppress_blank,
@@ -625,20 +918,28 @@ class WhisperModel:
            text = tokenizer.decode(tokens).strip()
            compression_ratio = get_compression_ratio(text)

+            decode_result = (
+                result,
+                avg_logprob,
+                temperature,
+                compression_ratio,
+            )
+            all_results.append(decode_result)
+
            needs_fallback = False

-            if (
-                options.compression_ratio_threshold is not None
-                and compression_ratio > options.compression_ratio_threshold
-            ):
-                needs_fallback = True  # too repetitive
+            if options.compression_ratio_threshold is not None:
+                if compression_ratio > options.compression_ratio_threshold:
+                    needs_fallback = True  # too repetitive

-                self.logger.debug(
-                    "Compression ratio threshold is not met with temperature %.1f (%f > %f)",
-                    temperature,
-                    compression_ratio,
-                    options.compression_ratio_threshold,
-                )
+                    self.logger.debug(
+                        "Compression ratio threshold is not met with temperature %.1f (%f > %f)",
+                        temperature,
+                        compression_ratio,
+                        options.compression_ratio_threshold,
+                    )
+                else:
+                    below_cr_threshold_results.append(decode_result)

            if (
                options.log_prob_threshold is not None
@@ -656,13 +957,27 @@ class WhisperModel:
            if (
                options.no_speech_threshold is not None
                and result.no_speech_prob > options.no_speech_threshold
+                and options.log_prob_threshold is not None
+                and avg_logprob < options.log_prob_threshold
            ):
                needs_fallback = False  # silence

            if not needs_fallback:
                break
+        else:
+            # all failed, select the result with the highest average log probability
+            decode_result = max(
+                below_cr_threshold_results or all_results, key=lambda x: x[1]
+            )
+            # to pass final temperature for prompt_reset_on_temperature
+            decode_result = (
+                decode_result[0],
+                decode_result[1],
+                temperature,
+                decode_result[3],
+            )

-        return result, avg_logprob, final_temperature, compression_ratio
+        return decode_result

    def get_prompt(
        self,
@@ -670,12 +985,19 @@ class WhisperModel:
        previous_tokens: List[int],
        without_timestamps: bool = False,
        prefix: Optional[str] = None,
+        hotwords: Optional[str] = None,
    ) -> List[int]:
        prompt = []

-        if previous_tokens:
+        if previous_tokens or (hotwords and not prefix):
            prompt.append(tokenizer.sot_prev)
-            prompt.extend(previous_tokens[-(self.max_length // 2 - 1) :])
+            if hotwords and not prefix:
+                hotwords_tokens = tokenizer.encode(" " + hotwords.strip())
+                if len(hotwords_tokens) >= self.max_length // 2:
+                    hotwords_tokens = hotwords_tokens[: self.max_length // 2 - 1]
+                prompt.extend(hotwords_tokens)
+            if previous_tokens:
+                prompt.extend(previous_tokens[-(self.max_length // 2 - 1) :])

        prompt.extend(tokenizer.sot_sequence)

@@ -686,6 +1008,8 @@ class WhisperModel:
            prefix_tokens = tokenizer.encode(" " + prefix.strip())
            if len(prefix_tokens) >= self.max_length // 2:
                prefix_tokens = prefix_tokens[: self.max_length // 2 - 1]
+            if not without_timestamps:
+                prompt.append(tokenizer.timestamp_begin)
            prompt.extend(prefix_tokens)

        return prompt
@@ -699,7 +1023,7 @@ class WhisperModel:
        prepend_punctuations: str,
        append_punctuations: str,
        last_speech_timestamp: float,
-    ):
+    ) -> None:
        if len(segments) == 0:
            return

@@ -715,13 +1039,12 @@ class WhisperModel:
        word_durations = np.array([word["end"] - word["start"] for word in alignment])
        word_durations = word_durations[word_durations.nonzero()]
        median_duration = np.median(word_durations) if len(word_durations) > 0 else 0.0
+        median_duration = min(0.7, float(median_duration))
        max_duration = median_duration * 2

        # hack: truncate long words at sentence boundaries.
        # a better segmentation algorithm based on VAD should be able to replace this.
        if len(word_durations) > 0:
-            median_duration = np.median(word_durations)
-            max_duration = median_duration * 2
            sentence_end_marks = ".。!！?？"
            # ensure words at sentence boundaries
            # are not longer than twice the median word duration.
@@ -838,6 +1161,13 @@ class WhisperModel:
        words, word_tokens = tokenizer.split_to_word_tokens(
            text_tokens + [tokenizer.eot]
        )
+        if len(word_tokens) <= 1:
+            # return on eot only
+            # >>> np.pad([], (1, 0))
+            # array([0.])
+            # This results in crashes when we lookup jump_times with float, like
+            # IndexError: arrays used as indices must be of integer (or boolean) type
+            return []
        word_boundaries = np.pad(np.cumsum([len(t) for t in word_tokens[:-1]]), (1, 0))
        if len(word_boundaries) <= 1:
            return []
@@ -907,14 +1237,18 @@ def get_compression_ratio(text: str) -> float:
    return len(text_bytes) / len(zlib.compress(text_bytes))


-def get_suppressed_tokens(tokenizer, suppress_tokens):
-    if not suppress_tokens or -1 in suppress_tokens:
-        return suppress_tokens
+def get_suppressed_tokens(
+    tokenizer: Tokenizer,
+    suppress_tokens: Tuple[int],
+) -> Optional[List[int]]:
+    if -1 in suppress_tokens:
+        suppress_tokens = [t for t in suppress_tokens if t >= 0]
+        suppress_tokens.extend(tokenizer.non_speech_tokens)
+    elif suppress_tokens is None or len(suppress_tokens) == 0:
+        suppress_tokens = []  # interpret empty string as an empty list
+    else:
+        assert isinstance(suppress_tokens, list), "suppress_tokens must be a list"

-    suppress_tokens = list(suppress_tokens)
-
-    # Ensure the following special tokens are suppressed when the user does
-    # not use the default set (-1).
    suppress_tokens.extend(
        [
            tokenizer.transcribe,
@@ -925,10 +1259,10 @@ def get_suppressed_tokens(tokenizer, suppress_tokens):
        ]
    )

-    return sorted(set(suppress_tokens))
+    return tuple(sorted(set(suppress_tokens)))


-def merge_punctuations(alignment: List[dict], prepended: str, appended: str):
+def merge_punctuations(alignment: List[dict], prepended: str, appended: str) -> None:
    # merge prepended punctuations
    i = len(alignment) - 2
    j = len(alignment) - 1
--- a/faster_whisper/utils.py
+++ b/faster_whisper/utils.py
@@ -2,25 +2,36 @@ import logging
 import os
 import re

-from typing import Optional
+from typing import List, Optional

 import huggingface_hub
 import requests

 from tqdm.auto import tqdm

-_MODELS = (
-    "tiny.en",
-    "tiny",
-    "base.en",
-    "base",
-    "small.en",
-    "small",
-    "medium.en",
-    "medium",
-    "large-v1",
-    "large-v2",
-)
+_MODELS = {
+    "tiny.en": "Systran/faster-whisper-tiny.en",
+    "tiny": "Systran/faster-whisper-tiny",
+    "base.en": "Systran/faster-whisper-base.en",
+    "base": "Systran/faster-whisper-base",
+    "small.en": "Systran/faster-whisper-small.en",
+    "small": "Systran/faster-whisper-small",
+    "medium.en": "Systran/faster-whisper-medium.en",
+    "medium": "Systran/faster-whisper-medium",
+    "large-v1": "Systran/faster-whisper-large-v1",
+    "large-v2": "Systran/faster-whisper-large-v2",
+    "large-v3": "Systran/faster-whisper-large-v3",
+    "large": "Systran/faster-whisper-large-v3",
+    "distil-large-v2": "Systran/faster-distil-whisper-large-v2",
+    "distil-medium.en": "Systran/faster-distil-whisper-medium.en",
+    "distil-small.en": "Systran/faster-distil-whisper-small.en",
+    "distil-large-v3": "Systran/faster-distil-whisper-large-v3",
+}
+
+
+def available_models() -> List[str]:
+    """Returns the names of available models."""
+    return list(_MODELS.keys())


 def get_assets_path():
@@ -41,12 +52,12 @@ def download_model(
 ):
    """Downloads a CTranslate2 Whisper model from the Hugging Face Hub.

-    The model is downloaded from https://huggingface.co/guillaumekln.
-
    Args:
-      size_or_id: Size of the model to download (tiny, tiny.en, base, base.en, small, small.en,
-        medium, medium.en, large-v1, or large-v2), or a CTranslate2-converted model ID
-        from the Hugging Face Hub (e.g. guillaumekln/faster-whisper-large-v2).
+      size_or_id: Size of the model to download from https://huggingface.co/Systran
+        (tiny, tiny.en, base, base.en, small, small.en, distil-small.en, medium, medium.en,
+        distil-medium.en, large-v1, large-v2, large-v3, large, distil-large-v2,
+        distil-large-v3), or a CTranslate2-converted model ID from the Hugging Face Hub
+        (e.g. Systran/faster-whisper-large-v3).
      output_dir: Directory where the model should be saved. If not set, the model is saved in
        the cache directory.
      local_files_only:  If True, avoid downloading the file and return the path to the local
@@ -62,16 +73,16 @@ def download_model(
    if re.match(r".*/.*", size_or_id):
        repo_id = size_or_id
    else:
-        if size_or_id not in _MODELS:
+        repo_id = _MODELS.get(size_or_id)
+        if repo_id is None:
            raise ValueError(
                "Invalid model size '%s', expected one of: %s"
-                % (size_or_id, ", ".join(_MODELS))
+                % (size_or_id, ", ".join(_MODELS.keys()))
            )

-        repo_id = "guillaumekln/faster-whisper-%s" % size_or_id
-
    allow_patterns = [
        "config.json",
+        "preprocessor_config.json",
        "model.bin",
        "tokenizer.json",
        "vocabulary.*",
@@ -137,3 +148,10 @@ class disabled_tqdm(tqdm):
    def __init__(self, *args, **kwargs):
        kwargs["disable"] = True
        super().__init__(*args, **kwargs)
+
+
+def get_end(segments: List[dict]) -> Optional[float]:
+    return next(
+        (w["end"] for s in reversed(segments) for w in reversed(s["words"])),
+        segments[-1]["end"] if segments else None,
+    )
--- a/faster_whisper/vad.py
+++ b/faster_whisper/vad.py
@@ -1,7 +1,6 @@
 import bisect
 import functools
 import os
-import warnings

 from typing import List, NamedTuple, Optional

@@ -25,9 +24,6 @@ class VadOptions(NamedTuple):
        split aggressively just before max_speech_duration_s.
      min_silence_duration_ms: In the end of each speech chunk wait for min_silence_duration_ms
        before separating it
-      window_size_samples: Audio chunks of window_size_samples size are fed to the silero VAD model.
-        WARNING! Silero VAD models were trained using 512, 1024, 1536 samples for 16000 sample rate.
-        Values other than these may affect model performance!!
      speech_pad_ms: Final speech chunks are padded by speech_pad_ms each side
    """

@@ -35,7 +31,6 @@ class VadOptions(NamedTuple):
    min_speech_duration_ms: int = 250
    max_speech_duration_s: float = float("inf")
    min_silence_duration_ms: int = 2000
-    window_size_samples: int = 1024
    speech_pad_ms: int = 400


@@ -61,15 +56,8 @@ def get_speech_timestamps(
    min_speech_duration_ms = vad_options.min_speech_duration_ms
    max_speech_duration_s = vad_options.max_speech_duration_s
    min_silence_duration_ms = vad_options.min_silence_duration_ms
-    window_size_samples = vad_options.window_size_samples
+    window_size_samples = 512
    speech_pad_ms = vad_options.speech_pad_ms
-
-    if window_size_samples not in [512, 1024, 1536]:
-        warnings.warn(
-            "Unusual window_size_samples! Supported window_size_samples:\n"
-            " - [512, 1024, 1536] for 16000 sampling_rate"
-        )
-
    sampling_rate = 16000
    min_speech_samples = sampling_rate * min_speech_duration_ms / 1000
    speech_pad_samples = sampling_rate * speech_pad_ms / 1000
@@ -84,14 +72,14 @@ def get_speech_timestamps(
    audio_length_samples = len(audio)

    model = get_vad_model()
-    state = model.get_initial_state(batch_size=1)
+    state, context = model.get_initial_states(batch_size=1)

    speech_probs = []
    for current_start_sample in range(0, audio_length_samples, window_size_samples):
        chunk = audio[current_start_sample : current_start_sample + window_size_samples]
        if len(chunk) < window_size_samples:
            chunk = np.pad(chunk, (0, int(window_size_samples - len(chunk))))
-        speech_prob, state = model(chunk, state, sampling_rate)
+        speech_prob, state, context = model(chunk, state, context, sampling_rate)
        speech_probs.append(speech_prob)

    triggered = False
@@ -261,12 +249,12 @@ class SileroVADModel:
            sess_options=opts,
        )

-    def get_initial_state(self, batch_size: int):
-        h = np.zeros((2, batch_size, 64), dtype=np.float32)
-        c = np.zeros((2, batch_size, 64), dtype=np.float32)
-        return h, c
+    def get_initial_states(self, batch_size: int):
+        state = np.zeros((2, batch_size, 128), dtype=np.float32)
+        context = np.zeros((batch_size, 64), dtype=np.float32)
+        return state, context

-    def __call__(self, x, state, sr: int):
+    def __call__(self, x, state, context, sr: int):
        if len(x.shape) == 1:
            x = np.expand_dims(x, 0)
        if len(x.shape) > 2:
@@ -276,16 +264,15 @@ class SileroVADModel:
        if sr / x.shape[1] > 31.25:
            raise ValueError("Input audio chunk is too short")

-        h, c = state
+        x = np.concatenate([context, x], axis=1)

        ort_inputs = {
            "input": x,
-            "h": h,
-            "c": c,
+            "state": state,
            "sr": np.array(sr, dtype="int64"),
        }

-        out, h, c = self.session.run(None, ort_inputs)
-        state = (h, c)
+        out, state = self.session.run(None, ort_inputs)
+        context = x[..., -64:]

-        return out, state
+        return out, state, context
--- a/faster_whisper/version.py
+++ b/faster_whisper/version.py
@@ -1,3 +1,3 @@
 """Version information."""

-__version__ = "0.6.0"
+__version__ = "1.0.3"
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,5 +1,5 @@
-av==10.*
-ctranslate2>=3.10,<4
+av>=11.0,<13
+ctranslate2>=4.0,<5
 huggingface_hub>=0.13
-tokenizers==0.13.*
+tokenizers>=0.13,<1
 onnxruntime>=1.14,<2
--- a/setup.py
+++ b/setup.py
@@ -37,7 +37,7 @@ setup(
    long_description=get_long_description(),
    long_description_content_type="text/markdown",
    author="Guillaume Klein",
-    url="https://github.com/guillaumekln/faster-whisper",
+    url="https://github.com/SYSTRAN/faster-whisper",
    classifiers=[
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
--- a/tests/test_transcribe.py
+++ b/tests/test_transcribe.py
@@ -1,6 +1,13 @@
 import os

 from faster_whisper import WhisperModel, decode_audio
+from faster_whisper.tokenizer import Tokenizer
+from faster_whisper.transcribe import get_suppressed_tokens
+
+
+def test_supported_languages():
+    model = WhisperModel("tiny.en")
+    assert model.supported_languages == ["en"]


 def test_transcribe(jfk_path):
@@ -34,6 +41,24 @@ def test_transcribe(jfk_path):
    assert segment.end == segment.words[-1].end


+def test_prefix_with_timestamps(jfk_path):
+    model = WhisperModel("tiny")
+    segments, _ = model.transcribe(jfk_path, prefix="And so my fellow Americans")
+    segments = list(segments)
+
+    assert len(segments) == 1
+
+    segment = segments[0]
+
+    assert segment.text == (
+        " And so my fellow Americans ask not what your country can do for you, "
+        "ask what you can do for your country."
+    )
+
+    assert segment.start == 0
+    assert 10 < segment.end < 11
+
+
 def test_vad(jfk_path):
    model = WhisperModel("tiny")
    segments, info = model.transcribe(
@@ -74,3 +99,109 @@ def test_stereo_diarization(data_dir):
    segments, _ = model.transcribe(right)
    transcription = "".join(segment.text for segment in segments).strip()
    assert transcription == "The horizon seems extremely distant."
+
+
+def test_suppressed_tokens_minus_1():
+    model = WhisperModel("tiny.en")
+
+    tokenizer = Tokenizer(model.hf_tokenizer, False)
+    tokens = get_suppressed_tokens(tokenizer, [-1])
+    assert tokens == (
+        1,
+        2,
+        7,
+        8,
+        9,
+        10,
+        14,
+        25,
+        26,
+        27,
+        28,
+        29,
+        31,
+        58,
+        59,
+        60,
+        61,
+        62,
+        63,
+        90,
+        91,
+        92,
+        93,
+        357,
+        366,
+        438,
+        532,
+        685,
+        705,
+        796,
+        930,
+        1058,
+        1220,
+        1267,
+        1279,
+        1303,
+        1343,
+        1377,
+        1391,
+        1635,
+        1782,
+        1875,
+        2162,
+        2361,
+        2488,
+        3467,
+        4008,
+        4211,
+        4600,
+        4808,
+        5299,
+        5855,
+        6329,
+        7203,
+        9609,
+        9959,
+        10563,
+        10786,
+        11420,
+        11709,
+        11907,
+        13163,
+        13697,
+        13700,
+        14808,
+        15306,
+        16410,
+        16791,
+        17992,
+        19203,
+        19510,
+        20724,
+        22305,
+        22935,
+        27007,
+        30109,
+        30420,
+        33409,
+        34949,
+        40283,
+        40493,
+        40549,
+        47282,
+        49146,
+        50257,
+        50357,
+        50358,
+        50359,
+        50360,
+    )
+
+
+def test_suppressed_tokens_minus_value():
+    model = WhisperModel("tiny.en")
+
+    tokenizer = Tokenizer(model.hf_tokenizer, False)
+    tokens = get_suppressed_tokens(tokenizer, [13])
+    assert tokens == (13, 50257, 50357, 50358, 50359, 50360)
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -1,6 +1,12 @@
 import os

-from faster_whisper import download_model
+from faster_whisper import available_models, download_model
+
+
+def test_available_models():
+    models = available_models()
+    assert isinstance(models, list)
+    assert "tiny" in models


 def test_download_model(tmpdir):
Author	SHA1	Message	Date
heimoshuiyu	28a4d11a73	Revert "Merge remote-tracking branch 'upstream/master' into prompt" This reverts commit `6e42088656`, reversing changes made to `4a59bb011d`.	2024-09-12 00:49:31 +08:00
heimoshuiyu	6e42088656	Merge remote-tracking branch 'upstream/master' into prompt	2024-09-04 17:48:06 +08:00
Mahmoud Ashraf	d57c5b40b0	Remove the usage of `transformers.pipeline` from `BatchedInferencePipeline` and fix word timestamps for batched inference (#921 ) * fix word timestamps for batched inference * remove hf pipeline	2024-07-27 09:02:58 +07:00
zh-plus	83a368e98a	Make vad-related parameters configurable for batched inference. (#923 )	2024-07-24 09:00:32 +07:00
Jilt Sebastian	eb8390233c	New PR for Faster Whisper: Batching Support, Speed Boosts, and Quality Enhancements (#856 ) Batching Support, Speed Boosts, and Quality Enhancements --------- Co-authored-by: Hargun Mujral <83234565+hargunmujral@users.noreply.github.com> Co-authored-by: MahmoudAshraf97 <hassouna97.ma@gmail.com>	2024-07-18 16:48:52 +07:00
heimoshuiyu	4a59bb011d	Merge remote-tracking branch 'upstream/master' into prompt	2024-07-10 10:16:35 +08:00
trungkienbkhn	fbcf58bf98	Fix language detection with non-speech audio (#895 )	2024-07-05 14:43:45 +07:00
Jordi Mas	1195359984	Filter out non_speech_tokens in suppressed tokens (#898 ) * Filter out non_speech_tokens in suppressed tokens	2024-07-05 14:43:11 +07:00
trungkienbkhn	c22db5125d	Bump version to 1.0.3 (#887 )	2024-07-01 16:36:12 +07:00
ABen	8862bee1f8	Improve language detection when using clip_timestamps (#867 )	2024-07-01 16:12:45 +07:00
Ki Hoon Kim	8d400e9870	Upgrade to Silero-Vad V5 (#884 ) * Fix window_size_samples to 512 * Update SileroVADModel * Replace ONNX file with V5 version	2024-07-01 15:40:37 +07:00
Fedir Zadniprovskyi	bced5f04c0	docs: add 'faster-whisper-server' community integration (#861 ) Co-authored-by: Fedir Zadniprovskyi <github.g1k56@simplelogin.com>	2024-06-05 22:27:41 +07:00
Fedir Zadniprovskyi	65551c081f	Docker file improvements (#848 ) Docker file improvements Co-authored-by: Fedir Zadniprovskyi <github.g1k56@simplelogin.com>	2024-05-20 09:13:19 +07:00
Napuh	f53be1e811	Add distil models to WhisperModel init and download_model docstrings (#847 ) * chore: add distil models to WhisperModel init docstring and download_model docstring	2024-05-20 08:51:22 +07:00
Natanael Tan	4acdb5c619	Fix #839 incorrect clip_timestamps being used in model (#842 ) * Fix #839 Changed the code from updating the TranscriptionOptions class instead of the options object which likely was the cause of unexpected behaviour	2024-05-17 16:35:07 +07:00
Peter Krantz	a1c3583c96	Update README.md (#841 ) Spelling correction for copy/pasters	2024-05-17 15:24:47 +07:00
trungkienbkhn	2036d12634	Add Dockerfile example (#828 )	2024-05-13 16:33:09 +07:00
trungkienbkhn	2f6913efc8	Bump version to 1.0.2 (#816 )	2024-05-06 09:02:54 +07:00
ddorian	e11d58599d	Allow av to include version 12. (#819 )	2024-05-06 08:57:35 +07:00
Keating Reid	49a80eb8a8	Clarify documentation for hotwords (#817 ) * Clarify documentation for hotwords * Remove redundant type specifications	2024-05-06 08:52:59 +07:00
trungkienbkhn	8d5e6d56d9	Support initializing more whisper model args (#807 )	2024-05-04 15:12:59 +07:00
trungkienbkhn	6eec07739e	Add benchmarking logic for memory, wer and speed (#773 )	2024-05-04 15:12:43 +07:00
jax	847fec4492	Feature/add hotwords (#731 ) * add hotword params --------- Co-authored-by: jax <jax_builder@gamil.com>	2024-05-04 15:11:52 +07:00
Keating Reid	46080e584e	Loosening tokenizers version constraint (#804 )	2024-05-04 15:10:24 +07:00
Sidharth Rajaram	3d1de60ef3	CUDA version and updated installation instructions (#785 ) * CUDA version note and updated instructions in README * ctranslate2 downgrade note, cuDNN v9 consideration * clearer note on cuDNN v9 package	2024-05-04 15:09:59 +07:00
heimoshuiyu	4ee1d54c14	Merge branch 'master' into prompt	2024-04-08 20:56:49 +08:00
otakutyrant	91c8307aa6	make faster_whisper.assets as a valid python package to distribute (#772 ) (#774 )	2024-04-02 18:22:22 +02:00
Purfview	b024972a56	Foolproof: Disable VAD if clip_timestamps is in use (#769 ) * Foolproof: Disable VAD if clip_timestamps is in use Prevent silly things to happen.	2024-04-02 18:20:34 +02:00
Purfview	8ae82c8372	Bugfix: code breaks if audio is empty (#768 ) * Bugfix: code breaks if audio is empty Regression since https://github.com/SYSTRAN/faster-whisper/pull/732 PR	2024-04-02 18:18:12 +02:00
trungkienbkhn	e0c3a9ed34	Update project github link to SYSTRAN (#746 )	2024-03-27 08:31:17 +01:00
Sanchit Gandhi	a67e0e47ae	Add support for distil-large-v3 (#755 ) * add distil-large-v3 * Update README.md * use fp16 weights from Systran	2024-03-26 14:58:39 +01:00
trungkienbkhn	1eb9a8004c	Improve language detection (#732 )	2024-03-12 15:44:49 +01:00
heimoshuiyu	e50d82c18c	Merge remote-tracking branch 'upstream/master' into prompt	2024-03-10 11:53:58 +08:00
trungkienbkhn	a342b028b7	Bump version to 1.0.1 (#725 )	2024-03-01 11:32:12 +01:00
Purfview	5090cc9d0d	Fix window end heuristic for hallucination_silence_threshold (#706 ) Removes the wishful heuristic causing more issues than it's fixing. Same as https://github.com/openai/whisper/pull/2043 Example of the issue: https://github.com/openai/whisper/pull/1838#issuecomment-1960041500	2024-02-29 17:59:32 +01:00
Gabriel F	09cd57e7f3	Fix typo 'ditil' (#721 )	2024-02-29 17:08:58 +01:00
trungkienbkhn	16141e65d9	Add pad_or_trim function to handle segment before encoding (#705 )	2024-02-29 17:08:28 +01:00
heimoshuiyu	4b64ef1f70	Merge branch 'master' into prompt	2024-02-23 10:52:53 +08:00
trungkienbkhn	06d32bf0c1	Bump version to 1.0.0 (#696 )	2024-02-22 09:49:01 +01:00
Purfview	30d6043e90	Prevent infinite loop for out-of-bound timestamps in clip_timestamps (#697 ) Same as https://github.com/openai/whisper/pull/2005	2024-02-22 09:48:35 +01:00
BBC-Esq	22c75d0cc3	Update README.md (#672 ) Add Faster-Whisper-Transcriber to community integrations.	2024-02-21 10:18:11 +01:00
trungkienbkhn	092067208b	Add clip_timestamps and hallucination_silence_threshold options (#646 )	2024-02-20 17:34:54 +01:00
Jordi Mas	6ffcbdfbc2	Fix typos in README.md (#668 )	2024-02-20 17:33:17 +01:00
Purfview	52695567c9	Bumps up PyAV version to support Python 3.12.x (#679 )	2024-02-20 17:31:07 +01:00
IlianP	c6b28ed3a0	Update README.md (#685 ) I'm surprised that WhisperX hasn't made it into this list yet, as it has more stars than faster-whisper itself 🚀	2024-02-20 17:28:00 +01:00
trungkienbkhn	4ab646035f	Upgrade ctranslate2 version to support CUDA 12 (#694 )	2024-02-20 17:26:55 +01:00
heimoshuiyu	d04e685ca2	Merge branch 'master' into prompt	2024-02-19 17:31:58 +08:00
Purfview	f144e4c83d	Expands the note for distil-whisper (#659 )	2024-01-28 21:48:40 +01:00
Purfview	3aec421849	Add: More clarity of what "max_new_tokens" does (#658 ) * Add: More clarity of what "max_new_tokens" does	2024-01-28 21:40:33 +01:00
Dominik Macháček	64b9f244bd	Whisper-Streaming mention (#656 ) under community integrations	2024-01-25 18:27:27 +01:00
Purfview	00efce1696	Bugfix: Illogical "Avoid computing higher temperatures on no_speech" (#652 )	2024-01-24 11:54:43 +01:00
metame	ad3c83045b	support distil-whisper (#557 )	2024-01-24 10:17:12 +01:00
Jürgen Fleiß	72ff979a2e	Add GUI faster-whisper project README.md (#554 ) Added aTrain GUI faster-whisper transcription and diarization tool as community project. Co-authored-by: JuergenFleiss <118339672+Juergen-J-F@users.noreply.github.com>	2024-01-18 13:01:02 +01:00
makaveli	615de0d2d9	add WhisperLive to community integration (#647 )	2024-01-18 12:54:14 +01:00
heimoshuiyu	b835bdaaf1	Merge remote-tracking branch 'upstream/master' into prompt	2023-12-25 17:56:50 +08:00
Purfview	44f7e58947	Update whisper-standalone-win description in README.md (#508 ) * Update whisper-standalone-win description in README.md	2023-12-14 13:03:46 +01:00
Purfview	ebcfd6b964	Fix broken prompt_reset_on_temperature (#604 ) * Fix broken prompt_reset_on_temperature Fixing: https://github.com/SYSTRAN/faster-whisper/issues/603 Broken because `generate_with_fallback()` doesn't return final temperature. Regression since PR356 -> https://github.com/SYSTRAN/faster-whisper/pull/356	2023-12-13 13:14:39 +01:00
trungkienbkhn	19329a3611	Word timing tweaks (#616 )	2023-12-13 12:38:44 +01:00
Purfview	65094b779e	Update info on cuBLAS and cuDNN libs in README.md (#513 )	2023-11-27 12:12:47 +01:00
Clayton Yochum	9641d5f56a	Force read-mode in `av.open` (#566 ) The `av.open` functions checks input metadata to determine the mode to open with ("r" or "w"). If an input to `decode_audio` is found to be in write-mode, without this change it can't be read. Forcing read mode solves this.	2023-11-27 10:43:35 +01:00
Dang Chuan Nguyen	e1a218fab1	Bump version to 0.10.0	2023-11-24 23:19:47 +01:00
Oscaarjs	3084409633	Add V3 Support (#578 ) * Add V3 Support * update conversion example --------- Co-authored-by: oscaarjs <oscar.johansson@conversy.se>	2023-11-24 23:16:12 +01:00
Guillaume Klein	5a0541ea7d	Bump version to 0.9.0	2023-09-18 16:21:37 +02:00
Guillaume Klein	e94711bb5c	Add property WhisperModel.supported_languages (#476 ) * Expose function supported_languages * Make it a method	2023-09-14 17:42:02 +02:00
Guillaume Klein	0048844f54	Expose function available_models (#475 ) * Expose function available_models * Add test case	2023-09-14 17:17:01 +02:00
Guillaume Klein	a49097e655	Add some missing typing annotations in transcribe.py	2023-09-12 15:45:54 +02:00
Guillaume Klein	81086f6d33	Always run the encoder at the beginning of the loop (#468 )	2023-09-12 14:44:37 +02:00
Guillaume Klein	f697945691	Update tokenizers requirement to include version 0.14 (#469 )	2023-09-12 14:44:22 +02:00
Guillaume Klein	727ab81f31	Improve error message for invalid task and language parameters (#466 )	2023-09-12 10:02:23 +02:00
Guillaume Klein	0285d46f6f	Add more details about the requirements in the README (#463 )	2023-09-08 14:35:17 +02:00
Guillaume Klein	ad388cd394	Bump version to 0.8.0	2023-09-04 11:56:48 +02:00
Guillaume Klein	4a41746e55	Log a warning when the model is English-only but the language is set to something else (#454 )	2023-09-04 11:55:40 +02:00
Guillaume Klein	1e6eb967c9	Add "large" alias for "large-v2" model (#453 )	2023-09-04 11:54:42 +02:00
Guillaume Klein	f0ff12965a	Expose generation parameter no_repeat_ngram_size (#449 )	2023-09-01 17:31:30 +02:00
Guillaume Klein	5871858a5f	Force the garbage collector to run after decoding the audio with PyAV (#448 )	2023-09-01 15:25:13 +02:00
MinorJinx	e87fbf8a49	Added audio duration after VAD to TranscriptionInfo object (#445 ) * Added VAD removed audio duration to TranscriptionInfo object Along with the duration of the original audio, this commit adds the seconds of audio removed by the VAD to the returned info obj * Chaning naming for duration_after_vad Instead of the property returning the audio duration removed, it now returns the final duration after the vad. If vad_filter is False or if it doesn't remove any audio, the original duration is returned.	2023-08-31 17:19:48 +02:00
Hrishikesh Barman	7b271da035	docs: add wscribe to community integrations (#427 ) wscribe is a utility to generate transcript specifically to make it easy for further manual edits accompanied by the wscribe-editor	2023-08-17 08:50:24 +02:00
Aisu Wata	1562b02345	added repetition_penalty to TranscriptionOptions (#403 ) Co-authored-by: Aisu Wata <aisu.wata0@gmail.com>	2023-08-06 10:08:24 +02:00
Purfview	1ce16652ee	Adds DEBUG log message for prompt_reset_on_temperature (#399 ) Produce DEBUG log message if prompt_reset_on_temperature threshold is met.	2023-08-04 09:06:17 +02:00
Purfview	857be6f621	Rename clear_previous_text_on_temperature argument (#398 ) `prompt_reset_on_temperature` is more clear what it does.	2023-08-03 18:44:37 +02:00
KH	1a1eb1a027	Add clear_previous_text_on_temperature parameter (#397 ) * Add clear_previous_text_on_temperature parameter * Add a description	2023-08-03 15:40:58 +02:00
Guillaume Klein	5c17de1771	Bump version to 0.7.1	2023-07-24 11:10:12 +02:00
Guillaume Klein	0f55c436fe	Invalidate the cached encoder output when no_speech threshold is met (#376 )	2023-07-24 10:57:15 +02:00
KH	e786e26f75	Return result with best log prob when all temperature fallbacks failed (#356 ) * Resolve Inference Selection Bug * Refactor for better readability * Filter out results with compression_ratio * Refactor to avoid variable repetition * Fix incorrect index and perform minor refactoring * Remove final_temperature variable	2023-07-20 16:13:11 +02:00
KH	687db319e0	Remove duplicate code (#359 )	2023-07-18 16:03:01 +02:00
Guillaume Klein	171d90dd1f	Bump version to 0.7.0	2023-07-18 15:23:47 +02:00
Guillaume Klein	0e051a5b77	Prepend prefix tokens with the initial timestamp token (#358 )	2023-07-18 15:22:39 +02:00
Guillaume Klein	2a37390fed	Minor reformatting in code snippet	2023-07-18 15:08:53 +02:00
heimoshuiyu	9f24e2c735	Merge branch 'master' into prompt	2023-06-24 18:03:05 +08:00
heimoshuiyu	9a646b69e6	format code	2023-04-20 02:00:57 +08:00
heimoshuiyu	49af9564ab	Ignore repeated prompt	2023-04-20 01:49:10 +08:00