heimoshuiyu/faster-whisper
1
0
Fork 0
Code Issues Pull Requests 1 Packages Projects Releases Wiki Activity

Add some performance numbers in the README

Browse Source
This commit is contained in:
Guillaume Klein
2023-02-14 16:58:05 +01:00
parent cbbe633082
commit 17a6d83d0e
1 changed files with 24 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
README.md
View File

@@ -2,7 +2,30 @@
This repository demonstrates how to implement the Whisper transcription using [CTranslate2](https://github.com/OpenNMT/CTranslate2/), which is a fast inference engine for Transformer models.
This implementation is about 4 times faster than [openai/whisper](https://github.com/openai/whisper) for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.
This implementation is up to 4 times faster than [openai/whisper](https://github.com/openai/whisper) for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.
## Benchmark
For reference, here's the time and memory usage that are required to transcribe **13 minutes** of audio.
### Medium model on GPU
| Implementation | Precision | Beam size | Time | Max. GPU memory | Max. CPU memory |
| --- | --- | --- | --- | --- | --- |
| openai/whisper | fp16 | 5 | 2m56s | 6285MB | 5598MB |
| faster-whisper | fp16 | 5 | 0m39s | 3347MB | 1721MB |
*Executed with CUDA 11.7.1 on a NVIDIA Tesla V100S.*
### Small model on CPU
| Implementation | Precision | Beam size | Time | Max. memory |
| --- | --- | --- | --- | --- |
| openai/whisper | fp32 | 5 | 10m39s | 2850MB |
| faster-whisper | fp32 | 5 | 3m24s | 2182MB |
| faster-whisper | int8 | 5 | 2m37s | 1466MB |
*Executed with 8 threads on a Intel(R) Xeon(R) Gold 6226R.*
## Installation