You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CUDA 12 support (libcublas.so.11 is not found)Β #1250
The documentation is no longer correct since the release of CUDA 12. The released binaries are compiled against CUDA 11. It would be more correct to say "Install CUDA 11.x to use the GPU".
Do you want to help fixing the documentation?
Regarding CUDA 12 support, it currently requires to recompile the binaries from the sources. There are no immediate plans to release binaries compiled with CUDA 12.
Speaking of building. I use ctranslate2 in my program that other people can download and it's installed with "pip." With "pip" installation method, which backends are installed, if any? I'm assuming that it's the defaults listed at the link you gave? For example, openblas would not be installed...ruy would not be installed...If so, I've made a huge mistake in advising people.
Also, is there any licensing violation if I build from source with all backends enabled and distribute to others...that way they don't have to "build" on their own to use my program?
Basically, are these the defaults when pip installed or is everything enabled...
Thanks, I was almost going to switch from using ctranslate2/faster-whisper in my programs because it wasn't supporting CUDA 12+ out of the box (i.e. without compiling from source)...As long as it's being worked on I'd love to keep using it.
Is there a timeline for when 12+ will be supported via the normal "pip install" method for Windows and other operating systems?
Also, is there any possible way to allow "flavors" of "pip install" such as with "ruy" and/or "openblas?" Nobody responded whether those options are enabled with a normal "pip install" compared to compiling from source. Thanks for all the great work!
@vince62s , is there any licensing issue if I were to take the time to learn how to compile from source and then distribute the compiled ctranslate2 version (i.e. with all mkl, accelerate, etc. enabled) and then simply distribute that with my program? The only reason I'd want to do that is to simplify the installation for people...
There is no licensing issue at all. I understand your point. Of course we will update to CUDA 12, it is not so difficult just requires some time and unfortunately there are very few contributors to this repo.
Embarrisingly, I don't even know what a docker file is. Isn't it something that allows a user to run a program self contained, like a .exe, instead of having to install all the dependencies?
If anyone else is encountering this on Colab (or other Ubuntu jammy systems), one workaround for now is to install the libcublas11 package. This can be accomplished by including the following at the start of a Colab notebook:
!apt install libcublas11
Reacted by Taka, Brent, Gabriele Lazzarelli, Finn Woelm, Yasmin Moslem and oimosensei
Do you have any more info on the tests that were run, scripts, logs even? I'd be curious to know because my current plan was to recommend users install CUDA 12+ with my program faster-whisper...
Nothing more, just that there were 3x tests for each quantization then the results were averaged.
And the latest CUDA 11 libs vs the latest CUDA 12 libs.
my current plan was to recommend users install CUDA 12+ with my program faster-whisper...
I'll keep my repo under CUDA 11.
Btw, you can have CUDA 12 installed and have CUDA 11 libs separate just for Faster-Whisper.
I know there was a list somewhere, where CUDA 12 support passed all checks but just hadn't been uploaded to pypi I believe? Can't seem to find it, but was generally wondering the status if you know? thanks @vince62s
Activity
guillaumekln commentedon May 30, 2023
The documentation is no longer correct since the release of CUDA 12. The released binaries are compiled against CUDA 11. It would be more correct to say "Install CUDA 11.x to use the GPU".
Do you want to help fixing the documentation?
Regarding CUDA 12 support, it currently requires to recompile the binaries from the sources. There are no immediate plans to release binaries compiled with CUDA 12.
wizd commentedon Nov 17, 2023
Any plan to support CUDA 12.x?
BBC-Esq commentedon Nov 17, 2023
I was wondering the same thing actually!
StephennFernandes commentedon Nov 20, 2023
@guillaumekln is there any update on cuda 12 ?
vince62s commentedon Nov 21, 2023
This requires some work in the github ci workflow.
You know you can build from source quite easily.
https://opennmt.net/CTranslate2/installation.html#download-the-source-code
wizd commentedon Nov 22, 2023
build manually possible. bellow is a bash history to reference:
BBC-Esq commentedon Nov 22, 2023
Speaking of building. I use ctranslate2 in my program that other people can download and it's installed with "pip." With "pip" installation method, which backends are installed, if any? I'm assuming that it's the defaults listed at the link you gave? For example, openblas would not be installed...ruy would not be installed...If so, I've made a huge mistake in advising people.
Also, is there any licensing violation if I build from source with all backends enabled and distribute to others...that way they don't have to "build" on their own to use my program?
Basically, are these the defaults when pip installed or is everything enabled...

Qubitium commentedon Dec 11, 2023
Here is updated ctranslate2 install for Ubuntu 22.04 with cuda 12.x. You should pip uninstall the existing ctranslate2 before proceeding.
intel mkl
ubuntu 22.04
clone
build
python binding
final step:
pip install the file generated in the "dist" folder. The name of the file is different due to python version and arch.
DeserterW commentedon Dec 14, 2023
Download
install download cublas
u can install cublas where u want and note it.
BBC-Esq commentedon Dec 14, 2023
Thanks, I was almost going to switch from using ctranslate2/faster-whisper in my programs because it wasn't supporting CUDA 12+ out of the box (i.e. without compiling from source)...As long as it's being worked on I'd love to keep using it.
Is there a timeline for when 12+ will be supported via the normal "pip install" method for Windows and other operating systems?
Also, is there any possible way to allow "flavors" of "pip install" such as with "ruy" and/or "openblas?" Nobody responded whether those options are enabled with a normal "pip install" compared to compiling from source. Thanks for all the great work!

BBC-Esq commentedon Dec 14, 2023
@vince62s , is there any licensing issue if I were to take the time to learn how to compile from source and then distribute the compiled ctranslate2 version (i.e. with all mkl, accelerate, etc. enabled) and then simply distribute that with my program? The only reason I'd want to do that is to simplify the installation for people...
vince62s commentedon Dec 14, 2023
There is no licensing issue at all. I understand your point. Of course we will update to CUDA 12, it is not so difficult just requires some time and unfortunately there are very few contributors to this repo.
BBC-Esq commentedon Dec 14, 2023
Yeah, I hear ya...I wish there were more contributors or, alternatively, that I actually knew C++ myself... :-)
taka-wang commentedon Dec 15, 2023
Hi @BBC-Esq
I thought you might find this Dockerfile interesting. I hope it comes in handy.
BBC-Esq commentedon Dec 15, 2023
Embarrisingly, I don't even know what a docker file is. Isn't it something that allows a user to run a program self contained, like a .exe, instead of having to install all the dependencies?
BBC-Esq commentedon Dec 15, 2023
Just researched docker so no need to educate me. Thanks!
jncraton commentedon Dec 16, 2023
If anyone else is encountering this on Colab (or other Ubuntu jammy systems), one workaround for now is to install the libcublas11 package. This can be accomplished by including the following at the start of a Colab notebook:
Purfview commentedon Dec 22, 2023
The tests shows that "Faster-Whisper with CUDA v12" has -10% drop in performance.
RTX 3050 GPU:
BBC-Esq commentedon Dec 22, 2023
Do you have any more info on the tests that were run, scripts, logs even? I'd be curious to know because my current plan was to recommend users install CUDA 12+ with my program faster-whisper...
Purfview commentedon Dec 22, 2023
Nothing more, just that there were 3x tests for each quantization then the results were averaged.
And the latest CUDA 11 libs vs the latest CUDA 12 libs.
I'll keep my repo under CUDA 11.
Btw, you can have CUDA 12 installed and have CUDA 11 libs separate just for Faster-Whisper.
BBC-Esq commentedon Jan 11, 2024
I know there was a list somewhere, where CUDA 12 support passed all checks but just hadn't been uploaded to pypi I believe? Can't seem to find it, but was generally wondering the status if you know? thanks @vince62s
SnowFox4004 commentedon Jan 29, 2024
That works for me, thanks so much! @Qubitium
Yangqing commentedon Feb 9, 2024
If you have to use cuda 12 and want a quick hacky fix, one way to try is:
ln -s /usr/local/cuda/lib64/libcublas.so.12 /usr/local/cuda/lib64/libcublas.so.11
since ctranslate2 uses standard cublas apis, this usually won't produce an error. You can keep all other configs as-is.
hustshawn commentedon Feb 14, 2024
Thanks @Yangqing. It works for me.
minhthuc2502 commentedon Feb 15, 2024
The major release 4.0.0 supports CUDA 12 now. Feel free to test it if there is any issue about this. Thanks!
intexcor commentedon Oct 28, 2025
Need cuda 13 support