Installing TensorRT is tricky and shall be matched with the version of tensorflow. This is how you do it. First check the required version
python3 -c "import tensorflow.compiler as tf_cc; \
print(tf_cc.tf2tensorrt._pywrap_py_utils.get_linked_tensorrt_version())"And then the result
2025-01-05 16:30:38.552990: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-01-05 16:30:38.576738: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-01-05 16:30:38.576767: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-01-05 16:30:38.577423: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-01-05 16:30:38.581769: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-01-05 16:30:39.023446: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
(8, 6, 1)This showing result for my tensorflow version, the tensor rt version is “(8.6.1)”. Next, go download tensorrt version eg: https://developer.nvidia.com/nvidia-tensorrt-8x-download
Extract it and go to “tensorrt/python” folder. Since i’m using mamba, I will go to my environment and install the tensor wheel packages based on my python version
(tf) ➜ python ls
tensorrt-8.6.1-cp310-none-linux_x86_64.whl tensorrt_dispatch-8.6.1-cp37-none-linux_x86_64.whl
tensorrt-8.6.1-cp311-none-linux_x86_64.whl tensorrt_dispatch-8.6.1-cp38-none-linux_x86_64.whl
tensorrt-8.6.1-cp36-none-linux_x86_64.whl tensorrt_dispatch-8.6.1-cp39-none-linux_x86_64.whl
tensorrt-8.6.1-cp37-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp310-none-linux_x86_64.whl
tensorrt-8.6.1-cp38-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp311-none-linux_x86_64.whl
tensorrt-8.6.1-cp39-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp36-none-linux_x86_64.whl
tensorrt_dispatch-8.6.1-cp310-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp37-none-linux_x86_64.whl
tensorrt_dispatch-8.6.1-cp311-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp38-none-linux_x86_64.whl
tensorrt_dispatch-8.6.1-cp36-none-linux_x86_64.whl tensorrt_lean-8.6.1-cp39-none-linux_x86_64.whl
And I installed
pip install tensorrt-8.6.1-cp311-none-linux_x86_64.whl Done, now your tensor rt warning is gone!
If you are looking for Ubuntu 24.04 Nvidia deb packages
https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/
For CUDNN, NUMA and others, you can follow this