Installation for latest VLLM or SGLANG pip install vllm -U trigger error like below
ImportError: python3.11/site-packages/torch/lib/../../nvidia/cusparse/lib/libcusparse.so.12: undefined symbol: __nvJitLinkComplete_12_4, version libnvJitLink.so.12To solve this problem is uninstall the torch packages and downgrade to 2.60 with cu121. I’m using nvcc version 12.2 and NVIDIA driver 12.7
pip uninstall torch torchvision torchaudio -y
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121