When running –quantize llm.int8 in adapter for LitLLama, I got this error
ImportError: cannot import name 'Linear8bitLt' from 'lit_llama.quantization'
First step, we need to make sure if bitsandbytes
is running well by
python -m bitsandbytes
And I received
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
...
packages/bitsandbytes/functional.py", line 12, in <module>
from scipy.stats import norm
ModuleNotFoundError: No module named 'scipy'
Now, I know the problem is scipy is not installed. To solve this is installing scipy
pip install scipy
And I re-run again for bitsandbytes