Categories
LLM

Solve Unsloth AttributeError: ‘NoneType’ object has no attribute ‘attn_bias’

I got this error when running SFT Training using Unsloth

trainer.py:3731, in Trainer.compute_loss(self, model, inputs, return_outputs, num_items_in_batch)
   3729         loss_kwargs["num_items_in_batch"] = num_items_in_batch
   3730     inputs = {**inputs, **loss_kwargs}
-> 3731 outputs = model(**inputs)
   3732 # Save past state if it exists
...
    198     causal_mask = xformers.attn_bias.BlockDiagonalCausalMask\
    199         .from_seqlens([q_len]*bsz)\
    200         .make_local_attention(window_size = sliding_window)

AttributeError: 'NoneType' object has no attribute 'attn_bias'

The solution is

pip install pip3-autoremove
pip-autoremove torch torchvision torchaudio -y 
pip install torch torchvision torchaudio xformers --index-url https://download.pytorch.org/whl/cu121
pip unsloth

Leave a Reply

Your email address will not be published. Required fields are marked *