February 2025 - Yodi Aditya

Install P2P Dual RTX 4090 Ubuntu 24.04

A good news, we can enable P2P for Dual RTX 4090 or more. When running simpleP2P script, we will got this results. Don’t worry, next is how to enable it easily!

[./simpleP2P] - Starting...
Checking for multiple GPUs...
CUDA-capable device count: 2

Checking GPU(s) for support of peer to peer memory access...
> Peer access from NVIDIA GeForce RTX 4090 (GPU0) -> NVIDIA GeForce RTX 4090 (GPU1) : No
> Peer access from NVIDIA GeForce RTX 4090 (GPU1) -> NVIDIA GeForce RTX 4090 (GPU0) : No
Two or more GPUs with Peer-to-Peer access capability are required for ./simpleP2P.
Peer to Peer access is not available amongst GPUs in the system, waiving test.

First thing, make sure to enable Resize BAR and disable IOMMU in BIOS (I’m using ASUS WRX80SAGE).

Next, uninstall all nvidia drivers (*yes, thats right!)

# Uninstall all nvidia
sudo apt-get --purge remove "*nvidia*"
sudo apt-get --purge remove "*cuda*" "*cudnn*" "*cublas*" "*cufft*" "*cufile*" "*curand*" "*cusolver*" "*cusparse*" "*gds-tools*" "*npp*" "*nvjpeg*" "nsight*" "*nvvm*" "*libnccl*"

# disable iommu
ll /sys/class/iommu/

# install dependencies
sudo apt install git cmake

# reboot
sudo reboot

Solve Unsloth AttributeError: ‘NoneType’ object has no attribute ‘attn_bias’

trainer.py:3731, in Trainer.compute_loss(self, model, inputs, return_outputs, num_items_in_batch) 3729 loss_kwargs["num_items_in_batch"] = num_items_in_batch 3730 inputs = {**inputs, **loss_kwargs} -> 3731 outputs = model(**inputs) 3732 # Save past state if it exists ... 198 causal_mask = xformers.attn_bias.BlockDiagonalCausalMask\ 199 .from_seqlens([q_len]*bsz)\ 200 .make_local_attention(window_size = sliding_window) AttributeError: 'NoneType' object has no attribute 'attn_bias'