I found issue where stutter and nvidia-smi took long time to print, which is unusual. After digging more, apparently this is caused by service of persistence mode.
To solve this, you can disable by comment /lib/systemd/system/nvidia-persistenced.service
sudo vim /lib/systemd/system/nvidia-persistenced.servicesystemctl edit nvidia-persistenced.serviceTo run persistence on your session
sudo nvidia-smi -pm 1