When installing TFX, I received error pip install tfx raises ResolutionTooDeep. During installation, its going over multiple different version of packages.
To solve this problem, I created requirements.txt with option to install necessary packages or all-packages that produced using pip freeze.
There are three options: TFX 1.10, 1.13 and the latest TFX 1.14.0
I got this error when running Tensorflow successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node,. To solve it
for a in /sys/bus/pci/devices/*; do echo 0 | sudo tee -a $a/numa_node; done
The problem when adjusting my dual RTX 4090 in Ubuntu Server 23.10 is, when running nvidia-settings its trigger error
ERROR: The control display is undefined; please run `nvidia-settings --help`
for usage information.
I’ve been search on internet and not found a better solution, then I decided try to install X and attached, while running nvidia-setting. The result its works. Here are the step by step on how to enable GPU fan speed in Ubuntu Server 23.10
Install XServer
sudo apt install -y xorg xinit
2. Give permission to run X for user
Edit the file and change console into anybody to give permission to running X and add permission. This also needed for Ubuntu 23.10 Desktop as well
sudo vim /etc/X11/Xwrapper.config
# If you have edited this file but would like it to be automatically updated
# again, run the following command as root:
# dpkg-reconfigure xserver-xorg-legacy
allowed_users=anybody
needs_root_rights=yes
Reboot!
If you don’t change this, it will trigger error.
/usr/lib/xorg/Xorg.wrap: Only console users are allowed to run the X server
If you got this error when running your notebook with warning
contains a task of very large size. The maximum recommended task size is 1000 KiB
This mean PySpark warning you to increase the partition or parallelism (and might memory as well).
Example code to configure it, where you can adjust based on your workstation memory. In my case, is 192GB is my max memory
import os
os.environ['PYSPARK_SUBMIT_ARGS'] = '--driver-memory 192g --executor-memory 16g pyspark-shell'
# add this one in your spark configuration
spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "true")
To solve Driver or CUDA 11 installation error in Ubuntu 23.10, the answer is to ensure its using the compatible GCC version. By default installation, it will using GCC 13 which is not working when compiling CUDA or NVIDIA Drivers (required GCC 10). Installing CUDA 11 is important to run Tensorflow that haven’t fully adapted with CUDA 12.
Failed to verify gcc version. See log at /var/log/cuda-installer.log for details.
First step to fixing this problem is to uninstall any nvidia and cuda installation made previously
Next, choose the GCC 10 as default by running this command
sudo update-alternatives --config gcc
Now, you all set! You can start to do installation of CUDA 11 in Ubuntu 23.10. Make sure to un-check the driver installation part (where we will install it later)
sudo ./cuda_11.8.0_520.61.05_linux.run
Next, I’m using Ubuntu NVIDIA default installation. So, I revert back the GCC to version 13, using the same command
sudo update-alternatives --config gcc
sudo apt install nvidia-driver-525
You can repeat the process like CUDNN, TensorRT and others installation following my previous article here