I use a clean installation of Ubuntu 23.04 Lunar Lobster and Nvidia driver 525. If you already have the driver installed, here are the steps to improve Automatic111 Stable Diffusion performance to 40-44 it/s
Ubuntu 23.04 default Python version is 3.11 version. In this case, I will using Anaconda to provide Python 3.10. Download Anaconda and install
chmod a+x /Anaconda3-2023.03-1-Linux-x86_64.sh
./Anaconda3-2023.03-1-Linux-x86_64.sh When using pandas drop duplicates, we may encountered rows that still have duplicating by checking via
df.column_name.value_counts()Not sure why Pandas drop duplicates performance showing inconsistent result. However, to remove duplicate row, produce 100% unique based on index or key column, you can use this
df_unique = df_unique.drop(df_unique[df_unique["key_column_name"].duplicated()].index)
df_unique.temp_id.value_counts()LightGBM can work faster in GPU. In PyCaret, I’m passing parameter use_gpu=True in TSForecastingExperiment() and got errors:
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1To enable this, we need to uninstall the current LightGBM and re-install the LightGBM with GPU. For Linux Ubuntu, its better to install pre-requisite packages
sudo apt install cmake build-essential libboost-all-devMake sure you already have Nvidia Toolkit installed
sudo apt install nvidia-cuda-toolkitThe first option, is installation inside conda environment
pip uninstall lightgbm -y
conda install -c conda-forge gcc=12.1.0pip install lightgbm --config-settings=cmake.define.USE_GPU=ON --config-settings=cmake.define.OpenCL_INCLUDE_DIR="/usr/local/cuda/include/" --config-settings=cmake.define.OpenCL_LIBRARY="/usr/local/cuda/lib64/libOpenCL.so"Second options, installation without environment
# Get LightGBM source.
git clone --recursive https://github.com/Microsoft/LightGBM.git
cd LightGBM/python-package/
# cmake specifying locations of OpenCL files.
sudo cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda-8.0/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda-8.0/include/ ..
# Compile.
sudo make
# Install for Python, using what we just compiled.
python setup.py install --precompileMany feedback about performance NVIDIA RTX after installing Stable Diffusion Automatic1111. I will explained a simple way to install and fix the RTX 4090 performance within 5 minutes
First, make sure you have Python 3.10 in your Windows. You can use Anaconda or native Python installation.
https://github.com/AUTOMATIC1111/stable-diffusion-webui.git2. Install Stable Diffusion with xformers
This part is tricky. By default, it will install Torch 2.1.0, however the latest xformers will required to use torch 2.0. Which later you will encountered the problems like :
AssertionError: Torch is not able to use GPU; add --skip-torch-cuda-test to COMMANDLINE_ARGS variable to disable this check
The solution for installation both xformers and torch inside stable difussion is to pass the arguments in installation
./webui.bat --xformersTensorflow running use GPU in Windows sometimes is difficult to do, where many articles not pointing exactly which Tensorflow version, NVIDIA drivers and other requirement needed to achieve it in Windows 11. Especially when you have NVIDIA RTX Graphic card like 4090 or 3090 or similar version.
I will help to explain on how to install it properly and make it run. Here are the steps to do:
1.Installing the Latest Anaconda.
Go to Anaconda website and download its community distribution. You can try to activate in your terminal windows with “conda activate” and you will enter your base.
If you are using powershelll, you can try “conda init powershell” to load the environment by default
2.Installing Microsoft Visual Studio 2022 (not to confuse with VSCode editor).
Choose the community version at https://visualstudio.microsoft.com/downloads/ and install Desktop Environment with C++
NumPy uses libraries like BLAS, LAPACK, BLIS, or MKL to execute vector, matrix, and linear algebra operations. It’s acknowledged that Intel with MKL (Math Kernel Library) is quite more mature in this math operation than other libraries due to resources and experiences.
If you want to leverage Intel OneAPI MKL as backend for your Numpy, especially on Intel chip (or AMD if you want to try), here are a quick step for installation in Ubuntu (I use the latest ubuntu 23.04 Lunar Lobster).
First, download the required softwares
sudo apt install build-essential python3-pip python3 python3-dev libomp-devA context, libomp-dev will help you to avoid the error “Solve bmkl_intel_thread.so.2: undefined symbol: omp_get_num_procs” when importing the numpy libraries in the python interpreter.
Second, we will install the Intel Base-kit: https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html?operatingsystem=linux&distributions=aptpackagemanager.
Or follow the commands below
The default installation of Stable Diffusion Automatic111 using the old version of Torch 1.13.1, Torchvision 0.14, and related Xformers, the webui.sh will receive “No module ‘xformers’. Proceeding without it.”.
Pytorch 2.0 and Xformers are offering a big upgrade. Here are quick step on how to upgrade the Automatic111 web UI and load with xformers.
If you are using Anaconda, go open its terminal then go to stable-diffusion-webui/venv/bin and source activate to enter “venv” environment with python 3.10 as default.
Then, we can start to upgrade the torchvision, torch and xformers with this command
pip install torchvision --upgrade
pip install torch==2.0
pip install -U xformersIf receive an error like this, you can upgrade the torchvision version.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
torchvision 0.14.1+cu117 requires torch==1.13.1, but you have torch 2.0.0 which is incompatible.
Successfully installed torch-2.0.0After everything is done completely, you can fire-up your webui.sh along with xformers module loaded with command : “bash webui.sh –xformers”
I move WordPress from Ubuntu Server 21.04 to 23.04 with several easy steps. Here are the quick guideline on how to migrate WordPress safely without breaking
First, Backup all necessary data
Open your terminal and start with your home path ~/
sudo mysqldump -u root -p yourdb > db.sql
Second, Compress etc, home and any folder needed
sudo zip -r /etc etc.zip
sudo zip -r /home/ubuntuTransfer the file to the new server. If you need to use pem-key and using either windows or linux, use SCP to transfer pem-key from local computer to old-server. This is needed so you can transfer file from old to new server
scp -i .\your-key.pem -r .\your-key.pem ubuntu@your-ec2-server-ip:/home/ubuntuNow your key pem is exists in home path in old server. Its time to transfer the whole zip files backup to the new server home path
scp -i your-key.pem *.zip ubuntu@your-new-server-EC2:/home/ubuntu/Once its done, you just need to extract the folder and replace whatever needed configuration and databases to the new server.
If you encountered by permission denied (13) on NGINX,
/php/php8.0-fpm.sock:", host: "www. com"
2023/04/23 16:31:33 [crit] 17224#17224: *28 stat() "/home/ubuntu/" failed (13: Permission denied), client: 182.253.152.107, server: .com, request: "GET / HTTP/1.1", host: "www. .com"
2023/04/23 16:31:33 [crit] 17224#17224: *28 stat() "/home/ubuntu/" failed (13: Permission denied), client: 182.253.152.107, server: .com, request: "GET / HTTP/1.1", host: "www..com"
2023/04/23 16:31:33 [error] 17224#17224: *28 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 182.25352.107, server: .com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/run/php/php8.0-fpm.sock:", host: "www..com"the quickfix solution
sudo gpasswd -a www-data ubuntu
chmod o+x YOUR-WEBSITE-FOLDER
sudo service php8.1-fpm restart
sudo service nginx restartDone! Now you can smoothly migrate WordPress from the old ubuntu server to the new ubuntu server without problem. The last step is to change the IP-address in NS. If you are using AWS Route 53, just change the IP address in the record and you are ready to go!
If you have this error when opening CMD, Powershell or terminal
WindowsPowerShell\profile.ps1 cannot be loaded because running scripts is disabled on
this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:3
+ . 'C:\Users\dev\Documents\WindowsPowerShell\profile.ps1'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : SecurityError: (:) [], PSSecurityException
+ FullyQualifiedErrorId : UnauthorizedAccessThe quick fix is open “Powershell” with administrator access and execute this command
Set-ExecutionPolicy UnrestrictedWhen using Anaconda conda in terminal Windows, we often found the Python and default packages is already installed. To create conda new environment empty packages in Windows cmd / powershell without any packages, follow this steps:
And we done. Next, you can open terminal:
conda init
conda create -n test
conda activate test
conda listVoilla! Now you have empty environment with Conda 100%!