NVIDIA System Memory Fallback feature for Stable Diffusion - a function for stable distribution of system memory

 

NVIDIA Control Panel looks quite old, but it still continues to receive feature updates. In a recent driver release, NVIDIA introduced a new control panel option designed specifically for GeForce graphics cards. This option is labeled “CUDA Backup Policy - Sysmem” and is specifically designed to handle memory-intensive problems such as new generative AI programs.

This feature is primarily intended for stable propagation, a process that can require up to 6 GB of memory to run concurrently. However, since popular GPUs still have a memory maximum of 6 to 8 GB , GPU memory may still be shared with other workloads, preventing programs like Stable Diffusion from being fully utilized. To address this issue, NVIDIA has introduced a new feature that prevents applications from running out of GPU memory by efficiently switching to shared system memory.

From a technical point of view, users can flexibly use this feature for any application, including games. However, this may not be the best choice since system memory is often noticeably slower than GPU memory, not to mention the necessary data transfer between them.

The main purpose of this feature is to prevent possible crashes on video cards equipped with 6 GB memory. In the latest versions of drivers, NVIDIA has prudently included an option to disable this backup mechanism. This has the potential to improve performance stability, but again runs the risk of running out of GPU memory.

Enabling system memory backup:

• Open the NVIDIA Control Panel.
• Under 3D Settings, click Manage 3D Settings.
• Go to the Program Settings tab.
• Select the Python executable for stable distribution from the drop-down list.
• Click on CUDA Backup Policy – ​​Sysmem and select the default driver, choose not to use Sysmem backup or choose to use Sysmem backup.
• Click Apply to confirm.
• Restart Stable Diffusion if it is already open.