Alauda Build of HAMi
  • English

    • Configure Overcommitment Ratio

    TOC

    Understand overcommitment ratioKey parametersConfigure the ratioNotes

    Understand overcommitment ratio

    HAMi allows you to set a resource overcommitment ratio for supported shared resources. For the NVIDIA backend, the commonly used parameters are vGPU core and memory scaling. The purpose of overcommitment is to improve device utilization. It does not increase the physical capacity available to a single workload.

    For example, if the vGPU cores overcommitment ratio is set to 2, the logically total allocatable cores of a single GPU card is 200%. However, a single Pod cannot request more than 100% of the GPU cores, as this would exceed the actual available resources. You can assign 100% of the vGPU cores to each of two Pods, which can result in the following scenarios:

    • If Pod A is busy while Pod B is idle, Pod A can utilize up to 100% of the vGPU cores.

    • If Pod A is idle while Pod B is busy, Pod B can utilize up to 100% of the vGPU cores.

    • If both Pod A and Pod B are busy simultaneously, vGPU cores are scheduled in time slice manner. Each Pod alternates access to the GPU, resulting in an effective utilization of approximately 50% per Pod.

    Key parameters

    • NVIDIA Device Core Scaling: the overcommitment ratio for NVIDIA device compute resources.
    • NVIDIA Device Memory Scaling: the overcommitment ratio for NVIDIA device memory resources.

    Configure the ratio

    Go to Administrator -> Marketplace -> Cluster Plugins, switch to the target cluster, and update Alauda Build of HAMi.

    Modify the plugin form parameters according to the backend and product version you are using. For NVIDIA backend examples, configure NVIDIA Device Core Scaling and NVIDIA Device Memory Scaling.

    Notes

    1. When the overcommitment ratio of vGPU cores is set, multiple applications may collectively request more than 100% of the vGPU cores. If all applications are running at full load, they compete for total 100% compute capacity of the GPU. As a result, the effective performance of each application will be lower compared to using a dedicated GPU. If one application becomes idle, the remaining active applications can utilize the freed compute capacity.

    2. When the overcommitment ratio of vGPU memory is set, multiple applications may collectively request more than 100% of the vGPU memory. If all requested memory is fully allocated, some applications may encounter CUDA out of memory errors. Use memory overcommitment carefully because it can lead to application failures.