nvidia-smi
Usage
List GPUs:
$ nvidia-smi -L
GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
MIG
Enable MIG Mode:
$ sudo nvidia-smi -i 0 -mig 1
Enabled MIG Mode for GPU 00000000:16:00.0
All done.
The UUID can also be used for parameter -i.
List available MIG profile for a GPU:
$ sudo nvidia-smi mig -i 0 -lgip
+-----------------------------------------------------------------------------+
| GPU instance profiles: |
| GPU Name ID Instances Memory P2P SM DEC ENC |
| Free/Total GiB CE JPEG OFA |
|=============================================================================|
| 0 MIG 1g.10gb 19 7/7 9.75 No 16 1 0 |
| 1 1 0 |
+-----------------------------------------------------------------------------+
| 0 MIG 1g.10gb+me 20 1/1 9.75 No 16 1 0 |
| 1 1 1 |
+-----------------------------------------------------------------------------+
| 0 MIG 1g.20gb 15 4/4 19.62 No 26 1 0 |
| 1 1 0 |
+-----------------------------------------------------------------------------+
| 0 MIG 2g.20gb 14 3/3 19.62 No 32 2 0 |
| 2 2 0 |
+-----------------------------------------------------------------------------+
| 0 MIG 3g.40gb 9 2/2 39.50 No 60 3 0 |
| 3 3 0 |
+-----------------------------------------------------------------------------+
| 0 MIG 4g.40gb 5 1/1 39.50 No 64 4 0 |
| 4 4 0 |
+-----------------------------------------------------------------------------+
| 0 MIG 7g.80gb 0 1/1 79.25 No 132 7 0 |
| 8 7 1 |
+-----------------------------------------------------------------------------+
Form the output we can see 7 instances with profile ID 19 can be created.
List the possible placements available:
$ nvidia-smi mig -i 0 -lgipp
GPU 0 Profile ID 19 Placements: {0,1,2,3,4,5,6}:1
GPU 0 Profile ID 20 Placements: {0,1,2,3,4,5,6}:1
GPU 0 Profile ID 15 Placements: {0,2,4,6}:2
GPU 0 Profile ID 14 Placements: {0,2,4}:2
GPU 0 Profile ID 9 Placements: {0,4}:4
GPU 0 Profile ID 5 Placement : {0}:4
GPU 0 Profile ID 0 Placement : {0}:8
Create MIG Instances:
$ sudo nvidia-smi mig -i 0 -cgi 19,19,19,19,19,19,19 -C
Successfully created GPU instance ID 13 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 13 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 11 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 11 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 12 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 12 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 7 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 7 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 8 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 8 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 9 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 9 using profile MIG 1g.10gb (ID 0)
Successfully created GPU instance ID 10 on GPU 0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID 0 on GPU 0 GPU instance ID 10 using profile MIG 1g.10gb (ID 0)
-C means create compute instances for the GPU instances.
List created GPU instances and compute instances:
$ sudo nvidia-smi mig -i 0 -lgi
+-------------------------------------------------------+
| GPU instances: |
| GPU Name Profile Instance Placement |
| ID ID Start:Size |
|=======================================================|
| 0 MIG 1g.10gb 19 7 0:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 8 1:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 9 2:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 10 3:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 11 4:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 12 5:1 |
+-------------------------------------------------------+
| 0 MIG 1g.10gb 19 13 6:1 |
+-------------------------------------------------------+
$ sudo nvidia-smi mig -i 0 -lci
+--------------------------------------------------------------------+
| Compute instances: |
| GPU GPU Name Profile Instance Placement |
| Instance ID ID Start:Size |
| ID |
|====================================================================|
| 0 7 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 8 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 9 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 10 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 11 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 12 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
| 0 13 MIG 1g.10gb 0 0 0:1 |
+--------------------------------------------------------------------+
Delete all compute instances of all GPU instances of a GPU:
$ sudo nvidia-smi mig -i 0 -dci
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 7
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 8
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 9
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 10
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 11
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 12
Successfully destroyed compute instance ID 0 from GPU 0 GPU instance ID 13
Delete all GPU instances of a GPU:
$ sudo nvidia-smi mig -i 0 -dgi
Successfully destroyed GPU instance ID 7 from GPU 0
Successfully destroyed GPU instance ID 8 from GPU 0
Successfully destroyed GPU instance ID 9 from GPU 0
Successfully destroyed GPU instance ID 10 from GPU 0
Successfully destroyed GPU instance ID 11 from GPU 0
Successfully destroyed GPU instance ID 12 from GPU 0
Successfully destroyed GPU instance ID 13 from GPU 0
Disable MIG:
$ sudo nvidia-smi -i 0 -mig 0
Disabled MIG Mode for GPU 00000000:16:00.0
All done.