NVIDIA’s guide shows how to run GLM-5.3 with 753B parameters in NVFP4 on two DGX Stations using vLLM and tensor parallelism. The approximately 464 GB checkpoint must be stored separately on the two machines. With 16 concurrent requests, peak total throughput is 607.8 tok/s, while single-user performance is 107.4 tok/s.
Source:explainx.ai · Read the original
Leave a Reply