NVIDIA Guide: Run 753B GLM-5.3 with vLLM on Two DGX Stations

Written by

in

NVIDIA’s guide shows how to run GLM-5.3 with 753B parameters in NVFP4 on two DGX Stations using vLLM and tensor parallelism. The approximately 464 GB checkpoint must be stored separately on the two machines. With 16 concurrent requests, peak total throughput is 607.8 tok/s, while single-user performance is 107.4 tok/s.

Source:explainx.ai · Read the original

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *