At a glance
| Part | What it needs | Details |
|---|---|---|
| GPU server | x86_64 with NVIDIA GPUs that the engine supports (compute capability 7.5 and newer). Linux. NVIDIA driver R580 or newer (CUDA 13.0 images) or R575 or newer (CUDA 12.9). Docker 25 or newer with the NVIDIA Container Toolkit, or Kubernetes. /dev/shm for a ring of 6–10 GiB. | GPU server |
| ADP card | A PCIe Gen5 x16 slot. A low-profile HHHL card with tall and short brackets. Less than 25 W typical, 45 W at most from the slot. 10–52 °C at 250 LFM airflow. | ADP card hardware |
| Cache SSDs | NVMe SSDs in the server of the card, on the NUMA node of the card, only for the card. One RAID 0 array. Up to 128 TB raw for each card, about 90 % usable. | SSDs |
| Card software | The ADP software stack for the kernel of the host: driver, firmware, storage service, CLI, and for a remote card the NVMe-oF target. The prerequisite packages. Key eviction on. | ADP card software |
| Storage server (remote card) | x86_64. Dedicated cores and hugepages for the NVMe-oF target. One RDMA NIC for each card, on its NUMA node. A separate boot disk. | Storage server |
| Network (remote card) | NVMe-oF over RDMA (RoCE v2 or InfiniBand), 100 Gb/s or more for each card. RoCE: MTU 9000 end to end. InfiniBand: a subnet manager and an IPoIB address for each port. The target ports open only to the GPU servers. | Network |
| Card cluster | Three or more hosts (storage servers or witness hosts) with Docker and the cluster service. Hugepages and RDMA devices for the gateways on the GPU servers. | Card cluster |
| Software from Awide Labs | The engine image with the integrations (or the client kit) and the gateway image of the same release, from nexus.awide.io. | Installation |
Local or remote card
| Local card | Remote card | |
|---|---|---|
| Card and SSDs | in the GPU server | in a storage server |
| ADP software stack | on the GPU server | on the storage server |
| On the GPU server | the stack, the gateway, the engine | the NVMe-oF initiator, the gateway, the engine |
| Network | none | RDMA, 100 Gb/s or more |
| Gateway backend | local-xdp | remote-xdp (or kv-spdk in a card cluster) |
| Shared by several GPU servers | no | yes (Cluster modes) |
Architecture compares the topologies, and ADP card setup installs both placements.
Planning a deployment?
Talk to our engineers about your models, context lengths, concurrency and where the ADP card should sit.