AI fabrics stopped being dumb pipes — reduction, load balancing and congestion control all migrated into the switch. But the algorithms those chips implement are not settled science, and an ASIC commits to its answers at tape-out. Why flexibility is a specification, not a footnote.
PyTorch has two mechanisms that abort a stuck distributed job and they mean opposite things. A watchdog timeout names a stalled collective; a heartbeat abort means the monitoring thread itself was blocked — a CUDA hang or a GIL deadlock, not the network. Plus the free straggler signal already sitting in the timeout output.
The interface exists, is up, and has the node IP — and bootstrap still fails. On modern GPU nodes the cause has moved from docker0 to SR-IOV virtual functions and secondary physical functions, and any static interface filter is a guess about node topology that silently stops being true.
Two structural traps make most published GPU cluster bandwidth figures meaningless: two-node benchmarks measure NVLink rather than the fabric, and in-network reduction breaks the arithmetic bus bandwidth is derived from. What to measure instead, and why per-pair variance matters more than any average.
Every rank waits for the slowest one, so a single degraded GPU taxes the whole cluster. Meta measured infrastructure failures hitting 0.2% of jobs but 18.7% of runtime. Telling a slow GPU from a slow link from a slow dataloader needs correlation against physical telemetry, not a ranked list of step times.
There is no cryptography in a transceiver EEPROM — the MSA requires only that a vendor name or OUI field contain valid data. Yet third-party modules still cause real problems, for a reason almost nobody names: on SONiC the vendor string selects SerDes tuning, not acceptance, and an unmatched module silently gets generic settings.
A training job stalls for seconds, then recovers, and every link still shows up. Alibaba measured 5,000 to 60,000 flap events a day in production, and a single flap halting training for over nine seconds. Why it happens, how to find it, and why tolerating it is a topology decision made at purchase time.
A failed collective produces hundreds of lines and roughly one of them is the cause. Which line matters, why status=5 is never it, how to tell RoCE from InfiniBand without asking, and the triage order that stops you debugging flush noise for a day.
Two causes account for most clusters measuring roughly half their expected bandwidth with every link reporting healthy: ACS routing traffic through the CPU root complex, and routing collisions on classic per-rack cabling. One is a setting you fix this afternoon. The other is a cabling decision made when the rack is specified.
Two roads lead to disaggregated memory: CXL, a new memory interconnect, and RDMA over standard Ethernet. They differ in programming model, hardware dependency, and — decisively — timeline. Here is how they compare, and why TORmem builds on Ethernet and RDMA today.
Memory disaggregation separates memory from the servers that use it, turning it into a shared pool many servers reach over a fast fabric. This explainer covers the problem it solves, how RDMA over Ethernet makes it practical, why it matters for AI, and how TORmem approaches it.
A new class of AI systems called world models processes time instead of tokens. These systems require persistent temporal memory far beyond GPU HBM capacity. This post explains why world models break GPU memory limits and introduces Memory-as-Infrastructure for AI Time Context enabled by TORmem.
For more than a decade, scale-out has defined how data centers grow. But modern AI workloads are no longer limited by raw compute—they are limited by memory. In this post, we explore why scale-out is breaking and why memory-centric architecture is the future.
GPUs keep getting faster, but inference doesn't scale the way it should. The reason is no longer compute. It's memory—specifically Key-Value (KV) cache. In this post, we explain why KV cache has become the dominant bottleneck in modern AI inference.
The TORmem AI Edge Platform is designed to address memory bottlenecks in edge AI. Built on a memory-centric architecture, it enables entry-scale deployments that seamlessly scale to large-memory workloads without redesigning infrastructure.
How Traditional Compute, GPU, and Storage Architectures Waste Enormous Capital — and How TORmem Solves It with Dynamic Memory Allocation.
TORmem takes a fundamentally different architectural approach—one focused on practical AI inference rather than theoretical peak performance. TORmem systems are designed around a memory-centric architecture that decouples memory growth from GPU constraints, delivers enterprise-grade stability in a compact form factor, and provides predictable, repeatable performance across real workloads.
In the world of technology, much of the focus is on software as the center of value. But memory disaggregation is a system problem — one that can only be solved by purpose-built hardware and custom software integrated as a single, cohesive system.
TORmem is proud to announce that we are joining the OpenCAPI consortium, a leading industry group dedicated to enabling the future of low-latency disaggregated memory — adopting the Open Memory Interface (OMI) in our products.
Building disaggregated memory appliances through a global component shortage. How TORmem designs for availability, multi-sources aggressively, and stays flexible enough to redesign around supply issues that stall larger companies.
TORmem Inc., a pioneering self-funded startup specializing in memory disaggregation solutions for data centers, today announced an ambitious high-speed networking roadmap that promises to reshape the enterprise switching market. The company unveiled its complete portfolio spanning 100G switches available immediately, 200G and 400G switches entering production in Q4 2025, and 800G switches currently under development – all at 50% less cost than established industry leaders.
OPF will be showcasing our LibreBMC project and talking about all things open hardware and software. The best part is that we won't be alone, as we have a number of OPF community members and partners showcasing at our booth as well, including Open Compute Project, Axiado, Raptor Computing, Oregon State University, and TORmem.
The industry is impatient for disaggregated and shared memory for a lot of reasons, and many system architects don't want to wait until PCI-Express 6.0 or 7.0 transports are in the field and the CXL 3.0 and beyond protocols that ride on it to reach out to external memory have been tweaked to do proper sharing across servers.
