Technology
Homelabber’s second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second
CRAYON summary
Tom's Hardware reports that a Reddit homelabber's second-generation server built around Optane PMem 200 memory now runs Moonshot AI's 2.8-trillion-parameter Kimi K3 model at 3.11 tokens per second. The build uses two RTX 3090 graphics cards and a RAM drive. According to the outlet, the builder says the Optane modules still need their own fans.
Based on Tom's Hardware's report - only one outlet has covered this so far. Read the original.▲ Sign in to vote stories up
Coverage 1 source · 1 report
Chronological.
-
Homelabber’s second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second
A Reddit homelabber’s Optane PMem 200 server now runs Moonshot AI’s Kimi K3 at 3.11 tokens/s with two RTX 3090s and a RAM drive.
Discussion 0 comments