CRAYON
Every story · every side  ·  est. 1995
Technology

Homelabber’s second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second

Photo: Tom's Hardware
CRAYON summary

Tom's Hardware reports that a Reddit homelabber's second-generation server built around Optane PMem 200 memory now runs Moonshot AI's 2.8-trillion-parameter Kimi K3 model at 3.11 tokens per second. The build uses two RTX 3090 graphics cards and a RAM drive. According to the outlet, the builder says the Optane modules still need their own fans.

Based on Tom's Hardware's report - only one outlet has covered this so far. Read the original.
▲ Sign in to vote stories up

Coverage 1 source · 1 report

Chronological.

  1. Tom's Hardware FIRST 1h ago Homelabber’s second-gen Optane build uses two RTX 3090s to run 2.8-trillion-parameter Kimi AI model at 3.11 tokens per second

    A Reddit homelabber’s Optane PMem 200 server now runs Moonshot AI’s Kimi K3 at 3.11 tokens/s with two RTX 3090s and a RAM drive.

Discussion 0 comments