I Built a 3D Office Where Local AI Agents Share One RTX 4090
The project, dubbed AI Office, stitches together Next.js 15, React Three Fiber, Zustand on the front end and FastAPI with PostgreSQL on the back end. Each avatar represents a self‑contained agent that follows a four‑step workflow—plan, research, draft, self‑check—using the locally hosted Qwen 3.8 27B model (quantized to q4_K_M, 16 K context) that occupies roughly 17 GB of the 24 GB RTX 4090 VRAM. Inference runs at about 100 tokens / s, delivering a typical two‑minute write‑up, extending to five minutes when web searches via a private SearXNG instance are required. The author built the entire stack in three days with Claude Code, reduced draw calls from 893 to 117 by switching to instanced meshes, and implemented a “coffee‑machine” queue visual to expose GPU contention when multiple agents request the model simultaneously.
This experiment sits at the intersection of on‑device LLM deployment and UI‑driven productivity tools. By keeping the model entirely local, the author sidesteps the latency and privacy concerns that dominate cloud‑based assistants, echoing recent pushes from companies like Apple and Nvidia to run large language models on consumer hardware. The choice of an open‑source 27B model over a 30B alternative reflects a broader trend of developers benchmarking quantized models for single‑GPU workloads, while the use of a relational database as the single source of truth mirrors enterprise patterns for state synchronization across distributed agents.
The prototype highlights both promise and pitfalls. The explicit self‑check stage and strict system‑prompt rules successfully curb hallucinations, yet the reliance on a single GPU creates a hard bottleneck; any orphaned process can halve throughput, as the author experienced. Moreover, the read‑only web‑search sandbox mitigates data leakage but may limit the agents’ ability to interact with authenticated services. Future iterations will need smarter scheduling, possibly multi‑GPU or off‑load strategies, and more robust sandboxing to balance capability with security.
Key Takeaways
Running a 27B LLM on a single RTX 4090 can support multiple concurrent agents, but GPU queuing becomes visible and must be managed in the UI.
Strict system prompts and a line‑by‑line self‑check can dramatically reduce factual hallucinations in locally hosted agents.
Instanced rendering cuts draw calls by an order of magnitude, enabling smooth 165 Hz updates even with dozens of moving avatars.
The sandboxed SearXNG search layer prevents private‑network access, yet its read‑only nature may constrain real‑world workflow automation.
About the Source
This analysis is based on reporting by HackerNoon. Here is a short excerpt for context:
A local 3D office where AI agents on one RTX 4090 plan, research, draft and self-check real tasks. The rendering was easy; reliability was the hard part.Read the original at HackerNoon
