I Taught a Stick-Figure NPC to Dodge Bullets With a Neural Network I Wrote From Scratch

The author implemented a reinforcement‑learning agent entirely in vanilla JavaScript, constructing a 4‑input, 8‑hidden, 4‑output neural net that receives the nearest bullet’s relative position and velocity and outputs Q‑values for four movement directions. By applying epsilon‑greedy exploration, a 2,000‑item experience replay buffer, and the Bellman update, the NPC progresses from random flailing to consistently weaving through a barrage of bullets after a few hundred episodes, earning +0.1 reward per survival frame and a –10 penalty on impact. A second prototype moves the heavy lifting to Python, using PyTorch for a richer eight‑dimensional state (relative player coordinates and distance) and pywebview to sync the browser UI with the training loop, allowing the agent to learn melee combat tactics against a human opponent.

This experiment spotlights a growing divergence between classic game AI—hand‑crafted finite‑state machines that have powered titles like Halo and Half‑Life for decades—and emergent learning‑based agents that acquire behavior without explicit scripting. By echoing DeepMind’s Atari DQN architecture in a browser, the demo proves that modern RL techniques are now lightweight enough for hobbyist environments, echoing industry moves such as Unity ML‑Agents and Epic’s recent integration of reinforcement learning into Unreal Engine. The shift promises more adaptive, player‑responsive enemies, but also raises questions about development pipelines, reproducibility, and the balance between deterministic design and stochastic learning outcomes.

Looking ahead, the primary hurdles are scalability and control: the JavaScript DQN runs slowly compared with GPU‑accelerated libraries, limiting scene complexity, while the PyTorch version still requires a separate runtime and careful synchronization. Game studios will need tooling that abstracts these challenges, offers safety nets against erratic agent behavior, and integrates with existing asset pipelines. Monitoring open‑source projects that bridge RL with mainstream engines, as well as cloud‑based inference services that can offload training, will be essential to gauge whether learning NPCs can move from novelty demos to production‑ready components.