Why AI Trading Agents Are Hard to Trust

Bloomberg examined roughly two million Polymarket wallets active since early 2025 and found that more than 100 thousand of them incurred losses exceeding $1,000 each. Retail participants collectively shed about $131 million, whereas the top 1 percent of accounts—largely automated bots—snagged over 80 percent of the net profit. University of San Diego professor Joshua Della Vedova notes that human traders actually predicted outcomes more accurately than the bots; the bots’ edge lay in entering positions earlier at better prices. This pattern highlights that speed and timing, not superior forecasting, drive the current AI‑trading advantage.

The report distinguishes legacy rule‑based scripts from the newer “agents” that leverage large language models to ingest news, form opinions, and act autonomously. A 2026 study of the TradingAgents framework demonstrated that even with identical market inputs, repeated runs produce divergent trade sequences and equity curves, confirming non‑determinism as an intentional design trait. Conventional back‑tests, which assume repeatable outcomes, therefore capture only a single draw from a wide distribution, inflating perceived performance. An industry estimate from the same year suggests roughly 95 percent of retail “AI” trading products are merely rule‑based tools repackaged with AI branding, offering no genuine adaptive judgment.

The practical upshot is that investors cannot rely on a single back‑test to validate an AI agent’s profitability; real‑world performance will hinge on opaque, context‑driven decisions that cannot be audited in advance. This uncertainty raises regulatory red flags and pushes sophisticated users toward hybrid architectures—using language models for data extraction and sentiment summarisation while delegating execution to deterministic, rigorously back‑tested rule sets. Watch for emerging platforms that explicitly separate research from trade execution, and for any standards bodies that begin to require reproducibility disclosures for AI‑driven trading systems.