4B Model Beats Postgres Query Optimizer by 81%

Small Model, Big Speedup: How a 4B LLM Outperformed Postgres's Query Optimizer

In a surprising demonstration of applied reinforcement learning, developer Rohan Bansal has trained a 4-billion-parameter language model to generate query plans that significantly outperform PostgreSQL's built-in optimizer. The model, built on a distilled Qwen variant, achieved a 1.81x geometric mean speedup across 113 join-heavy queries, translating to a 44.7% reduction in total latency compared to Postgres's default plans.

The project, detailed in Bansal's technical blog post, showcases how a relatively small model can be fine-tuned to excel at a narrow, verifiable task—in this case, navigating the combinatorial explosion of join orderings and scan strategies. The key insight: while finding optimal query plans is NP-hard, verifying whether a plan is good is simple: just measure execution time.

The Problem: Postgres's Optimizer Leaves Performance on the Table

Despite decades of research, query optimizers remain imperfect. As noted in a 2015 paper by Leis et al. and reaffirmed in a 2025 follow-up, optimizers often produce suboptimal plans due to inaccurate cardinality estimates. Postgres relies on statistics and assumes uniform data distribution, which can lead to catastrophic misestimations that cascade through join trees.

The search space is staggering: a simple 3-table join has 4,608 possible execution plans, while an 11-table join explodes to over 378 sextillion combinations. Postgres uses dynamic programming and heuristics to prune this space, but it can still miss better plans.

The Solution: Agentic Reinforcement Learning

Bansal's approach involved using pg_hint_plan, a Postgres extension that allows developers to inject structured hints into SQL comments to influence the planner. The 4B model, a distilled version of Qwen 3.8, was trained as an agent that proposes these hints through a custom harness called qo-agent.

The training pipeline comprised two stages. First, off-policy distillation using 400 trajectories generated by OpenAI's GPT-6 Astra, teaching the model the harness's tool-use format. Then, agentic reinforcement learning with a custom GRPO variant, where the model's proposed plans were actually executed against Postgres and rewarded based on measured speedups.

Overcoming Measurement Noise

A critical challenge was measurement noise. Bansal found that Linux page cache and Postgres's shared buffers introduced significant variability, with some queries showing bimodal execution times. Through calibration, he discovered that increasing shared_buffers from 128MB to 2GB reduced the "fooling rate" (phantom speedups due to noise) from 5% to under 2%, while also making queries run faster overall.

Results and Analysis

The final model, after 1,200 optimizer updates, achieved:

• 1.81x geometric mean speedup across all 113 JOB queries (best-of-15 selection)
• 44.7% total latency reduction (1.81x workload speedup)
• Zero regressions when selecting the best candidate across three rollouts
• Valid candidate plans for 101 out of 113 queries (up from 14 for the vanilla model)

The model learned several effective strategies: forcing nested loop joins over hash joins, preferring index scans, rewriting join orders via Leading hints, and enabling parallel execution. Interestingly, it rarely used Rows corrections, suggesting it focused on structural changes rather than cardinality fixes.

Cost and Implications

The entire project cost approximately $1,200, including $800 for H100 rental and $400 for OpenAI API fees. This demonstrates that training specialized small models is accessible to individuals and small teams.

Bansal's work has broader implications for database optimization: instead of relying on a one-size-fits-all optimizer, organizations could train lightweight models on their specific workloads and databases, amortizing the upfront training cost across repeated queries. The approach is particularly suited for analytic workloads where queries run thousands of times with suboptimal plans.

Looking Ahead

The researcher suggests several future directions, including comparing against structured hint sweeping (like Bao), exploring on-policy distillation, and testing trace inversion for reasoning summaries. The code is open-sourced on GitHub, inviting replication and further innovation.

This experiment underscores a growing trend: small, specialized models can outperform both general-purpose LLMs and traditional algorithms on narrowly defined tasks, offering a cost-effective path to optimization in database management and beyond.