prompt-evolution-ga
Prompt Evolution GA
When you ask an AI to write a program, the exact wording of your instructions changes how good the result is. This project hunts for better wording automatically: five hand-written instruction sets get mixed and tweaked across generations, the way breeding keeps what works. Scoring is not opinion: each one's generated code is actually run against real test cases. The honest finding: the best hand-written instruction held its own.
Built for a Computational Intelligence course; the paper is co-authored with Ghufran Alvi, and the implementation here is our founder Salman Adnan's work.
Overview
A genetic algorithm that evolves system-level instruction prompts for LLM code generation. Each candidate prompt is scored by an objective fitness function: how many real test cases the generated C++ passes.
Gemini generates a C++ solution per competitive-programming problem, g++ compiles it, the binary runs against real test cases, and fitness is the fraction of tests passed. Crossover and mutation are performed by a second, stronger LLM operating directly on the prompt text.
Key features
- Chromosome: a natural-language instruction prompt, starting from 5 hand-written seeds.
- Fitness: the mean pass rate of gemini-2.5-flash-generated C++ over 65 APPS problems, compiled with g++ -O2 -std=c++23.
- Operators: LLM-mediated crossover plus three mutations, done by gemini-2.5-pro.
- Selection: tournament, roulette, or tournament with elitism.
- 12 generations, population 5, then evaluation on a 200-problem held-out split, with parallel fitness evaluation, per-call cost tracking, and resumable checkpoints.
Results
- Mean pass rate on the 200 test problems: baseline (best hand-written seed) 0.7900, tournament-evolved 0.7857, roulette 0.7717, elitism 0.7713. The strong hand-written seed held its own, which is reported plainly.
- Three runs made roughly 13,000 LLM calls and cost about $17; a single full run is about 4,000 to 4,700 calls, 4 to 6 hours, and $5 to $6.
Tech stack
A challenge worth noting
Making fitness objective at all: rather than asking a model whether output looks right, each candidate is scored by compiling and running the generated C++ against the problem's real test cases. That made the signal trustworthy but pinned the project to a working g++ toolchain, which is a system dependency that has nothing to do with genetic algorithms.