20 Loop Design Patterns Every AI Engineer Should Know

Most AI engineers know how to build an agent.

Very few know how to build a system that gets better after the first attempt.

That gap is worth six figures.

Here's the difference:

An agent is a worker.

A loop is what makes the worker improve.

The most capable AI systems in production today are not single model calls.

They are loops.

Generate → Evaluate → Learn → Improve.

Over and over.

Until the output is actually good.

Here are 20 loop design patterns that show up repeatedly in production AI systems.

Save this. You will build with these.

Agents vs Loops

Old way: Prompt → Response → Done.

New way: Generate → Critique → Rewrite → Score → Retry → Remember → Improve.

One is a factory worker who does the job once.

The other is a factory worker who studies every mistake, rewrites the playbook, and gets 3% better every single shift.

The teams shipping production AI right now are not writing better prompts.

They are building better loops.

CATEGORY 1 — QUALITY IMPROVEMENT LOOPS (Make the output better before it leaves the system)

1. Generate → Critique → Rewrite

The most important loop in AI engineering.

Generate output. Critic reviews it. Generator rewrites based on feedback. Repeat until quality threshold is met.

Not one model. Two roles. One pipeline.

1[Generator] → draft
2[Critic]    → "paragraph 3 is vague. missing evidence. tone is off."
3[Generator] → rewrite based on critique
4[Critic]    → "better. but conclusion still weak."
5[Generator] → final rewrite

Used for: writing, code review, reports, strategy docs, sales emails.

The insight: the model that generates is not the best judge of its own output.

A separate critic finds what the generator missed every time.

2. Score-and-Retry Loop

Generate. Score. Retry if below threshold.

Simple. Powerful. Underused.

score = evaluate(output)

1score = evaluate(output)
2
3while score < threshold:
4    output = generate(prompt)
5    score = evaluate(output)
6    attempts += 1
7    if attempts > max_retries:
8        return best_so_far

Best when quality is measurable — extraction accuracy, format compliance, factual correctness, lead scoring.

The generator doesn't know it's being graded.

The evaluator does.

That separation is the pattern.

3. Multi-Critic Loop

One critic has blind spots.

Use four.

→ Correctness critic: is it factually accurate?

→ Style critic: is it clear and well-written?

→ Safety critic: is it appropriate and safe?

→ Domain critic: does it meet specialist standards?

Each evaluates independently.

Final output must satisfy all four before it exits.

Used in: medical AI, legal document review, financial analysis, regulated content.

4. Adversarial Critique Loop

The critic's only job is to break the answer.

Not improve it. Break it.

Questions the adversarial critic asks:

→ What assumptions fail here? → What evidence is missing? → What would a skeptic say? → Where is this confidently wrong?

The generator then defends or rewrites.

The best answer survives the attack.

Used for: research synthesis, investment thesis review, strategic planning, risk analysis.

5. Judge Ensemble Loop

One judge gives noisy scores.

Five judges average out the noise.

Run the same output through multiple evaluators.

Aggregate scores.

Only outputs with high consensus advance.

Used when: single-model evaluation is unreliable, stakes are high, edge cases matter.

CATEGORY 2 — MEMORY LOOPS (Learn from what happened so next time is smarter)

6. Reflexion Loop

The most important self-improvement pattern that exists.

Agent fails. Agent analyzes why it failed. Agent stores the lesson. Agent retries with that lesson in context.

Each iteration: smarter than the last.

1attempt 1: fails
2reflection: "I assumed X but X was wrong. Next time verify X first."
3attempt 2: incorporates lesson → partial success
4reflection: "Better. But I skipped Y. Add Y check."
5attempt 3: succeeds

The difference between a system that fails once and a system that only fails once.

7. Memory Update Loop

After every task, store three things:

→ What decision was made → What the outcome was → What would be done differently

Future runs inherit this knowledge.

The system in month 6 is not the same as the system in month 1.

It has read 6 months of its own history.

8. Error Library Loop

Store every failure.

Wrong answer. Bad output. Failed execution. Edge case.

Before acting on a new task:

Search the error library first.

If a similar failure exists → apply the known fix before even starting.

The system stops making the same mistake twice.

The most underused pattern in production AI.

9. Success Pattern Loop

Most engineers only store failures.

Store successes too.

When a task goes well:

→ Save the approach → Save the context → Save what made it work

Retrieve successful patterns when facing similar tasks.

Learn from wins. Not just mistakes.

10. Memory Compression Loop

Memory grows forever.

Unlimited memory is unusable memory.

After N items accumulate:

Compress them.

Many specific memories → fewer higher-level abstractions.

1Before compression:
2"Failed on task A because of X"
3"Failed on task B because of X"
4"Failed on task C because of X"
5
6After compression:
7"Pattern: X causes failures. Always check X first."

Context stays manageable. Patterns stay accessible. System stays fast.

CATEGORY 3 — PLANNING LOOPS (Adapt the plan when reality changes)

11. Plan → Execute → Replan

The most common mistake in AI agent design:

Treating the plan as fixed.

Plans break on contact with reality.

The pattern:

Create plan → execute step → observe outcome → update plan → continue

Not a waterfall.

A spiral.

Each lap around tightens the approach.

Used when: environment changes, tasks have dependencies, long-horizon goals.

12. Dynamic Workflow Loop

Most pipelines are fixed.

Step 1 → Step 2 → Step 3. Always.

Dynamic workflows change based on results.

If output A → run branch X If output B → run branch Y If output C → skip to step 5

The pipeline decides its own shape at runtime.

Used in: multi-document research, customer support routing, adaptive content pipelines.

13. Goal Decomposition Loop

Large goal enters.

System breaks it into subgoals.

Each subgoal breaks into tasks.

Each task breaks into steps.

Decompose until each unit is small enough to execute in one call.

1Goal: "Write a comprehensive competitive analysis"
2↓
3Subgoal 1: "Identify top 5 competitors"
4Subgoal 2: "Analyze each competitor's product"
5Subgoal 3: "Compare pricing models"
6Subgoal 4: "Identify gaps"
7↓
8Each subgoal → tasks → individual model calls

The loop keeps decomposing until the system can act.

14. Progress Evaluation Loop

Every N steps: stop and ask.

"Are we actually getting closer to the goal?"

If yes: continue current strategy. If no: change strategy, tools, or plan.

The system monitors its own progress.

Not just executes blindly.

Used in: long-running research agents, multi-day autonomous tasks, debugging agents.

15. Constraint Satisfaction Loop

Keep running until all constraints are met.

1while not all_constraints_satisfied(output):
2    output = improve(output, unsatisfied_constraints)
3
4constraints = [
5    budget_under_limit,
6    quality_above_threshold,
7    latency_under_200ms,
8    tone_matches_brand,
9    no_hallucinations
10]

Very common in production systems.

The output is not done until every business rule passes.

CATEGORY 4 — EXPLORATION LOOPS (Find the best answer by trying multiple paths)

16. Branch-and-Explore Loop

Don't commit to one path.

Explore several simultaneously.

1paths = [
2    generate(approach="conservative"),
3    generate(approach="aggressive"),
4    generate(approach="creative")
5]
6
7scores = [evaluate(p) for p in paths]
8best = paths[scores.index(max(scores))]

Compare outcomes. Choose the best branch. Discard the rest.

Used for: content variations, architecture decisions, debugging multiple hypotheses, A/B generation.

17. Tree Search Loop

Branch-and-Explore goes one level deep.

Tree Search goes as deep as needed.

Expand the most promising nodes. Prune the weakest ones. Keep exploring until the solution is found.

1root → [A, B, C]
2A → [A1, A2]   # A looks promising, expand it
3B → prune      # B is weak, stop here
4A1 → [A1a, A1b]
5A1a → solution ✓

Used for: complex reasoning chains, multi-step planning, code debugging, research synthesis.

Computationally expensive but finds solutions single-pass calls cannot.

18. Debate Loop

Two agents. One topic. Opposite positions.

Agent A argues for the answer. Agent B argues against it.

Each round challenges assumptions, demands evidence, exposes weak logic.

Final answer emerges through disagreement.

Not through agreement.

The adversarial pressure finds what confident single-agent answers miss.

Used for: investment decisions, strategic planning, risk assessment, research critique.

CATEGORY 5 — SYSTEM OPTIMIZATION LOOPS (The loop improves the loop)

19. Prompt Optimization Loop

Most engineers write a prompt once and never touch it again.

Prompt optimization loops change that.

The system:

→ Runs the prompt on a test set

→ Scores every output

→ Identifies where the prompt fails

→ Rewrites the prompt to fix those failures → Reruns and rescores

The prompt gets better automatically.

Without a human touching it.

1current_prompt = "Summarize this document."
2
3for iteration in range(max_iterations):
4    outputs = [run(current_prompt, doc) for doc in test_set]
5    scores = [evaluate(o) for o in outputs]
6    avg_score = mean(scores)
7
8    if avg_score >= target:
9        break
10
11    failures = [o for o, s in zip(outputs, scores) if s < threshold]
12    current_prompt = improve_prompt(current_prompt, failures)
13    # Prompt rewrites itself based on where it fails

Used in: production pipelines, automated content systems, classification tasks.

The best prompts in production AI were not written by a human.

They were evolved.

20. Workflow Optimization Loop

This is where it gets interesting.

The loop improves the loop.

The system measures its own performance:

→ latency: how long does each step take?

→ cost: how many tokens does each call use?

→ quality: what is the output score at each stage?

Then it modifies its own workflow.

Too slow? Parallelize two steps. Too expensive? Replace a GPT-4 call with a smaller model where quality holds. Quality dropping? Add a critic before the final output.

1metrics = measure_workflow(outputs, latency, cost)
2
3if metrics.latency > target_latency:
4    workflow = parallelize(slow_steps)
5
6if metrics.cost > budget:
7    workflow = replace_with_cheaper_model(high_cost_steps)
8
9if metrics.quality < threshold:
10    workflow = add_critic_before(final_output_step)

This is where truly self-improving systems begin.

Not just outputs that improve.

Systems that redesign themselves.

The pattern behind all 20 patterns

Every single loop above shares one structure:

Act → Observe → Evaluate → Adjust

That is the entire recipe.

The output is never final on the first attempt.

The output is a starting point.

The loop is what turns a starting point into something production-worthy.

The full map

Category 1 — Quality Loops (Make output better before it leaves)

→ 1. Generate → Critique → Rewrite

→ 2. Score-and-Retry

→ 3. Multi-Critic

→ 4. Adversarial Critique

→ 5. Judge Ensemble

Category 2 — Memory Loops (Learn from what happened)

→ 6. Reflexion

→ 7. Memory Update

→ 8. Error Library

→ 9. Success Pattern

→ 10. Memory Compression

Category 3 — Planning Loops (Adapt when reality changes)

→ 11. Plan → Execute → Replan

→ 12. Dynamic Workflow

→ 13. Goal Decomposition

→ 14. Progress Evaluation

→ 15. Constraint Satisfaction

Category 4 — Exploration Loops (Find best answer by trying many paths)

→ 16. Branch-and-Explore

→ 17. Tree Search

→ 18. Debate

Category 5 — System Optimization Loops (The loop improves the loop)

→ 19. Prompt Optimization

→ 20. Workflow Optimization

Most engineers think agents are the future.

Agents are just workers.

Loops are what make workers improve.

The biggest shift happening in AI right now is not better models.

It's moving from:

Prompt → Response

to

Generate → Evaluate → Learn → Improve

The teams that master loop design will not build better prompts.

They will build systems that get better every single day after deployment.

Without anyone touching them.

If this was useful:

→ Repost to share it with every AI engineer you know

→ Follow @sairahul1 for more patterns like this

→ Bookmark this — pick one loop and implement it this week

I write about AI, building products, and systems that work without you.