Critic-Then-Revise

What it is

Critic-Then-Revise is a generative AI workflow where an initial LLM output is immediately subjected to a structured critique by another LLM, which then provides targeted feedback for a subsequent revision. This process typically involves two distinct roles: a "generator" LLM that produces the initial draft, and a "critic" LLM that evaluates this draft against a predefined rubric or set of constraints. If the draft fails to meet the specified criteria, the critic's feedback is automatically fed back to the generator, prompting it to revise and resubmit. This iterative loop continues until the output satisfies the rubric or a maximum number of retries is reached. The core advantage lies in embedding quality control directly into the generation process, reducing the need for extensive human intervention in early drafts.

Why it matters

For a small team operating a portfolio of products, maintaining consistent quality and voice across diverse content needs is a constant challenge. Critic-Then-Revise addresses this by automating a significant portion of the editorial review, ensuring that generated content adheres to specific guidelines from the outset. This workflow allows us to produce higher volumes of targeted content, like those required for Programmatic SEO initiatives, without compromising on the nuanced requirements of each product's brand. It shifts the human effort from line-editing to rubric design and oversight, enabling greater throughput and freeing up time for strategic work. The system acts as a persistent, objective editor, applying the same standards consistently across every piece of content, which is crucial for building a robust Content as Funnel Inventory.

How TV applies it

At Total Ventures, we deploy Critic-Then-Revise extensively across our content generation pipelines. For instance, when creating product descriptions or blog posts for F1, Inky, or PPH, an initial draft is generated by an LLM like Claude Code or Gemini. This draft is then passed to a separate "critic" instance, often running the same model but with a distinct prompt focused on evaluating against a detailed rubric. This rubric includes checks for factual accuracy, brand voice adherence, keyword density, and structural requirements. If the draft scores below a threshold, the critic provides specific, actionable feedback – "add more examples of user benefit," "tone too formal for Inky," or "missing call to action." This feedback is then automatically fed back to the generator LLM for revision. This iterative process is a key component of our broader AI Agent Orchestration strategy, where autonomous agents handle complex tasks with minimal human oversight. This ensures that content destined for platforms like Vercel-hosted sites or Resend-powered email campaigns meets our quality benchmarks before human review.

Common failure modes

While powerful, Critic-Then-Revise is not without its challenges. The most frequent failure mode stems from poorly defined or ambiguous rubrics. If the criteria are subjective or contradictory, the critic LLM may struggle to provide actionable feedback, or the generator LLM may fail to interpret it correctly, leading to unproductive revision loops. Another common issue is the "hallucinating critic," where the critic LLM invents issues or misinterprets the content, providing incorrect feedback. Conversely, an overly lenient critic might approve substandard content. Cost is also a consideration; each iteration involves additional LLM calls, which can accumulate rapidly if the process requires many retries. Finally, setting an appropriate maximum retry limit is critical to prevent infinite loops and manage compute resources effectively.