Skip to content
Writing
AIAutomationLeadership

How I Set Up AI for Coding and QA

The machinery behind an agentic delivery pipeline: what the agents read, what they can touch, what never reaches them, and what QA checks.

· 3 min readOn the map: AI & automation

Most teams treat AI as autocomplete. That caps the value at faster typing. My team uses it to cover the distance between a ticket exists and a checked pull request is waiting for review — and the setup around the agents matters more than the model doing the work.

The stages

  1. The ticket is the input. An agent pulls the Jira ticket directly — acceptance criteria, links, context — rather than a human retyping it as a prompt.
  2. The design is read, not described. The agent reads the Figma file itself, so spacing, states and tokens come from the source, not from someone's paraphrase of a frame.
  3. Implementation arrives as a pull request. A branch and a diff against the real codebase, not a snippet in a chat window.
  4. Separate agents run QA. A different stage checks the work before any human does.
  5. A human approves. Nothing merges without a person.

The wiring is less exotic than it sounds. Tools such as Claude Code connect to issue trackers, design files and GitHub through the Model Context Protocol, an open standard for tool integrations; the documentation's own example is implementing a Jira issue and opening a pull request. The integration is the easy part.

Sources of truth, not summaries

The typical failure of AI-assisted work isn't bad code. It's code that solves a slightly different problem than the one filed. Every paraphrase of a ticket or a design is a chance to drop a requirement or invent one.

Connecting the agent to the ticket system and the design tool removes those paraphrases. It is a boring architectural decision that matters more than any prompt.

Context is written down, in the repository

An agent starts every task knowing nothing about the project. It cannot hold a large codebase in its head, and much of what matters — which pattern we are standardizing on, which module is being retired — isn't in the code at all.

So the context lives in files the agent reads first: repository instruction files with build commands and conventions, short architecture notes, and a definition of done. Claude Code loads a project CLAUDE.md at the start of every session and shares it with the team through source control, so the rules change through review like any other code. The same documentation is clear that these files are context, not enforcement — which is why anything that must hold is enforced somewhere else.

If a convention is not written down, the agent will not follow it — and neither will the next new hire.

Least privilege, by default

OWASP lists excessive agency — too much functionality, too many permissions, too much autonomy — among the top risks of LLM systems. The setup answers each one:

  • Only the tools the task needs. Ticket, design, repository. Nothing else is connected.
  • Write access to a branch, never to the main line. The agent can push its branch and open a pull request. Merging is not in its permissions.
  • No production credentials within reach. An agent that cannot reach production cannot break it.
  • A person approves the merge. Branch protection makes an approving review a precondition for merging, so the gate is a setting, not a habit.

What never goes in

Secrets, credentials, client data and anything confidential never enter an AI tool. Not in a prompt, not in a ticket the agent reads, not in a context file.

A rule containing judgment calls fails under deadline pressure. "Don't share anything sensitive" asks a tired engineer, the evening before a release, to classify what counts. Categories remove the classification. The line fits in one sentence, it is decided in advance, and nobody has to deliberate over it mid-sprint.

What the QA agents check

An agent asked to check its own work thinks the work is good. Verification is a separate stage for the same reason an engineer doesn't approve their own pull request. The QA agents:

  • Check the change against the ticket's acceptance criteria, item by item.
  • Generate and run tests for the behavior the ticket describes, alongside the existing suite.
  • Compare the rendered UI against the design for layout, states and tokens.
  • Run automated accessibility checks. These catch the mechanical failures; they do not replace manual testing.

This stage cheaply catches skipped criteria, missing states and regressions in adjacent components. It does not catch architectural drift. That belongs to the human reviewer.

How I know it is healthy

I watch three signals, as trends rather than targets:

  • Rework. How often a pull request goes back because it missed the ticket. Rising rework usually means the context files have fallen behind the codebase.
  • Review time. If review stretches, the diffs are too large or the QA stage is letting noise through.
  • Defects that reach QA or production. The pipeline exists to catch problems earlier. Anything that escapes further tells me which stage to tighten.

None of these measure the model. They measure the setup, which is the part I own.

AI boosts the delivery process while keeping the best practices. The setup is what keeps the second half of that sentence true.