1 / 5

What a software factory has to do

Scripts check links and freshness; tests check behaviour. AI and people review meaning.

Writing documents and code is widely available. Keeping them aligned takes more.

Speaker notes

Agents now write requirements, specs, code and tests quickly. For a system that has to be maintained and audited for years, that is not the hard part. The hard part is that all of these documents still agree with each other and with the code a year later.

The first property is meaningful integration: a design decision addresses requirements, code implements the design, and tests verify required behaviour. IDs and links need maintained meaning and versions. The second is a complete intended design for a selected version, available before implementation finishes. Proposed, approved but unimplemented, and implemented changes must be distinguishable. A delivered code version must identify the design it implements.

Third, drift detection must lead to tracked reconciliation. A freshness or suspect-link flag is a prompt to review, not proof that code is wrong. A finding needs an affected area, a review or correction, and a recheck. Automatic document rewriting is not required. Fourth, the connected information should support assessing changes and comparing design alternatives against the same requirements, including constraints, assumptions and trade-offs.

Fifth, executable tests must check the implementation against its requirements at the end of the work, with results bound to the tested code version. Test-file links and code coverage alone are insufficient. Sixth, delivery provenance identifies the build, governing documents, reviews, approvals and waivers. It references test results without counting test adequacy a second time.

Seventh, process portability means the factory can continue running elsewhere. Exportable code and Markdown alone do not qualify. A proprietary licensed runtime can qualify if it can be deployed on different supported systems. Changing engines is not required, and model-service dependencies are stated separately.

For every check, distinguish available capability, trigger, blocking enforcement and tracked resolution. Structural scripts, executable tests, AI review and human decisions supply different evidence. Flexible configuration is not a weakness. Governance is discussed in the report without a score. Deployment, operation and measured effectiveness are outside the scored comparison.

2 / 5

What exists today

TypeExamplesGood atMissing
Issue-to-PR agentsCopilot coding agent, Jules, Devin, Azure Boards → CopilotA ticket becomes a pull requestA persistent specification is not required
Spec workflowsKiro, Spec Kit, OpenSpec, BMADRequirements and design before codeCompleteness, version alignment and check enforcement vary
Requirements toolsJama, Polarion; Azure DevOps + GitHub CopilotRequirements and test traceability; Copilot writes codeNo dedicated design step in the Azure Boards → Copilot handoff
Atlassian suiteConfluence, Jira, Rovo DevDocuments, work tracking and coding assistanceFull cross-layer reconciliation was not established
Agent orchestrationFactory.ai, Augment CosmosParallel agents and executable QAMaintained requirement/design model not established

Writing, maintaining and tracing documents are different capabilities. The reviewed products cover different parts.

Speaker notes

The market splits into five groups. Issue-to-PR agents such as GitHub Copilot coding agent take a ticket and open a pull request, and a person merges it. Azure Boards can hand a work item to Copilot the same way. The reviewed issue-to-pull-request workflow does not require a persistent specification between the ticket and code.

Spec workflows such as AWS Kiro, GitHub Spec Kit, OpenSpec and BMAD write requirements and a design before the code. Kiro writes requirements, design and tasks per feature, and those files stay behind as history. Spec Kit offers a living-spec option. OpenSpec merges each change into a current spec. BMAD keeps a PRD, UX, architecture and specs up to date through its own workflows. Checks can be invoked directly or connected to supported triggers. Their scope and enforcement vary; an integrated intended design and a governing version for each delivered build need separate evidence.

Requirements tools such as Jama Connect, Siemens Polarion and Azure DevOps Test Plans have done the long-term part for years. Jama and Polarion keep living requirements, approved baselines, coverage reports and suspect links that flag downstream items after an upstream change. Azure DevOps shows which requirements have no tests and links work items to commits, builds and releases. Azure Boards can send a work item to GitHub Copilot, which writes code and opens a linked draft pull request. That handoff has no dedicated design stage. Jama and Polarion connect to development tools through integrations.

Atlassian shows the problem inside one vendor. Confluence holds PRDs and decision pages, Jira holds the work, and Rovo Dev writes code from Jira items. Test management comes from marketplace apps. Page links and issue keys do not establish a maintained cross-layer model. Rovo’s AI review can flag possible acceptance-criteria mismatches, but tracked design reconciliation remains unestablished. Atlassian is also retiring standalone Rovo Dev and moving its features into Jira.

Orchestration products such as Factory.ai and Augment Cosmos run many agents in parallel with AI review. Factory's Software Factory dashboard is in private preview. Vercel's internal factory for its AI SDK reports 25 to 35% of merged pull requests, and a person merges every one.

Sources: About Copilot coding agent; Azure Boards and GitHub Copilot; Kiro specs; Spec Kit: evolving specs; OpenSpec quickstart; BMAD: plan inside an organization; Jama traceability; Jama electronic signatures; Polarion quality assurance; Azure DevOps requirements traceability; Azure DevOps end-to-end traceability; Atlassian: Rovo Dev licensing; Rovo Dev for architecting solutions; Factory Software Factory; Augment Cosmos; Vercel: building a software factory.

3 / 5

How they compare

ProductLifecycle traceabilityVersioned system designTracked drift resolutionChange impact and alternativesRequirement test resultsDelivery audit trailProcess portability
GitHub Copilot coding agent
Azure DevOps + Copilot
Atlassian suite (Confluence, Jira, Rovo Dev)
AWS Kiro
GitHub Spec Kit
OpenSpec
BMAD Method + Test Architect
Tessl
Jama Connect
Siemens Polarion
Factory.ai (Droid + Missions + QA)
Google Jules
Cognition Devin
OpenHands

Ratings describe documented support, not percentages. ? Unverified means evidence is insufficient. Not supported requires a documented restriction. Whether checks start automatically and whether failures block delivery are described separately. No overall product scores are calculated.

Product names link to detailed evidence in the report. Azure–Copilot and Rovo require hosted services. Factory’s airgapped Droid runtime is documented; continuity of its complete connected QA workflow remains unverified.

Coverage depends on the complete criterion, the configured checks and the retained evidence.

Speaker notes

The columns use the full definitions on slide 1. The labels show coverage: zero means a documented restriction, one limited support, two substantial support with a specific remaining gap, and three complete documented coverage in the stated configuration. A question mark means the evidence cannot justify a coverage level. These are ordinal documentation assessments, not percentages, vendor totals or hands-on trials.

The report distinguishes the trigger, check mechanism, blocking enforcement and tracked resolution. Flexible configuration receives no penalty. Edition and availability remain chapter details.

Azure DevOps and configured Polarion test/build integrations can connect executable test results to requirements and a build. Jama test-run history is useful, but the tested code-version association was not established in the reviewed native test-run documentation. None of these ratings substitutes test-file links or generic code coverage for implementation verification.

BMAD includes the optional Test Architect module here. Its trace matrix measures requirement coverage; passing tests remain a separate check. Persistent BMAD/Tessl documents and merged OpenSpec capability specs are partial under the stricter requirement for a complete approved target and an explicit governing design version for delivered code.

Process portability scores execution, not file retention. The Azure–Copilot and Atlassian–Rovo bundles require hosted services, so their self-managed core products earn no portability credit for the complete bundle. Kiro and Tessl have unresolved service dependencies. Jama and Polarion have deployable runtimes. Factory documents an airgapped Droid runtime, but complete connected QA continuity remains unverified; its private-preview dashboard is excluded.

A freshness warning, failed scenario or AI review finding needs a tracked decision, correction and recheck. The appropriate resolution may change code, documentation or both; automatic rewriting is not required.

Sources: Kiro correctness; Spec Kit: evolving specs; OpenSpec quickstart; OpenSpec with ADRs; BMAD: plan inside an organization; Tessl spec-driven development; Tessl spec verification; Jama baselines; Jama coverage report; Polarion collections; Azure DevOps automated tests and test cases; Cucumber overview.

4 / 5

Sample custom factory built in ~4 man-months

LevelWhereLinked byCount
Requirementsrequirements/*.mdREQ-* IDs305
Architecturearchitecture/ADR-*.md, overviewADR-* IDs14 ADRs
Specsspecs/spec-*.mdfront matter implements: [REQ-…], adrs: [ADR-…], depends_on: [spec-…]46
UI designui-design/mockups/*.html + manifest + inventoryscreen IDs; criterion IDs in manifests16 screens
Work itemscode/todo.mdTODO-CODE-*; criterion IDs in “Done when”
Teststest files@covers AC-…; test named in each “Done when”

Example: spec-021 (account transfers) cites 7 requirements and 4 ADRs and defines 32 acceptance criteria. Documents name their upstream requirements and decisions. This illustrates traceability structure, not full satisfaction of all seven criteria.

Agents wrote most of the scripts and workflows. The versioned, connected design can outlast the tools that maintain it.

Speaker notes

This is an example of a factory built for one product, a personal finance app, by one person with agents over about four man-months. It is not a product. It illustrates linked documents and local checks built for that product.

There are six levels. Requirements carry IDs such as REQ-TRF-001. Architecture decisions are numbered ADRs. Each spec states in its header which requirements it implements, which ADRs it follows and which specs it depends on. Each UI screen is a mockup built from specs, with a manifest listing the acceptance criteria it shows. Work items name the acceptance criteria they deliver, and tests name the criteria they cover.

Account transfers is a typical spec. It implements seven requirements, follows four architecture decisions and defines thirty-two acceptance criteria. Because every link is an ID in a fixed place, a script can follow the chain from a requirement to its tests and back.

The factory itself is not the asset. Its scripts and workflows were built with agents, and the next generation of agents will build them faster and better, as with skills and workflows today. What carries over is the structured, linked knowledge: requirements, decisions, screens and specs that any new tool can read and check.

5 / 5

Custom factory (slide 4): keeping levels in sync

ProblemMechanismHow
Finding everything about one itemEvery requirement, architecture decision, screen, spec, acceptance criterion and work item has a unique ID. A tool or an AI agent can search the whole repository for it.Structure
A requirement or criterion is edited after work was built on itEach document that refers to a requirement or criterion records which version it was checked against. When the text changes, every document that refers to it is flagged for review.Deterministic tool
What does a change affect?An impact tool lists everything connected to an ID: the requirements and decisions it rests on, and the specs, screens, work items and tests built on it.Deterministic tool
Acceptance criteria without linked testsA report lists criteria with no test citation. This identifies missing links, not passed implementation tests.Deterministic tool
Work marked done without proofA closure check requires each completion condition to name an existing passing test. This checks the work item, not complete requirement verification for a delivered build.Deterministic tool
Architecture decision ignored in codeEach architecture decision lists code rules that a checker enforces. Each rule is itself tested.Deterministic tool
A spec changes after deliveryA workflow reopens dependent work for review and correction. The required resolution is decided before implementation continues.AI workflow with judgment
Requirement and spec decisionsA separate AI role decides on requirement and spec changes and never writes code.AI workflow with judgment, person approves

Freshness flags identify a review need. Tracked decisions and corrections resolve it.

Speaker notes

This is the custom framework shown on slide 4. IDs alone do not keep its documents true. Something has to notice when they stop agreeing.

The base is the unique ID. Every requirement, architecture decision, screen, spec, acceptance criterion and work item has one, and every document that relies on another names it by that ID. That makes the whole hierarchy searchable, by a deterministic tool or by an AI agent.

On top of the IDs sit deterministic tools. When a requirement or criterion is edited, everything that refers to it is flagged for review, the same idea as suspect links in Jama or Polarion. An impact tool shows everything connected to an item, in both directions. A coverage tool lists acceptance criteria without test citations. The closure check requires a passing test for each work-item completion condition. Neither alone is a complete requirement-verification report for the delivered build. Architecture decisions are enforced in code by rules, and each rule is tested so it cannot silently stop working.

Some steps need judgment, and those are AI workflows. When a delivered spec changes, the work that depends on it is reopened. A discrepancy requires a decision about whether code, the spec or both need correction. The approved resolution governs the next implementation step. Decisions about requirements and specs go to a separate AI role that never writes code, and a person approves them. The tools find the problems. AI workflows and people fix them.