Software factories: draft report

Draft report · October 2026

Software factories: what exists, what is missing

AI tools can write code, tests and design documents. Maintaining software for years also requires knowing what it should do, whether the delivered code does it, and which records need updating when something changes. This report compares how current products support that work. The reviewed evidence does not establish one product that covers the whole process.

prompt AIrun agentsrun agentic loopssoftware factoryYou steer the agentprompt by prompt.vibe codingOne-shot tasks withskills, workflows,custom agents.custom review agentRepeat one task untilit passes the checks.review and fix a PRuntil it is mergedSpecialised agents split,build, and verify workin parallel, for days.Factory Missions, Vercel AI SDK

An AI agent is software that uses an AI model to carry out a task, such as changing code and running tests. The picture shows four ways to use it. First, a person gives instructions one step at a time. Second, an agent completes a prepared task. Third, an agent repeats its work until it passes a check. Fourth, a software factory coordinates specialised agents that divide, build and verify work, potentially for hours or days. This report examines what that coordinated process must retain and check so the resulting software can be maintained and audited.

1. What a software factory has to do

An enterprise needs documentation that can be maintained for years, stays aligned with the system, and supports an evidence trail from the delivered code back to its requirements. Seven properties make those needs concrete. This comparison ends with the delivered build. Deployment and operation are outside its scope.

PropertyDefinitionWhy it matters
Lifecycle traceabilityMaintained relationships explain which requirements each design decision addresses, which code implements that design, and which tests verify the required behaviour.IDs and hyperlinks provide navigation. Their meaning and version must also be recorded so the connected information can support analysis and verification.
Versioned system designA complete, integrated description of the intended system for a selected version is available before implementation finishes. Proposed, approved but unimplemented, and implemented changes are distinguishable. Delivered code identifies the governing design version.The intended design may be ahead of the code. A reader must be able to see both the approved target and what a particular code version implements. Unmerged change deltas alone leave that picture fragmented.
Tracked drift resolutionA check identifies a possible disagreement and opens or updates an affected-area finding or task. Review decides whether code, documentation or both need correction. The finding is tracked through correction and recheck.A changed hash or suspect link signals a need to review. It does not establish an actual disagreement. An alert without tracked follow-up is partial support. Automatic document rewriting is not required because the correct resolution may need missing information or a design decision.
Change impact and alternativesMaintained relationships, constraints and version baselines identify affected requirements, designs, code and tests, and support evaluating alternatives against the same requirements. The assessment records trade-offs, assumptions and unanswered questions.Impact analysis is one use of the connected information. It should also help compare possible designs. Generic architecture conversation is insufficient; architecture modelling is claimed only where maintained models are actually analysed.
Requirement test resultsAt the end of implementation, executable tests check delivered behaviour against requirements or acceptance criteria. Results identify the tested code version and show passed, failed, missing and unverified requirements.This establishes which required behaviour the delivered implementation demonstrated. Checklist review, code coverage and links to test files do not establish that those tests ran and passed.
Delivery audit trailA retained record identifies the delivered build and changes, governing requirement and design versions, reviews, approvals and waivers. It references the applicable test results.This establishes what was delivered, under which agreed design and decisions. Test adequacy and requirement verification are assessed in the preceding criterion, not counted again here.
Process portabilityThe workflow definitions, orchestration and checks can continue running on different supported systems without the original hosted factory service.Exportable code and documents establish data retention, not portability of the factory. A licensed proprietary runtime can qualify when it can be deployed elsewhere. The supported execution environments and remaining dependencies must be stated.

For example, an approved transfer design may include a reversal flow that is not implemented yet. The complete target design should include that flow and mark its implementation state. The current delivered build must still identify the earlier design version it implements. Proposed alternatives remain distinguishable from both.

For each capability, distinguish what is available, what triggers it, what a failure prevents, and how findings are resolved. Structural scripts, executable behaviour tests, AI reviews and human decisions supply different evidence. A configured workflow can meet a criterion; fixed workflows receive no preference.

Process portability requires continued execution without the original hosted factory service. A local client or worker, exported files, or a server edition of only the non-agentic core does not pass that prerequisite. An unresolved essential dependency is marked ?, rather than given partial credit. A deployable licensed runtime can qualify; changing workflow engines is not required. Model inference, repository hosts and CI integrations are listed separately so readers can see what else must remain available.

2. What exists today

The table groups the reviewed products by their main role in software delivery, showing what each group provides and where additional tools or processes are needed.

TypeExamplesGood atMissing
Issue-to-PR agentsGitHub Copilot coding agent, Google Jules, Cognition Devin, Azure Boards → CopilotA ticket becomes a pull request; a person mergesA persistent specification is not required by the reviewed workflow
Spec workflowsAWS Kiro, GitHub Spec Kit, OpenSpec, BMAD Method, TesslRequirements and design before codeCompleteness, version alignment and enforcement vary
Requirements toolsJama Connect, Siemens Polarion; Azure DevOps + GitHub CopilotRequirements and test traceability; Copilot writes codeNo dedicated design step in the Azure Boards → Copilot handoff
Atlassian suiteConfluence, Jira, Rovo DevDocuments, work tracking and coding assistanceFull cross-layer reconciliation was not established
Agent orchestrationFactory.ai, Augment CosmosMany agents in parallel, AI reviewMaintained requirement/design model not established

Issue-to-PR agents

These take a ticket and open a pull request. A person reviews and merges it; GitHub Copilot coding agent cannot approve or merge its own pull request. Azure Boards can hand a work item to Copilot, which opens a linked draft pull request; a person starts it, the code must be on GitHub, and there is no design step. Vendor documentation: Azure Boards and Copilot

Spec workflows

Kiro writes requirements, design and tasks per feature, and those files stay behind as history. Spec Kit has a constitution and per-feature specs, with a documented living-spec option. OpenSpec keeps current specs per capability and merges each change into them. BMAD keeps a PRD, UX, architecture and specs current through its own workflows. Tessl links each spec to its code files and tests. Checks may be invoked directly or connected to supported automation. Their scope, tracking of findings and blocking enforcement must be assessed separately.

Requirements tools

Jama Connect and Siemens Polarion keep multi-level requirements, designs, risks and tests with a defined traceability model, baselines, suspect links, coverage reports and electronic signatures. Azure DevOps links work items and test cases, rolls automated test results up to requirements, and links work items to commits, builds and releases. Azure Boards can send a work item to GitHub Copilot, which writes code and opens a linked draft pull request. That handoff has no dedicated design stage. Jama and Polarion connect to development tools through integrations.

Atlassian suite

Confluence holds PRDs and decision pages, Jira holds the work, and Rovo Dev writes code from Jira items and reviews pull requests against Jira acceptance criteria. That review is an AI judgment. Test management comes from marketplace apps. Page and issue links do not establish a maintained requirement-to-design-to-code-to-test model. The reviewed sources do not establish a complete tracked reconciliation flow across those layers. Atlassian is retiring standalone Rovo Dev and moving its features into Jira. Vendor page: Rovo Dev licensing

Agent orchestration

Factory.ai Missions split a goal into features and milestones and run workers with validation. Factory’s Software Factory dashboard is in private preview. Augment Cosmos describes a living spec but documents no mechanism that keeps it current. Vercel’s internal factory for its AI SDK reports 25 to 35% of weekly merged pull requests, and a person merges every one. Vendor article: Vercel , Vendor page: Augment Cosmos

3. How they compare

The table compares eleven product configurations against the seven criteria above. Select a product name for its chapter, or a rating for the supporting explanation and sources.

Ratings 2026-10
ProductLifecycle traceabilityVersioned system designTracked drift resolutionChange impact and alternativesRequirement test resultsDelivery audit trailProcess portability
GitHub Copilot coding agent
Azure DevOps + Copilot
Atlassian suite (Confluence, Jira, Rovo Dev)
AWS Kiro
GitHub Spec Kit
OpenSpec
BMAD Method + Test Architect
Tessl
Jama Connect
Siemens Polarion
Factory.ai (Droid + QA)

Ratings describe documented support, not percentages. ? Unverified means evidence is insufficient. Not supported requires a documented restriction. Whether checks start automatically and whether failures block delivery are described separately. No overall product scores are calculated.

Corrections to a rating: contact@softwarefactoryreport.com

More filled bars mean broader documented support. Limited covers part of a criterion; Substantial covers most of it with a specific gap; Complete meets its full definition. Three pale bars mean Not supported, based on an established restriction. ? Unverified means the evidence is insufficient. These categories are not percentages. The definitions for each criterion explain what separates the categories.

4. Detailed product assessments

Select a product below to open its detailed assessment on the companion page. Each assessment describes how the product works and what records it keeps. The eleven rated products also have explanations and sources for their ratings, including how checks start, whether failures block delivery, how findings are resolved and which services or product editions are required. Additional products retain the earlier survey notes without new ratings.

Evidence reviewed through October 23, 2026. The latest audit examined all 77 ratings across eleven products. These are documentation findings, not product trials.

5. Method and limits

What separates limited, substantial and complete support

These anchors apply to the full definitions in section 1. They describe coverage, not whether a person or an event starts a check. Configuration and manual invocation do not lower a level by themselves. An essential unknown remains ? even when a neighboring component is portable or locally executable.

CriterionLimitedSubstantialComplete
Lifecycle traceabilitySome meaningful document or requirement/test relationships; code or design legs remain disconnected.Most lifecycle layers are connected; a maintained design/code relationship or version context remains unproven.Maintained, meaningful and version-aware relationships connect requirements, design, code and tests.
Versioned system designPersistent intended design for a feature or area; an integrated approved system target is missing.A maintained integrated target or controlled baseline exists; delivered-code/design binding or state distinctions remain incomplete.A complete selected-version target distinguishes proposed, approved-unimplemented and implemented states; delivered code identifies its governing design.
Tracked drift resolutionA scoped mismatch or freshness alert supports review; durable correction and recheck are incomplete.Affected-area tasks/findings and repeat review support correction; the complete retained reconciliation record remains incomplete.A finding is retained through a decision about code/docs/both, correction and recorded recheck for the affected versions.
Change impact and alternativesLinked impact input or constrained design planning exists, but impact and alternatives are not integrated.Scoped impact and constrained alternatives are both documented; affected tests or baseline-linked completeness remains incomplete.Maintained relationships and baselines support affected requirements/design/code/tests and recorded alternatives, constraints and trade-offs.
Requirement test resultsRequirement-to-test mapping and some executed behavior-test evidence exist, but their combined result record is incomplete.Requirement-linked behavior tests have executions/results; retained tested-code identity or missing/unverified visibility is incomplete.Executable requirement-linked results identify tested code and distinguish passed, failed, missing and unverified requirements.
Delivery audit trailReview, baseline or check evidence concerns a specific change; complete delivery/build association is missing.Build-linked change, test and review evidence exists; governing design or approval/waiver association remains incomplete.A delivered build identifies its changes, governing requirement/design versions, reviews, approvals/waivers and applicable result references.
Process portabilityThe named workflow and its essential checks continue without the original factory service in a narrowly supported alternative environment; major configuration or state-transfer limits remain.The named workflow and essential checks continue without the original factory service across supported environments; a bounded integration or state-transfer limit remains.The named workflow, orchestration and checks can continue on another supported system without the original hosted factory service; dependencies are explicit.

How to validate the claims in a product trial

The following checks have not been run. Use a small stock-reservation example with one requirement, a complete design baseline, two code revisions and executable tests. Add an approved but unimplemented expiry feature and keep a separate design alternative proposed. Record the exact edition, modules, integrations and model endpoints. Test both successful and failing paths.

CriterionTrial and evidence to retain
Lifecycle traceabilityTrace the stock requirement through its design decision, changed code and executable test, then back. Change a relationship and inspect whether its meaning, versions and broken links remain visible.
Versioned system designKeep delivered code on D1, approve a complete D2 with expiry still unimplemented, and leave D3 proposed. Confirm that the integrated target and each build’s governing version remain unambiguous.
Tracked drift resolutionIntroduce code that accepts excess stock and separately make a harmless document edit. Retain the affected-area finding, decision about code/docs/both, correction, recheck and closure history. A freshness alert alone is not semantic detection.
Change impact and alternativesChange the stock constraint. Compare an atomic database update and a serialized queue against the same requirements. Retain affected artifacts, baselines, constraints, assumptions, trade-offs and unresolved questions.
Requirement test resultsRun tests on code A, introduce a regression in B, retain an untested requirement and try reusing A’s results for B. Verify requirement mapping, tested-code identity and passed/failed/missing/unverified visibility.
Delivery audit trailIdentify a delivered build and follow its changes, governing documents, reviews and result references. Waive a missing test explicitly; the waiver must not turn an unverified requirement into a passed result.
Process portabilityRecreate the assessed process on another supported system and deny access to the original factory service. Allow only declared model/infrastructure dependencies. Replay work and checks; retain workflow definitions, required process state and logs identifying any essential step that stops.

For every trial, record who or what started the check, where it ran, what failure prevented, configured bypasses and whether the finding remained tracked to closure. Documentation can establish a supported configuration; these trials would demonstrate that configuration in use.

Conclusions

Appendix: research material

This appendix explains the research behind the report. It starts with studies of problems that can arise when AI agents develop software over many steps. The later tables preserve the earlier product surveys and the questions they asked. Those surveys used different criteria from the current comparison. For the current ratings, use sections 1–3 and the linked product assessments. Each finding links to the source cited during research; not every source was checked again for this revision. Products supported only by marketing claims (Warp Factories and an open-source “ai-factory”) are omitted.

A. What studies found about AI coding limits

These studies examine different tasks, models and measures of success. Their figures describe those particular tests. They do not predict the success rate of every AI coding task or establish a ranking of the products in this report.

B. Organisational responsibilities and approvals

These questions concern how an organisation runs its software delivery process. They are outside this report’s seven scored criteria. Who owns the current documents? Who can approve a changed requirement? Which checks only advise, and which block completion? How are exceptions, history and evidence retained and exported?

The research also considered how agents divide tasks, receive permissions, work in parallel and combine their changes. Cost and reliability figures below describe the studies and products at the time of research. Differences in models and tasks prevent a comparable effectiveness score.

C. Change specs and living documentation, per product

A change specification describes one proposed change. Living documentation is updated to describe the system as it should now work. This earlier survey asked which approach each product supports and what happens to the documents after a change is finished.

Product1. Spec unit2. After implementation3. Later change to same area4. Links to code/tests, drift detection5. Rating + one-line evidence
AWS Kiro (specs, steering)Per feature or bugfix: .kiro/specs/<name>/ with requirements.md, design.md, tasks.md. Steering files (product.md, tech.md, structure.md) are persistent context, not specs.Specs are committed to the repo as history for future developers and for Kiro to revisit. No merge into a capability spec.Docs describe asking Kiro to update the spec or editing it manually, but the default flow is a new spec per feature. No documented merge of deltas into a current-state spec.Tasks trace to requirement numbers. No documented spec-to-code or spec-to-test links. No drift check.Change-spec only. Docs call specs history to accumulate, not a current-state contract. Vendor documentation: https://kiro.dev/docs/specs.md
GitHub Spec KitPer feature: specs/<nnn-name>/ with spec.md, plan.md, tasks.md. memory/constitution.md is persistent principles.Feature directory stays in specs/. /speckit.converge checks completeness and appends tasks until Converged.Two documented options: flow-forward creates a new feature directory as history; living spec revises the same spec.md first, then regenerates plan.md and tasks.md and runs /speckit.analyze./speckit.analyze checks spec/plan/tasks consistency. /speckit.converge checks code against artifacts. Both are manual commands, not CI.Living spec (documented and shipped). Living-spec update loop is documented with commands. Source repository: https://github.com/github/spec-kit/blob/bba473c2/docs/guides/evolving-specs.md
OpenSpec (Fission AI)Per capability in openspec/specs/<domain>/spec.md as source of truth. Per change in openspec/changes/<name>/ with proposal.md, delta specs, design.md, tasks.md.openspec archive merges ADDED/MODIFIED/REMOVED deltas into openspec/specs/ and moves the change to changes/archive/.New change folder holds deltas against the existing capability spec. Archive applies them. Specs are never edited directly except through a change.Deltas reference requirements, not code files. openspec validate checks format, not code sync. No built-in drift detection.Living spec (documented and shipped). Archive merges deltas into the source of truth. Source repository: https://github.com/fission-ai/openspec/blob/main/docs/concepts.md
Tessl (Framework + Registry)Per component or capability: .spec.md with targets file globs and [@test] links. Registry also hosts usage specs per library version.Specs stay in the repo as long-term memory. Workflow says review specs, then build, then update specs with discovered requirements and re-verify.Workflow says review existing specs, then create or update specs before implementation. Specs are edited in place, not replaced per change.targets links spec to code files. [@test] links requirements to tests. Tile ships validate-specs.sh and check-spec-links.sh. No evidence of automatic background drift detection.Living spec (documented and shipped). Spec format with targets and test links plus validation scripts is published. Vendor documentation: https://tessl.io/registry/tessl-labs/spec-driven-development
Factory.ai (Droid Spec Mode)Per task: a plan produced in Spec Mode. Optional save to .factory/docs or IMPLEMENTATION_PLAN.md.Plan is approved, then Droid implements. Saved plan stays as a file. No merge into a capability spec.Large-feature guide says reference the master plan for each phase and mark phases complete. No update of a shared current-state spec.No spec-to-code or spec-to-test links documented. No drift check.Change-spec only. Spec Mode is read-only planning before edits, saved as a plan file. Vendor documentation: https://docs.factory.com/autonomy-and-safety/specification-mode
Augment CosmosPer feature: PRD plus tech spec authored by Project Builder as Cosmos files, refined in multiplayer sessions.PRD and tech spec persist as Cosmos files. Implementation proceeds via Ticket Manager and PR Author.Guides say the living spec is read from and written back to as Experts finish work and requirements change. No documented merge command or versioning rule.Guides claim Experts implement, verify, and ship against the same spec with Context Engine grounding. No documented link format or drift check.Living spec (claimed only). Living-spec behavior appears in guides only, with no enforcement docs. Vendor page: https://www.augmentcode.com/guides/spec-driven-development-ai-agents-explained
GitHub Copilot coding agentPer issue or prompt: a session with a task checklist and one pull request. Plan mode produces a chat plan, optionally saved as Markdown.Session ends after merge. Branch and PR persist. Plan text does not merge into a capability spec.Later work starts a new session from a new issue or prompt. Persistent files are copilot-instructions.md and AGENTS.md, which are instructions, not functional specs.Session reads repo context and runs tests. No persistent spec-to-code links. No spec drift detection.Change-spec only. One issue produces one branch and one PR, then the session ends. Vendor documentation: https://docs.github.com/en/copilot/concepts/coding-agent/coding-agent
Atlassian Rovo Dev + ConfluencePer feature: Confluence PRD or spec page (stories, acceptance criteria) plus Jira work item.Confluence page and Jira item persist as docs. Jira Coding Agent creates a draft PR from the work item.No documented flow for updating the same PRD page on later changes or merging deltas. Smart Buttons generate new PRD pages from page content.Jira key links work item to PR. No spec-to-code or spec-to-test link format. No drift detection between Confluence page and code.Change-spec only. Docs show PRD generation then code generation, with no spec maintenance loop. Vendor documentation: https://support.atlassian.com/rovo/docs/generate-code-from-a-work-item-in-jira/
Azure DevOps (work items, Test Plans, wiki)Per requirement: work items (Epic, Feature, User Story, PBI) plus wiki pages and Test Plans/Suites/Cases.Work items, wiki pages, and test artifacts persist with state and history. Branch, commit, PR, and build link to the work item.Existing work items and wiki pages are edited. Requirement-based suites link test cases to requirements. Automated tests associate to test cases, so results flow to requirement quality.Development control shows branches, commits, PRs, builds. Test links use Tested By/Tests. Traceability reports show gaps. Test failures signal divergence, not spec-text diffing.Living spec (documented and shipped). Work items plus linked tests and code stay live with traceability. Vendor documentation: https://learn.microsoft.com/en-us/azure/devops/cross-service/end-to-end-traceability?view=azure-devops
Jama ConnectPer item: requirements, test cases, risks organized in a tree with a traceability model. Baselines snapshot points in time.Items stay as the current record with versions and history. Baselines preserve history. Reuse with sync shares items across projects.Change management with impact analysis flags affected items. Suspect links mark downstream artifacts when a requirement changes. Reviews approve the change.Forward and backward traceability links persist. Coverage views show gaps. Suspect links detect divergence after edits.Living spec (documented and shipped). Suspect links and live traceability keep specs current. Vendor documentation: https://help.jamasoftware.com/ah/en/getting-to-know-jama-connect-features.html
Siemens PolarionPer requirement in LiveDocs: structured, versioned specification documents with work items and link roles (for example, test case verifies requirement).LiveDocs stay as the controlled spec. Collections freeze approved snapshots. Baselines and change history persist.Requirements are edited in place. Traceability links show affected derived requirements, test cases, and risks. Workflow conditions can block approval without required links.Multilevel traceability widget shows requirements, linked code resources, and test results. Coverage reports list missing links. No proof of automatic text drift detection.Living spec (documented and shipped). Versioned LiveDocs with enforced link roles and coverage. Vendor article: https://blogs.sw.siemens.com/polarion/accelerate-and-scale-early-validation-with-polarion/
BMAD MethodPer product: PRD plus SPEC.md, architecture spine, epics, and stories. Quick-spec handles small fixes.PRD and specs persist in planning artifacts. Stories implement slices. Sprint status tracks progress.PRD is updated through bmad-prd in Update mode. Rule: if a ticket needs an answer the PRD lacks, the answer goes into the PRD first and Sofia is re-run. Ticket Requirements win on conflict and Sofia surfaces it.FR and story IDs cross-reference PRD, SPEC, and epics. Sign-off gates are manual reviews. No documented automatic code or test drift check.Living spec (documented and shipped). PRD update mode and re-run rule are documented. Vendor documentation: https://docs.bmad-method.org/plan/plan-inside-an-organization/
Cucumber / SpecFlow + LivingDoc (extra 1)Per capability: Gherkin feature files with scenarios in business language.Feature files stay in the repo. Bindings tie them to code. LivingDoc renders them as readable docs with test results.Scenarios are edited when requirements change. Step definitions are updated to match. CI runs the executable specs.Step definitions link scenarios to code. LivingDoc links specs to results. Failing tests detect divergence.Living spec (documented and shipped). Executable specs fail when code diverges. Historical SpecFlow documentation (no longer available)
OpenSpec spec-as-source plugin, Tessl Registry (extra 2)Per capability: OpenSpec spec.md with targets frontmatter and Verified by [@test] per requirement.openspec archive still merges deltas. Plugin adds check-spec-links.sh, check-target-ownership.sh, build-spec-manifest.py, and CI workflow.Later change must update the spec first: a changed target without its spec fails the ownership check. spec-verify and work-review catch semantic drift.targets owns code files. [@test] owns test files. Checks 1-4 run in CI. Check 5 is agent semantic review.Living spec (documented and shipped). Ownership and link scripts with CI are published. Vendor documentation: https://tessl.io/registry/spec-driven-development/spec-as-source

D. Artifact hierarchy, per product

An artifact is a retained item such as a requirement, design document, code file or test. This survey asked which kinds each product keeps, how they relate to one another and how changes move between them.

Product1. Artifact types (file/record names)2. Persistence3. Hierarchy and links (derivation, IDs)4. Propagation on higher-level change5. Rating, evidence, source
AWS Kiro.kiro/specs/<name>/requirements.md (or bugfix.md), design.md, tasks.md; .kiro/steering/product.md, tech.md, structure.md, custom steering; AGENTS.md support.Steering persists and is included by default; specs persist in repo and are version-controlled; continuous refinement supported. Per-feature specs remain as history.Steering is baseline context. Workflow is requirements -> design -> tasks. Tasks map to requirements. EARS notation in requirements. Sync Files maps tasks to requirements.Manual/AI-invoked. Edit requirements.md, then Refine design.md to update design and tasks, then Sync Files in tasks.md. Must be invoked in spec session. No auto script check found.Rich. Three spec files plus three foundation steering files with defined order. Vendor documentation: https://kiro.dev/docs/specs/
GitHub Spec Kit.specify/memory/constitution.md; specs/<###>-<name>/spec.md; plan.md plus research.md, data-model.md, contracts/, quickstart.md; checklists; tasks.md.Three persistence models: flow-forward (completed directories kept as history), living spec (spec.md is contract), flow-back (accepted discoveries maintained). specs/ is not overwritten by template refresh.Constitution -> spec -> plan -> tasks -> implement -> converge. Tasks ordered per user story. Analyze reports coverage gaps with Requirement Key to task mapping. Constitution violations are CRITICAL.Read-only AI check. /speckit.analyze reports gaps across spec/plan/tasks without editing. User fixes at source and re-runs. /speckit.converge only appends tasks. Must be invoked.Rich. Constitution, spec, plan artifacts, checklist, tasks with cross-artifact analysis. Vendor documentation: https://github.github.io/spec-kit/quickstart.html
OpenSpecopenspec/changes/<name>/proposal.md, specs/<capability>/spec.md delta, design.md, tasks.md, .openspec.yaml; openspec/specs/<capability>/spec.md main truth; changes/archive/YYYY-MM-DD-<name>/ history.Main specs/ persists as source of truth. Per-change folder persists until archive, then moved to archive. Delta sync to main on archive after confirmation.Schema defines order: proposal first, specs and design in either order, tasks needs both, apply needs tasks. Proposal Capabilities section lists capability paths. Delta ops: ADDED, MODIFIED, REMOVED, RENAMED. Scenarios use WHEN/THEN.AI-invoked plus CLI rule check. openspec validate rejects bad deltas. /opsx:verify reports completeness. /opsx:archive warns on incomplete tasks or unsynced specs and offers sync. Must invoke commands.Rich. Four artifacts with requires links in schema.yaml. Vendor documentation: https://openspec.dev/docs/schemas/spec-driven
Tessl*.spec.md with YAML frontmatter (name, description, targets); inline API contract block; [@test] links to test files; tile skills/rules/docs/scripts (requirement-gathering, spec-writer, spec-verification, spec-before-code)..spec.md persists. Styleguide says keep specs synchronized with implementation. targets must exist. Test files persist alongside.Requirement -> approved spec -> code. Spec links to implementation via targets globs and to verification via inline [@test] next to each requirement. targets is required.Script check plus AI-invoked. validate-specs.sh checks frontmatter/structure. check-spec-links.sh checks targets and [@test] paths exist. spec-verification skill reviews sync. Must be invoked.Few. One spec file type linked to targets and tests. Vendor documentation: https://tessl.io/registry/tessl-labs/spec-driven-development/files/docs/spec-format.md
BMAD Methodprd-<slug>.md (+ addendum, .memlog.md), DESIGN.md + EXPERIENCE.md + ux-<slug>.md, architecture-<slug>.md spine, per-epic SPEC.md, tickets.toml + story-<slug>-plan.md + sprint-status.yaml, QA assessments/gates, PRFAQ/brief.PRD is owned and updated only via bmad-prd Update mode. Spine and UX via their skills. Tickets and joined plans persist. Finished work stays finished. Historical plans kept.PRD is source. UX and spine derive from PRD. Specs derive from PRD plus spine and UX by reference, not copy. Tickets derive from specs. Capability and ticket IDs stay stable.AI-invoked. Run bmad-prd Update, then update spine, then re-run bmad-spec which updates SPEC.md in place, then re-slice stories. Gates: PRD validate, spine review, breakdown approval, retrospective. Must invoke skills.Rich. PRD, UX, spine, SPEC, tickets/plans with stated derivation. Vendor documentation: https://docs.bmad-method.org/plan/plan-inside-an-organization/
Augment CosmosIntake (rough spec, ticket, Figma mockup, doc); approved design doc (committed as design PR); dependency-ordered tickets; PR Author PRs; pair/deep review reports; Slack pings.Design doc persists as record after merge. Design PR is committed. Tickets and PRs persist per project. No named local spec folder documented.Brief -> design -> tickets -> PR waves -> verification. Approved design is source of truth. Worker briefs contain unit plus relevant design sections plus design doc link.Agent-fleet check, human-gated. Review fleet verifies every draft against approved design before human review. Nothing implements until design approved. Must run Project Builder/Cosmos workflow. No local lint script found.Few. Brief, design doc, tickets/PRs with design-link checks. Vendor page: https://www.augmentcode.com/solutions/large-projects
Factory.ai (Droids)Spec Mode plan (saved Markdown to .factory/docs or ~/.factory/specs if enabled); AGENTS.md; DESIGN.md guidance; .factory/skills/; Jira/Linear/Slack tickets; PRs; Mission plans.Saved specs persist only if Save spec as Markdown enabled. AGENTS.md persists in git. Org/user memory persists across sessions. Plans survive compaction.Ticket/prompt -> Spec Mode research plan -> approval -> Normal Mode implementation -> PR. AGENTS.md/DESIGN.md are always-on governance. Spec references linked artifacts and files. No requirement ID scheme documented.Manual/AI-invoked. Approve Spec Mode plan via ExitSpecMode to implement. No evidence of auto-flagging lower artifacts when AGENTS.md changes. Must re-invoke Droid.Few. Spec plan file plus AGENTS.md/DESIGN.md plus tickets/PRs, links claimed only. Vendor documentation: https://docs.factory.ai/autonomy-and-safety/specification-mode
Atlassian Rovo Dev + Confluence + JiraConfluence PRD template (background, objectives, user stories, UX mockups, scope), RFC/decision pages, whiteboards; Jira Epic/Story/Task/Subtask (plus Product Discovery ideas); Rovo Dev code plan in Jira session; Bitbucket/GitHub PRs with reviews.Confluence pages and Jira items persist and version. Whiteboard stickies convert to live Jira issues. Rovo plans live in session/Jira panel and are refinable. No single spec filename; persistence is in Confluence/Jira records.Confluence PRD/RFC -> Jira Epic/Stories/Tasks breakdown -> Rovo Dev plan -> code/PR -> review against Jira acceptance criteria. Links: /jira embeds, Jira keys back to PRD, Teamwork Graph, design-to-epic mapping prompts.AI-invoked plus automation. Rovo drafts/updates PRD from Jira, creates linked epics/stories, updates Confluence and Jira links. Jira Automations can trigger Rovo Dev. Review loop is Specs -> Implementation -> Review. Must prompt or trigger automation.Rich. PRD/UX/architecture pages, Jira Epic/Story/Task, code plan, PR review with links. Vendor article: https://www.atlassian.com/blog/developer/rovo-dev-for-architecting-solutions
Azure DevOpsBoards: Epic -> Feature -> Requirement (User Story, Product Backlog Item, Requirement, Issue) -> Task, plus Bug; Test Plans -> Test Suites (static, requirement-based, query-based) -> Test Cases + Shared Steps/Parameters + runs/results; Repos commits/PRs/builds.Work items persist with history, states, area/iteration paths. Test plans/suites/cases persist per cycle. Test cases reused across plans without copy. No baseline object like Jama/Polarion.Parent-child links form Epic/Feature/Requirement/Task. Tested-By/Related links tie tests to requirements. Development control ties work to branches, commits, PRs, builds. Requirement-based suites auto-link cases to backlog items.Links plus manual review. Bugs created in test auto-link to requirements/builds. No evidence of automatic suspect flagging. Coverage via board cards, Analytics, Power BI, or extensions. AI test-plan management via MCP must be prompted. Rule checks are process workflow rules, not cross-artifact consistency.Rich. Six-plus work/test types with parent-child and tested-by links. Vendor documentation: https://learn.microsoft.com/en-us/azure/devops/boards/work-items/about-work-items
Jama ConnectItem types: stakeholder, system, software/hardware requirements, risks (PHA/FMEA), designs, test plans/cases/runs, defects, baselines/catalogs/branches, reviews, change requests; Traceability Information Model (TIM) defines allowed relations.Versioned items. Baselines at milestones. Reuse catalogs with sync (shared Global ID) or branch. Audit trail. Review Center approvals persist.User need -> system -> software/hardware -> design output -> test case/run/defect, plus risks. Forward and backward traceability. Live Trace Explorer scores quality, coverage, verification. Relationship rules enforce methodology.Automatic plus AI-invoked. Upstream change flags downstream links as suspect. Impact Analysis shows direct and indirect links. Engineer reviews and clears suspect. AI generates traceable test cases from requirements when invoked.Rich. Multi-level requirements plus risks, designs, tests, baselines with TIM and suspect links. Vendor documentation: https://help.jamasoftware.com/ah/en/getting-to-know-jama-connect-features/traceability-from-requirements-to-test.html
Siemens PolarionLiveDocs specification documents (requirements, design, risk, test specs) with paragraph-level IDs; Work Items (requirement, change, task, bug, test case, test record, risk); baselines/collections/branches; approvals/e-signatures; traceability matrices.Versioned work items. Baselines with comparison. Live-Branches with reuse/derive. Time machine history. Audit trails and signatures gate release.Stakeholder -> system -> software -> test chain (ASPICE, ISO 26262). Structural parent-child in LiveDocs. Typed links (relates_to, verifies, derived_from). Cross-project and cross-repo links.Automatic rule-based. Changed items mark links suspect (auto_suspect). Impact and traceability analysis before change. Workflow conditions can block Done until docs/tests attached. Must review and clear. No AI propagation step documented.Rich. LiveDocs at multiple levels plus work items, tests, baselines with link roles and suspect. Vendor page: https://www.siemens.com/en-us/products/polarion/requirements/
Cline Memory Bank (extra 1)memory-bank/projectbrief.md, productContext.md, systemPatterns.md, techContext.md, activeContext.md, progress.md; optional feature/API/test/deployment docs; .clinerules.All files persist in repo. activeContext.md changes most often. progress.md reviewed on resume. projectbrief.md is foundation and source for scope. Updated after milestones.projectbrief -> productContext/systemPatterns/techContext -> activeContext -> progress. Agent must read all files at start of every task. Large detail splits to linked docs.AI-invoked. Commands: follow custom instructions, initialize memory bank, update memory bank (reviews all files). No script consistency check found. Auto-Compact preserves context but still needs manual update.Rich. Six files with stated hierarchy diagram. Vendor documentation: https://docs.cline.bot/features/memory-bank
Task Master AI (extra 2)PRD file (.taskmaster/docs/prd.txt); tasks.json (id, title, description, status, dependencies, priority, details, testStrategy, subtasks); scripts/task-complexity-report.json; tags for contexts.tasks.json and complexity report persist in repo. PRD remains source. Dashboard syncs with local repo as truth. Updates via CLI mutate tasks.PRD -> tasks with stable IDs and dependencies arrays -> subtasks (parentId.subtaskId such as 5.2). testStrategy per task is verification contract. next selects by dependencies and status.CLI/AI-invoked. update, update-task, update-subtask, expand realign tasks. analyze-complexity plus complexity-report guide expansion. Must invoke commands. No auto propagation.Few. PRD, tasks with subtasks and dependencies, complexity report. Vendor documentation: https://docs.task-master.dev/capabilities/task-structure
Sourcegraph Amp (extra 3)AGENTS.md (project briefing, @-mentions to specs/docs, glob scoping); threads (T-..., synced to ampcode.com, private/team/public, fork/continue/compact/search); file changes/PRs.AGENTS.md persists in git. Threads persist server-side, searchable, shareable, resumable across devices. Compaction summarizes to save context.Governance (AGENTS.md) informs thread. Thread holds plan and execution context. @-mentions link specs/docs/threads. No defined requirement -> spec -> task derivation with IDs.Manual. Ask Amp to update AGENTS.md from thread. Editing a prior message reverts later file changes. No cross-artifact suspect or consistency check found. Must invoke thread actions.Few. AGENTS.md plus threads with mention links, no derivation chain. Vendor page: https://ampcode.com/manual?preview

E. First survey run: six properties

This survey used the six research questions below. These are the original survey criteria, rather than the seven properties used in the main comparison. Ratings: Yes, Partial, No, Unknown, Claimed only.

ProductRequirements through design to code and testsRules for document contentsChecks run by toolsChanges to an existing systemDivision of work between tools and AIHuman approval points
Microsoft Azure DevOps: Boards, Test Plans, MCPPartialPartialPartialPartialYesUnknown
GitHub Copilot coding agentPartialPartialPartialPartialPartialPartial
GitHub Spec KitPartialPartialPartialPartialYesYes
GitHub Agent HQPartialUnknownPartialPartialPartialPartial
Atlassian Rovo DevPartialPartialNoPartialPartialPartial
Atlassian Jira/Confluence Rovo agentsNoNoNoNoPartialPartial
Atlassian CompassNoNoNoPartialNoUnknown
AWS KiroPartialPartialPartialPartialYesPartial
Factory Droids and MissionsPartialPartialPartialPartialYesYes
Factory Software FactoryPartialUnknownPartialPartialPartialPartial
Vercel AI SDK factory (internal workflow; paper is analysis, not a product)YesPartialPartialPartialYesYes
Google JulesPartialPartialUnknownPartialPartialPartial
Google AntigravityPartialPartialPartialPartialYesPartial
Cognition DevinPartialPartialPartialPartialYesYes
Augment CosmosPartialPartialPartialPartialYesYes
TesslNoNoPartialNoYesPartial
GitLab Duo Agent PlatformPartialPartialPartialPartialPartialPartial
Modern Requirements Copilot4DevOpsClaimed onlyClaimed onlyUnknownClaimed onlyClaimed onlyUnknown

F. Second survey run: eight properties

This earlier survey used eight questions. Its table preserves the original shorthand: Y means yes, P partly, C claimed without supporting evidence, N no and U unknown. D1–D8 identify the questions below. These are not the seven criteria used in the current comparison.

  1. D1: Delivery steps. Does the workflow connect requirements, design, code and tests?
  2. D2: Document guidance. Does it define what the retained documents should contain?
  3. D3: Repeatable checks. Do tools check document structure, relationships and consistency using defined rules?
  4. D4: Later changes. Does it handle changes to an existing system and keep its documents current?
  5. D5: Tools and judgment. Does it distinguish checks performed by scripts from reviews that need AI or human judgment?
  6. D6: Approvals and evidence. Does it define who approves work and retain a record of those decisions?
  7. D7: Parallel work. Can agents work simultaneously, and how are their changes combined?
  8. D8: Long runs. What evidence is available about their cost and reliability?
ProductRequirements through design to code and testsRules for document contentsChecks run by toolsChanges to an existing systemDivision of work between tools and AIHuman approval and audit recordsParallel work and integrationCost and reliability
Azure DevOps (Boards, Test Plans, traceability)PPPPPYPP
GitHub Copilot coding agentPNPPPYPP
GitHub Spec KitPPPPYPPP
GitHub Agent HQ (mission control)PNNNPPPP
Atlassian Rovo Dev + Jira/Confluence agentsPNPPPYPP
Atlassian CompassNNPNPPNU
AWS KiroPYPPYPPP
Factory.ai (Droids, Missions)PPPPPYPP
Vercel AI SDK factoryPPPPYYPP
Google JulesPNPPPYPP
Google AntigravityPNNNPPPP
Cognition DevinPNPPPYPP
Augment CosmosPPPPYYPP
Tessl (specs, verifiers, registry)PPPPPPUP
Modern Requirements + Copilot4DevOps + Agents4DevOpsPPPPNYNU
Jama Connect (+ MCP)PYPPPYNU
Siemens Polarion (+ Polarion Copilot)NPPPPYNU
OpenAI internal Codex factory (Perf Factory, Sevbot)PNPPNPPP

G. Sources cited by the research runs

Product survey, first run (56)

Product survey, second run (68)

Change specs and living documentation (29)

Artifact hierarchy (34)

Fact-check (16)

Portability follow-up (20)

Check lifecycle, Luna (14)

Check lifecycle, Muse (0)

Tightened criteria, Luna Max (8)

Tightened criteria, Muse XHigh (30)

77-cell audit, Luna Max (3)

77-cell audit, Muse XHigh (9)

Audit reconciliation and validation protocol (3)