initial commit
This commit is contained in:
158
.agents/skills/bmad-testarch-ci/steps-c/step-01-preflight.md
Normal file
158
.agents/skills/bmad-testarch-ci/steps-c/step-01-preflight.md
Normal file
@@ -0,0 +1,158 @@
|
||||
---
|
||||
name: 'step-01-preflight'
|
||||
description: 'Verify prerequisites and detect CI platform'
|
||||
nextStepFile: './step-02-generate-pipeline.md'
|
||||
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
||||
---
|
||||
|
||||
# Step 1: Preflight Checks
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Verify CI prerequisites and determine target CI platform.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- 🚫 Halt if requirements fail
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Verify Git Repository
|
||||
|
||||
- `.git/` exists
|
||||
- Remote configured (if available)
|
||||
|
||||
If missing: **HALT** with "Git repository required for CI/CD setup."
|
||||
|
||||
---
|
||||
|
||||
## 2. Detect Test Stack Type
|
||||
|
||||
Determine the project's test stack type (`test_stack_type`) using the following algorithm:
|
||||
|
||||
1. If `test_stack_type` is explicitly set in config (not `"auto"`), use that value.
|
||||
2. Otherwise, auto-detect by scanning project manifests:
|
||||
- **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `vite.config.*`, `next.config.*`, `src/components/`, `src/pages/`, `src/app/`
|
||||
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`, `jest.config.*`, `vitest.config.*`, `src/routes/`, `src/controllers/`, `src/api/`, `Dockerfile`, `serverless.yml`
|
||||
- **Both present** → `fullstack`
|
||||
- **Only frontend** → `frontend`
|
||||
- **Only backend** → `backend`
|
||||
- **Cannot determine** → default to `fullstack` and note assumption
|
||||
|
||||
Record detected `test_stack_type` in step output.
|
||||
|
||||
---
|
||||
|
||||
## 3. Verify Test Framework
|
||||
|
||||
- Check for framework configuration based on detected stack:
|
||||
- **Frontend/Fullstack**: `playwright.config.*` or `cypress.config.*` exists
|
||||
- **Backend (Node.js)**: `jest.config.*` or `vitest.config.*` or test scripts in `package.json`
|
||||
- **Backend (Python)**: `pyproject.toml` with `[tool.pytest]` or `pytest.ini` or `setup.cfg` with pytest config
|
||||
- **Backend (Java/Kotlin)**: `pom.xml` with surefire/failsafe plugins or `build.gradle` with test task
|
||||
- **Backend (Go)**: `*_test.go` files present (Go convention — no config file needed)
|
||||
- **Backend (C#/.NET)**: `*.csproj` with xUnit/NUnit/MSTest references
|
||||
- **Backend (Ruby)**: `Gemfile` with rspec or `.rspec` config file
|
||||
- If `test_framework` is `"auto"`, detect from config files and project manifests found
|
||||
- Verify test dependencies are installed (language-appropriate package manager)
|
||||
|
||||
If missing: **HALT** with "Run `framework` workflow first."
|
||||
|
||||
---
|
||||
|
||||
## 4. Ensure Tests Pass Locally
|
||||
|
||||
- Run the main test command based on detected stack and framework:
|
||||
- **Node.js**: `npm test` or `npm run test:e2e`
|
||||
- **Python**: `pytest` or `python -m pytest`
|
||||
- **Java**: `mvn test` or `gradle test`
|
||||
- **Go**: `go test ./...`
|
||||
- **C#/.NET**: `dotnet test`
|
||||
- **Ruby**: `bundle exec rspec`
|
||||
- If failing: **HALT** and request fixes before CI setup
|
||||
|
||||
---
|
||||
|
||||
## 5. Detect CI Platform
|
||||
|
||||
- If `ci_platform` is explicitly set in config (not `"auto"`), use that value.
|
||||
- Otherwise, scan for existing CI configuration files:
|
||||
- `.github/workflows/*.yml` → `github-actions`
|
||||
- `.gitlab-ci.yml` → `gitlab-ci`
|
||||
- `Jenkinsfile` → `jenkins`
|
||||
- `azure-pipelines.yml` → `azure-devops`
|
||||
- `.harness/*.yaml` → `harness`
|
||||
- `.circleci/config.yml` → `circle-ci`
|
||||
- If found, ask whether to update or replace
|
||||
- If not found, infer from git remote (github.com → `github-actions`, gitlab.com → `gitlab-ci`)
|
||||
- If still unresolved, default to `github-actions`
|
||||
|
||||
Record detected `ci_platform` in step output.
|
||||
|
||||
---
|
||||
|
||||
## 6. Read Environment Context
|
||||
|
||||
- Read environment context based on detected stack:
|
||||
- **Node.js**: Read `.nvmrc` if present (default to Node 24+ LTS if missing); read `package.json` for dependency caching strategy
|
||||
- **Python**: Read `.python-version` or `pyproject.toml` for Python version; note `pip`/`poetry`/`pipenv` for caching
|
||||
- **Java**: Read `pom.xml`/`build.gradle` for Java version; note Maven/Gradle for caching
|
||||
- **Go**: Read `go.mod` for Go version; note Go module cache path
|
||||
- **C#/.NET**: Read `*.csproj`/`global.json` for .NET SDK version; note NuGet cache
|
||||
- **Ruby**: Read `.ruby-version` or `Gemfile` for Ruby version; note Bundler cache
|
||||
|
||||
---
|
||||
|
||||
### 7. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-01-preflight']
|
||||
lastStep: 'step-01-preflight'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-01-preflight'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-01-preflight'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section of the document.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
110
.agents/skills/bmad-testarch-ci/steps-c/step-01b-resume.md
Normal file
110
.agents/skills/bmad-testarch-ci/steps-c/step-01b-resume.md
Normal file
@@ -0,0 +1,110 @@
|
||||
---
|
||||
name: 'step-01b-resume'
|
||||
description: 'Resume interrupted workflow from last completed step'
|
||||
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
||||
---
|
||||
|
||||
# Step 1b: Resume Workflow
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Resume an interrupted workflow by loading the existing progress document, displaying progress, verifying previously created artifacts, and routing to the next incomplete step.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: Output document with progress frontmatter
|
||||
- Focus: Load progress and route to next step
|
||||
- Limits: Do not re-execute completed steps
|
||||
- Dependencies: Output document must exist from a previous run
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
### 1. Load Output Document
|
||||
|
||||
Read `{outputFile}` and parse YAML frontmatter for:
|
||||
|
||||
- `stepsCompleted` — array of completed step names
|
||||
- `lastStep` — last completed step name
|
||||
- `lastSaved` — timestamp of last save
|
||||
|
||||
**If `{outputFile}` does not exist**, display:
|
||||
|
||||
"⚠️ **No previous progress found.** There is no output document to resume from. Please use **[C] Create** to start a fresh workflow run."
|
||||
|
||||
**THEN:** Halt. Do not proceed.
|
||||
|
||||
---
|
||||
|
||||
### 2. Verify Previously Created Artifacts
|
||||
|
||||
Since this is a file-creation workflow, verify that artifacts from completed steps still exist on disk:
|
||||
|
||||
- If `step-02-generate-pipeline` is in `stepsCompleted`, check that the pipeline config file exists (e.g., `.github/workflows/test.yml` or equivalent)
|
||||
- If any expected artifact is missing, warn the user and suggest re-running from the step that creates it
|
||||
|
||||
---
|
||||
|
||||
### 3. Display Progress Dashboard
|
||||
|
||||
Display:
|
||||
|
||||
"📋 **Workflow Resume — CI/CD Pipeline Setup**
|
||||
|
||||
**Last saved:** {lastSaved}
|
||||
**Steps completed:** {stepsCompleted.length} of 4
|
||||
|
||||
1. Preflight Checks (step-01-preflight) — {✅ if in stepsCompleted, ⬜ otherwise}
|
||||
2. Generate Pipeline (step-02-generate-pipeline) — {✅ if in stepsCompleted, ⬜ otherwise}
|
||||
3. Configure Quality Gates (step-03-configure-quality-gates) — {✅ if in stepsCompleted, ⬜ otherwise}
|
||||
4. Validate & Summary (step-04-validate-and-summary) — {✅ if in stepsCompleted, ⬜ otherwise}"
|
||||
|
||||
---
|
||||
|
||||
### 4. Route to Next Step
|
||||
|
||||
Based on `lastStep`, load the next incomplete step:
|
||||
|
||||
- `'step-01-preflight'` → Load `./step-02-generate-pipeline.md`
|
||||
- `'step-02-generate-pipeline'` → Load `./step-03-configure-quality-gates.md`
|
||||
- `'step-03-configure-quality-gates'` → Load `./step-04-validate-and-summary.md`
|
||||
- `'step-04-validate-and-summary'` → **Workflow already complete.** Display: "✅ **All steps completed.** Use **[V] Validate** to review outputs or **[E] Edit** to make revisions." Then halt.
|
||||
|
||||
**If `lastStep` does not match any value above**, display: "⚠️ **Unknown progress state** (`lastStep`: {lastStep}). Please use **[C] Create** to start fresh." Then halt.
|
||||
|
||||
**Otherwise**, load the identified step file, read completely, and execute.
|
||||
|
||||
The existing content in `{outputFile}` provides context from previously completed steps. Use it as reference for remaining steps.
|
||||
|
||||
---
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Output document loaded and parsed correctly
|
||||
- Previously created artifacts verified
|
||||
- Progress dashboard displayed accurately
|
||||
- Routed to correct next step
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Not loading output document
|
||||
- Incorrect progress display
|
||||
- Routing to wrong step
|
||||
- Re-executing completed steps
|
||||
|
||||
**Master Rule:** Resume MUST route to the exact next incomplete step. Never re-execute completed steps.
|
||||
@@ -0,0 +1,279 @@
|
||||
---
|
||||
name: 'step-02-generate-pipeline'
|
||||
description: 'Generate CI pipeline configuration with adaptive orchestration (agent-team, subagent, or sequential)'
|
||||
nextStepFile: './step-03-configure-quality-gates.md'
|
||||
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
||||
---
|
||||
|
||||
# Step 2: Generate CI Pipeline
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Create platform-specific CI configuration with test execution, sharding, burn-in, and artifacts.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
- ✅ Resolve execution mode from explicit user request first, then config
|
||||
- ✅ Apply fallback rules deterministically when requested mode is unsupported
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 0. Resolve Execution Mode (User Override First)
|
||||
|
||||
```javascript
|
||||
const orchestrationContext = {
|
||||
config: {
|
||||
execution_mode: config.tea_execution_mode || 'auto', // "auto" | "subagent" | "agent-team" | "sequential"
|
||||
capability_probe: config.tea_capability_probe !== false, // true by default
|
||||
},
|
||||
timestamp: new Date().toISOString().replace(/[:.]/g, '-'),
|
||||
};
|
||||
|
||||
const normalizeUserExecutionMode = (mode) => {
|
||||
if (typeof mode !== 'string') return null;
|
||||
const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
|
||||
|
||||
if (normalized === 'auto') return 'auto';
|
||||
if (normalized === 'sequential') return 'sequential';
|
||||
if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
|
||||
return 'subagent';
|
||||
}
|
||||
if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
|
||||
return 'agent-team';
|
||||
}
|
||||
|
||||
return null;
|
||||
};
|
||||
|
||||
const normalizeConfigExecutionMode = (mode) => {
|
||||
if (mode === 'subagent') return 'subagent';
|
||||
if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
|
||||
return mode;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
// Explicit user instruction in the active run takes priority over config.
|
||||
const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
|
||||
|
||||
const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(orchestrationContext.config.execution_mode) || 'auto';
|
||||
const probeEnabled = orchestrationContext.config.capability_probe;
|
||||
|
||||
const supports = { subagent: false, agentTeam: false };
|
||||
if (probeEnabled) {
|
||||
supports.subagent = runtime.canLaunchSubagents?.() === true;
|
||||
supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
|
||||
}
|
||||
|
||||
let resolvedMode = requestedMode;
|
||||
if (requestedMode === 'auto') {
|
||||
if (supports.agentTeam) resolvedMode = 'agent-team';
|
||||
else if (supports.subagent) resolvedMode = 'subagent';
|
||||
else resolvedMode = 'sequential';
|
||||
} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
|
||||
resolvedMode = supports.subagent ? 'subagent' : 'sequential';
|
||||
} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
|
||||
resolvedMode = 'sequential';
|
||||
}
|
||||
```
|
||||
|
||||
Resolution precedence:
|
||||
|
||||
1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
|
||||
2. `tea_execution_mode` from config
|
||||
3. Runtime capability fallback (when probing enabled)
|
||||
|
||||
## 1. Resolve Output Path and Select Template
|
||||
|
||||
Determine the pipeline output file path based on the detected `ci_platform`:
|
||||
|
||||
| CI Platform | Output Path | Template File |
|
||||
| ---------------- | ------------------------------------------- | ----------------------------------------------- |
|
||||
| `github-actions` | `{project-root}/.github/workflows/test.yml` | `./github-actions-template.yaml` |
|
||||
| `gitlab-ci` | `{project-root}/.gitlab-ci.yml` | `./gitlab-ci-template.yaml` |
|
||||
| `jenkins` | `{project-root}/Jenkinsfile` | `./jenkins-pipeline-template.groovy` |
|
||||
| `azure-devops` | `{project-root}/azure-pipelines.yml` | `./azure-pipelines-template.yaml` |
|
||||
| `harness` | `{project-root}/.harness/pipeline.yaml` | `./harness-pipeline-template.yaml` |
|
||||
| `circle-ci` | `{project-root}/.circleci/config.yml` | _(no template; generate from first principles)_ |
|
||||
|
||||
Use templates from `./` when available. Adapt the template to the project's `test_stack_type` and `test_framework`.
|
||||
|
||||
---
|
||||
|
||||
## Security: Script Injection Prevention
|
||||
|
||||
> **CRITICAL:** Treat `${{ inputs.* }}` and the entire `${{ github.event.* }}` namespace as unsafe by default. ALWAYS route them through `env:` intermediaries and reference as double-quoted `"$ENV_VAR"` in `run:` blocks. NEVER interpolate them directly.
|
||||
|
||||
When the generated pipeline is extended into reusable workflows (`on: workflow_call`), manual dispatch (`on: workflow_dispatch`), or composite actions, these values become user-controllable and can inject arbitrary shell commands.
|
||||
|
||||
**Two rules for generated `run:` blocks:**
|
||||
|
||||
1. **No direct interpolation** — pass unsafe contexts through `env:`, reference as `"$ENV_VAR"`
|
||||
2. **Inputs must be DATA, not COMMANDS** — never accept command-shaped inputs (e.g., `inputs.install-command`) that get executed as shell code. Even through `env:`, running `$CMD` where CMD comes from an input is still command injection. Use fixed commands and pass inputs only as arguments.
|
||||
|
||||
```yaml
|
||||
# ✅ SAFE — input is DATA interpolated into a fixed command
|
||||
- name: Run tests
|
||||
env:
|
||||
TEST_GREP: ${{ inputs.test-grep }}
|
||||
run: |
|
||||
# Security: inputs passed through env: to prevent script injection
|
||||
npx playwright test --grep "$TEST_GREP"
|
||||
|
||||
# ❌ NEVER — direct GitHub expression injection
|
||||
- name: Run tests
|
||||
run: |
|
||||
npx playwright test --grep "${{ inputs.test-grep }}"
|
||||
|
||||
# ❌ NEVER — executing input-derived env var as a command
|
||||
- name: Install
|
||||
env:
|
||||
CMD: ${{ inputs.install-command }}
|
||||
run: $CMD
|
||||
```
|
||||
|
||||
Include a `# Security: inputs passed through env: to prevent script injection` comment in generated YAML wherever this pattern is applied.
|
||||
|
||||
**Safe contexts** (do NOT need `env:` intermediaries): `${{ steps.*.outputs.* }}`, `${{ matrix.* }}`, `${{ runner.os }}`, `${{ github.sha }}`, `${{ github.ref }}`, `${{ secrets.* }}`, `${{ env.* }}`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Pipeline Stages
|
||||
|
||||
Include stages:
|
||||
|
||||
- lint
|
||||
- test (parallel shards)
|
||||
- contract-test (if `tea_use_pactjs_utils` enabled)
|
||||
- burn-in (flaky detection)
|
||||
- report (aggregate + publish)
|
||||
|
||||
---
|
||||
|
||||
## 3. Test Execution
|
||||
|
||||
- Parallel sharding enabled
|
||||
- CI retries configured
|
||||
- Capture artifacts (HTML report, JUnit XML, traces/videos on failure)
|
||||
- Cache dependencies (language-appropriate: node_modules, .venv, .m2, go module cache, NuGet, bundler)
|
||||
|
||||
Write the selected pipeline configuration to the resolved output path from step 1. Adjust test commands based on `test_stack_type` and `test_framework`:
|
||||
|
||||
- **Frontend/Fullstack**: Include browser install, E2E/component test commands, Playwright/Cypress artifacts
|
||||
- **Backend (Node.js)**: Use `npm test` or framework-specific commands (`vitest`, `jest`), skip browser install
|
||||
- **Backend (Python)**: Use `pytest` with coverage (`pytest --cov`), install via `pip install -r requirements.txt` or `poetry install`
|
||||
- **Backend (Java/Kotlin)**: Use `mvn test` or `gradle test`, cache `.m2/repository` or `.gradle/caches`
|
||||
- **Backend (Go)**: Use `go test ./...` with coverage (`-coverprofile`), cache Go modules
|
||||
- **Backend (C#/.NET)**: Use `dotnet test` with coverage, restore NuGet packages
|
||||
- **Backend (Ruby)**: Use `bundle exec rspec` with coverage, cache `vendor/bundle`
|
||||
|
||||
### Contract Testing Pipeline (if `tea_use_pactjs_utils` enabled)
|
||||
|
||||
When `tea_use_pactjs_utils` is enabled, add a `contract-test` stage after `test`:
|
||||
|
||||
**Required env block** (add to the generated pipeline):
|
||||
|
||||
```yaml
|
||||
env:
|
||||
PACT_BROKER_BASE_URL: ${{ secrets.PACT_BROKER_BASE_URL }}
|
||||
PACT_BROKER_TOKEN: ${{ secrets.PACT_BROKER_TOKEN }}
|
||||
GITHUB_SHA: ${{ github.sha }} # auto-set by GitHub Actions
|
||||
GITHUB_BRANCH: ${{ github.head_ref || github.ref_name }} # NOT auto-set — must be defined explicitly
|
||||
```
|
||||
|
||||
> **Note:** `GITHUB_SHA` is auto-set by GitHub Actions, but `GITHUB_BRANCH` is **not** — it must be derived from `github.head_ref` (for PRs) or `github.ref_name` (for pushes). The pactjs-utils library reads both from `process.env`.
|
||||
|
||||
1. **Consumer test + publish**: Run consumer contract tests, then publish pacts to broker
|
||||
- `npm run test:pact:consumer`
|
||||
- `npm run publish:pact`
|
||||
- Only publish on PR and main branch pushes
|
||||
|
||||
2. **Provider verification**: Run provider verification against published pacts
|
||||
- `npm run test:pact:provider:remote:contract`
|
||||
- `buildVerifierOptions` auto-reads `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`, `GITHUB_SHA`, `GITHUB_BRANCH`
|
||||
- Verification results published to broker when `CI=true`
|
||||
|
||||
3. **Can-I-Deploy gate**: Block deployment if contracts are incompatible
|
||||
- `npm run can:i:deploy:provider`
|
||||
- Ensure the script adds `--retry-while-unknown 6 --retry-interval 10` for async verification
|
||||
|
||||
4. **Webhook job**: Add `repository_dispatch` trigger for `pact_changed` event
|
||||
- Provider verification runs when consumers publish new pacts
|
||||
- Ensures compatibility is checked on both consumer and provider changes
|
||||
|
||||
5. **Breaking change handling**: When `PACT_BREAKING_CHANGE=true` env var is set:
|
||||
- Provider test passes `includeMainAndDeployed: false` to `buildVerifierOptions` — verifies only matching branch
|
||||
- Coordinate with consumer team before removing the flag
|
||||
|
||||
6. **Record deployment**: After successful deployment, record version in broker
|
||||
- `npm run record:provider:deployment --env=production`
|
||||
|
||||
Required CI secrets: `PACT_BROKER_BASE_URL`, `PACT_BROKER_TOKEN`
|
||||
|
||||
**If `tea_pact_mcp` is `"mcp"`:** Reference the SmartBear MCP `Can I Deploy` and `Matrix` tools for pipeline guidance in `pact-mcp.md`.
|
||||
|
||||
---
|
||||
|
||||
### 4. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-02-generate-pipeline']
|
||||
lastStep: 'step-02-generate-pipeline'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-02-generate-pipeline'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-02-generate-pipeline'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section of the document.
|
||||
|
||||
### 5. Orchestration Notes for This Step
|
||||
|
||||
For this step, treat these work units as parallelizable when `resolvedMode` is `agent-team` or `subagent`:
|
||||
|
||||
- Worker A: resolve platform path/template and produce base pipeline skeleton (section 1)
|
||||
- Worker B: construct stage definitions and test execution blocks (sections 2-3)
|
||||
- Worker C: contract-testing block (only when `tea_use_pactjs_utils` is true)
|
||||
|
||||
If `resolvedMode` is `sequential`, execute sections 1→4 in order.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
@@ -0,0 +1,135 @@
|
||||
---
|
||||
name: 'step-03-configure-quality-gates'
|
||||
description: 'Configure burn-in, quality gates, and notifications'
|
||||
nextStepFile: './step-04-validate-and-summary.md'
|
||||
knowledgeIndex: '{project-root}/_bmad/tea/testarch/tea-index.csv'
|
||||
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
||||
---
|
||||
|
||||
# Step 3: Quality Gates & Notifications
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Configure burn-in loops, quality thresholds, and notification hooks.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Burn-In Configuration
|
||||
|
||||
Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
|
||||
|
||||
- Run N-iteration burn-in for flaky detection
|
||||
- Gate promotion based on burn-in stability
|
||||
|
||||
**Stack-conditional burn-in:**
|
||||
|
||||
- **Frontend or Fullstack** (`test_stack_type` is `frontend` or `fullstack`): Enable burn-in by default. Burn-in targets UI flakiness (race conditions, selector instability, timing issues).
|
||||
- **Backend only** (`test_stack_type` is `backend`): Skip burn-in by default. Backend tests (unit, integration, API) are deterministic and rarely exhibit UI-related flakiness. If the user explicitly requests burn-in for backend, honor that override.
|
||||
|
||||
**Security: Script injection prevention for reusable burn-in workflows:**
|
||||
|
||||
When burn-in is extracted into a reusable workflow (`on: workflow_call`), all `${{ inputs.* }}` values MUST be passed through `env:` intermediaries and referenced as quoted `"$ENV_VAR"`. Never interpolate them directly.
|
||||
|
||||
**Inputs must be DATA, not COMMANDS.** Do not accept command-shaped inputs (e.g., `inputs.install-command`, `inputs.test-command`) that get executed as shell code — even through `env:`, running `$CMD` is still command injection. Use fixed commands (e.g., `npm ci`, `npx playwright test`) and pass inputs only as data arguments.
|
||||
|
||||
```yaml
|
||||
# ✅ SAFE — fixed commands with data-only inputs
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
- name: Run burn-in loop
|
||||
env:
|
||||
TEST_GREP: ${{ inputs.test-grep }}
|
||||
BURN_IN_COUNT: ${{ inputs.burn-in-count }}
|
||||
BASE_REF: ${{ inputs.base-ref }}
|
||||
run: |
|
||||
# Security: inputs passed through env: to prevent script injection
|
||||
for i in $(seq 1 "$BURN_IN_COUNT"); do
|
||||
echo "Burn-in iteration $i/$BURN_IN_COUNT"
|
||||
npx playwright test --grep "$TEST_GREP" || exit 1
|
||||
done
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Quality Gates
|
||||
|
||||
Define:
|
||||
|
||||
- Minimum pass rates (P0 = 100%, P1 ≥ 95%)
|
||||
- Fail CI on critical test failures
|
||||
- Optional: require traceability or nfr-assess output before release
|
||||
|
||||
**Contract testing gate** (if `tea_use_pactjs_utils` is enabled):
|
||||
|
||||
- **can-i-deploy must pass** before any deployment to staging or production
|
||||
- Block the deployment pipeline if contract verification fails
|
||||
- Treat consumer pact publishing failures as CI failures (contracts must stay up-to-date)
|
||||
- Provider verification must pass for all consumer pacts before merge
|
||||
|
||||
---
|
||||
|
||||
## 3. Notifications
|
||||
|
||||
Configure:
|
||||
|
||||
- Failure notifications (Slack/email)
|
||||
- Artifact links
|
||||
|
||||
---
|
||||
|
||||
### 4. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-03-configure-quality-gates']
|
||||
lastStep: 'step-03-configure-quality-gates'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-03-configure-quality-gates'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-03-configure-quality-gates'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section of the document.
|
||||
|
||||
Load next step: `{nextStepFile}`
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
@@ -0,0 +1,92 @@
|
||||
---
|
||||
name: 'step-04-validate-and-summary'
|
||||
description: 'Validate pipeline and summarize'
|
||||
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
||||
---
|
||||
|
||||
# Step 4: Validate & Summarize
|
||||
|
||||
## STEP GOAL
|
||||
|
||||
Validate CI configuration and report completion details.
|
||||
|
||||
## MANDATORY EXECUTION RULES
|
||||
|
||||
- 📖 Read the entire step file before acting
|
||||
- ✅ Speak in `{communication_language}`
|
||||
|
||||
---
|
||||
|
||||
## EXECUTION PROTOCOLS:
|
||||
|
||||
- 🎯 Follow the MANDATORY SEQUENCE exactly
|
||||
- 💾 Record outputs before proceeding
|
||||
- 📖 Load the next step only when instructed
|
||||
|
||||
## CONTEXT BOUNDARIES:
|
||||
|
||||
- Available context: config, loaded artifacts, and knowledge fragments
|
||||
- Focus: this step's goal only
|
||||
- Limits: do not execute future steps
|
||||
- Dependencies: prior steps' outputs (if any)
|
||||
|
||||
## MANDATORY SEQUENCE
|
||||
|
||||
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
||||
|
||||
## 1. Validation
|
||||
|
||||
Validate against `checklist.md`:
|
||||
|
||||
- Config file created
|
||||
- Stages and sharding configured
|
||||
- Burn-in and artifacts enabled
|
||||
- Secrets/variables documented
|
||||
|
||||
Fix gaps before completion.
|
||||
|
||||
---
|
||||
|
||||
## 2. Completion Summary
|
||||
|
||||
Report:
|
||||
|
||||
- CI platform and config path
|
||||
- Key stages enabled
|
||||
- Artifacts and notifications
|
||||
- Next steps (set secrets, run pipeline)
|
||||
|
||||
---
|
||||
|
||||
### 3. Save Progress
|
||||
|
||||
**Save this step's accumulated work to `{outputFile}`.**
|
||||
|
||||
- **If `{outputFile}` does not exist** (first save), create it with YAML frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
stepsCompleted: ['step-04-validate-and-summary']
|
||||
lastStep: 'step-04-validate-and-summary'
|
||||
lastSaved: '{date}'
|
||||
---
|
||||
```
|
||||
|
||||
Then write this step's output below the frontmatter.
|
||||
|
||||
- **If `{outputFile}` already exists**, update:
|
||||
- Add `'step-04-validate-and-summary'` to `stepsCompleted` array (only if not already present)
|
||||
- Set `lastStep: 'step-04-validate-and-summary'`
|
||||
- Set `lastSaved: '{date}'`
|
||||
- Append this step's output to the appropriate section of the document.
|
||||
|
||||
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
||||
|
||||
### ✅ SUCCESS:
|
||||
|
||||
- Step completed in full with required outputs
|
||||
|
||||
### ❌ SYSTEM FAILURE:
|
||||
|
||||
- Skipped sequence steps or missing outputs
|
||||
**Master Rule:** Skipping steps is FORBIDDEN.
|
||||
Reference in New Issue
Block a user