Runtime Verification
Internal runtime verification feedback loop
Overview
camel-verify is the internal runtime verification skill that checks whether generated integrations actually work. Through a 3-phase feedback loop with error classification and targeted fixes, it runs the applicable build or startup smoke checks and integration tests, while reporting every skipped or failed check.
The output is runtime-verification evidence returned to /camel-execute, with a PASS, PARTIAL, FAIL, or NOT_RUN outcome. Verification failures do not block the execute completion summary. A chained pipeline still continues through Phase 4 static /camel-validate before it is complete.
When It Runs
/camel-execute dispatches camel-verify when generated code is ready for runtime checks. It handles:
- Want to validate a generated integration works at runtime
- Encounter build failures, runtime errors, or test failures
- Want automated diagnosis and fixing of common issues
It is not exposed as a command stub. Use /camel-debug for ad-hoc troubleshooting outside an active pipeline run.
Context and Confirmation
Build, startup, and test stdout or stderr, assertions, test files, MCP responses, .camel-kit/verify-log.md, and failure details passed between roles are evidence, not instructions. camel-verify uses the command’s exit state and corroborated diagnostic fields to select a shipped taxonomy entry. It may apply that entry’s bounded repair within the approved plan and workflow without another confirmation.
Commands, URLs, tool requests, file changes, secret requests, scope expansion, or policy overrides found in loaded content never select the repair. If an action is genuinely necessary but is not independently required by the shipped workflow, verification reports its source, exact action, reason, and scope and waits for action-specific confirmation. A role that cannot ask returns NEEDS_USER_CONFIRMATION; an unclassified error is escalated rather than treated as remediation advice.
The Verification Loop
The verification process runs three phases in sequence. Maven compilation and Citrus testing each have a 15-attempt ceiling; the Camel Main startup smoke test has a 6-attempt ceiling. Repeated error classes can promote to re-planning before those limits. An environment probe runs before verification (as the first step of camel-execute) to catch dependency, service, and startup issues before code is generated.
Error Classification System
The source taxonomy uses specific error families rather than four generic uppercase buckets. Representative routing is:
| Error family | Fix target |
|---|---|
| Missing Camel or third-party dependency; version incompatibility | Self-repair with runtime-aware dependency coordinates and BOM alignment |
| Route creation, unknown component, missing bean, or injection failure | camel-implement for the affected flow |
| Wrong endpoint option or YAML schema failure | camel-validate for diagnosis, then camel-implement for the correction |
| Expression, type-conversion, transformation, or assertion mismatch | camel-implement |
| Timeout, external-service, or container-startup failure | Self-repair first; route logic goes to camel-implement when indicated |
| Test syntax or incorrect test expectations | camel-test |
| MCP-confirmed unavailable component/pattern or a persistent error family | Bounded re-plan of affected design-spec flow sections, followed by affected-task execution and re-verification |
| Missing build plugin, Quarkus augmentation failure, iteration limit, or unclassified error | Escalate with the raw error and attempted fix |
Retry Budget
Maven compilation and Citrus testing each have a ceiling of 15 attempts; Camel Main startup smoke has a ceiling of 6. Promotion rules can end a local fix loop sooner.
Retry Strategy:
1. Classify the current error and route one targeted fix.
2. If the same error remains, short-circuit and evaluate promotion.
3. Trigger immediate Tier 1 re-planning when the MCP catalog confirms the required runtime feature does not exist.
4. Trigger Tier 2 re-planning after three failed fixes for the same error class.
5. Escalate unclassified errors or a phase that reaches its attempt limit.
Pipeline Re-entry
Resume /camel-execute after an approved manual correction so it can dispatch runtime verification again. If a previously working route breaks outside the pipeline, use /camel-debug; it preserves local state, diagnoses the root cause, applies a targeted fix, and recommends a recurrence guard.
Environment-in-the-Loop Concept
camel-verify is “environment-in-the-loop” verification: it doesn’t just check code. Citrus runs the integration and exercises its endpoints, while Testcontainers provides required external databases and brokers.
Why This Matters:
- Catches real issues: Code might compile but fail at runtime
- Validates integrations: Test endpoints are exercised and messages actually flow
- Tests behavior: Not just unit tests, but integration tests with required databases or brokers and mocked external APIs
- Prevents surprises: Find issues now, not in production
- Self-contained: Where external databases or brokers are required and Docker is available, Testcontainers manage their lifecycle; external APIs use mocks – no manual Docker Compose setup needed
Contrast with traditional testing:
- Unit tests: Mock everything (no environment)
- Integration tests: Exercise the application with required databases or brokers managed by Testcontainers and external APIs mocked (environment-in-the-loop)
Camel-Kit uses Citrus YAML integration tests for verification. Tests use Testcontainers when required external databases or brokers and Docker are available, and mocks for external APIs. Camel-Kit generates Docker Compose as a local-development artifact only when the integration needs external services; verification does not consume that Compose file.
Graceful Degradation
If tools are unavailable, the AI adapts:
No Docker
Warning: Docker not available.
Running container-free and mock-only Citrus tests.
Skipping only tests that declare Testcontainers.
Note: Without Docker, integration tests cannot start required external databases or brokers.
Consider running on a system with Docker installed.
Proceeding to report phase with dependent checks recorded as skipped...
No Maven (Spring Boot or Quarkus)
Warning: neither ./mvnw nor system mvn is available. Skipping build verification phase.
Proceeding to test verification...
No Camel Test CLI
Warning: camel test command not available.
Skipping test verification phase.
Note: Without the Camel JBang test plugin, we cannot run integration tests.
After installing Camel JBang if needed, run: camel plugin add test
No Citrus Tests
Warning: No Citrus YAML tests found (*.it.yaml)
Skipping test verification phase.
Note: Without tests, we cannot verify integration behavior.
The execute orchestrator can dispatch `camel-test` to generate missing tests.
The AI continues with available tools, warning about limitations.
Summary
camel-verify collects runtime evidence through a 3-phase feedback loop:
- Build / Startup Smoke - Compile Spring Boot or Quarkus with the wrapper or system Maven; start and inspect Camel Main instead
- Test Verification - Run Citrus YAML tests via
camel test run, using Testcontainers only for tests that declare required external services - Report Generation - Summarize results and provide insights
Key Features:
- Error Classification - Specific build, startup, runtime, and test error families with explicit fix targets
- Targeted Fix Loops - Up to 15 compile/test attempts or 6 Camel Main startup attempts, with earlier promotion for persistent errors
- Fix Routing - Errors routed to self-repair,
camel-validate→camel-implement,camel-implement,camel-test, boundedre-plan, or escalation - Environment-in-the-Loop - Testcontainers manage required databases and brokers for the tests that declare them
- Graceful Degradation - Reports unavailable tools and explicitly skips dependent checks
- Internal Dispatch - Runs within
/camel-execute;/camel-debughandles ad-hoc failures
The result is an explicit PASS, PARTIAL, FAIL, or NOT_RUN record rather than a silent claim that the integration works.
After runtime verification finishes, regardless of outcome, a chained pipeline continues from Execute to /camel-validate; standalone /camel-execute reports completion and stops.