Skip to content

⚙️ Delivery — Activities

Purpose

Overview of core activities and role assignments during the Delivery phase, from technical integration to user training and acceptance.

When to use this?

You have passed Gate 3 and are ready to deploy the AI system to production. This page guides you through technical integration, human oversight implementation, user training, and compliance dossier completion.

🎯 Objective

Execute the Delivery phase activities to transition the AI system to production with technical integration, human oversight, user adoption, and regulatory compliance all verified and documented.

✅ Entry Criteria (Definition of Ready)

  • Gate 3 (Production-Ready) is approved.
  • All automated tests pass on the release candidate.
  • Production environment is provisioned and accessible.
  • Go-live plan is drafted with rollback procedure.

⚙️ Core Activities

1. Technical Integration

Connecting the AI to the existing software systems and security (access management). This is the technical foundation for Go-live.

Steps:

  1. Map System Dependencies: Document all systems the AI connects to — databases, APIs, authentication providers, message queues. Create a dependency diagram that shows data flow and integration points.
  2. Configure API Connections: Set up API endpoints, authentication tokens, rate limits, and error handling. Test each connection individually before integrating.
  3. Implement Access Management: Configure role-based access control (RBAC). Define who can access which functions and data. Align with the organisation's identity management system (e.g., Active Directory, SSO).
  4. Run Stability Tests: Execute load tests (expected volume), stress tests (peak volume), and failover tests (component failure). Document the results and address any failures before proceeding.
  5. Configure Monitoring Integration: Connect the AI system to the organisation's monitoring infrastructure. Ensure logs, metrics, and traces flow to the central dashboard.

Test integrations in a staging environment

Never test integrations directly in production. Use a staging environment that mirrors production as closely as possible. Test all integration points, including error handling and timeout scenarios.

2. Human Oversight

Implementing human supervision procedures (Human-in-the-Loop) as required for the chosen Collaboration Mode. Human oversight is mode-specific — the procedures for Mode 2 differ from those for Mode 4.

Steps:

  1. Define Oversight Procedures per Mode:

    • Mode 2 (Advisory): Human reviews every AI suggestion before acting. Document the review checklist and approval workflow.
    • Mode 3 (Collaborative): Human and AI iterate together. Document the collaboration protocol — who initiates, who reviews, how disagreements are resolved.
    • Mode 4 (Delegated): AI executes independently; human reviews escalations and performs periodic sampling. Define the confidence threshold for escalation, the sampling frequency, and the audit procedure.
    • Mode 5 (Autonomous): Human sets policy and monitors via circuit breakers. Define the policy constraints, the circuit breaker triggers, and the emergency stop procedure.

  2. Document Escalation Paths: For each escalation scenario, record:

    • What triggers the escalation (low confidence, boundary violation, user complaint).
    • Who is notified (role, contact information).
    • Response time expectation (e.g., within 15 minutes for critical escalations).
    • Resolution procedure (how the issue is investigated and resolved).

  3. Define Intervention Levels: Establish clear agreements on the degree of autonomy:

    • Level 1 — Full Human Control: Human initiates and approves every action.
    • Level 2 — Human Review: AI proposes, human reviews and approves.
    • Level 3 — Human Sampling: AI executes, human reviews a sample of outputs.
    • Level 4 — Human Exception: AI executes, human intervenes only on escalation.

  4. Test the Oversight Procedures: Run simulation exercises where the system triggers escalations. Verify that the right people are notified, respond within the expected time, and resolve the issue correctly.

3. Adoption & Training

Training users not only in the buttons, but in the new way of working. Adoption determines whether the system delivers value or sits unused.

Steps:

  1. Develop Training Materials: Create workflow guides that show the before/after comparison. Include screenshots, step-by-step instructions, and troubleshooting tips.
  2. Conduct Training Sessions: Run hands-on training sessions with representative users. Let them practice with real tasks, not just demonstrations.
  3. Teach Quality Awareness: Explain that AI output is probabilistic — it can be wrong. Teach users how to critically evaluate output: check sources, verify facts, apply domain knowledge.
  4. Set Up the Feedback Channel: Create a simple, accessible channel for user feedback. This could be a form, a Slack channel, or a dedicated email. Communicate the channel clearly and respond to every submission.
  5. Measure Adoption: Track usage metrics (active users, sessions, tasks completed) and satisfaction scores. Report adoption progress to the Business Sponsor.

Practical Example

Situation: A national recruitment agency deployed an AI system to screen CVs against job descriptions and produce ranked shortlists for recruiters. The system operated in Mode 3 (Collaborative): recruiters could accept, reject, or modify each AI ranking and add notes explaining their decision. Approach: The AI Product Manager developed training materials that contrasted the old workflow (manually reading 80+ CVs per vacancy, taking 3–4 hours) with the new workflow (reviewing an AI-generated shortlist of 12 candidates, taking 45 minutes). Training sessions were run in groups of 8 recruiters, each working with a real open vacancy from their own portfolio. A key module in the training was "Quality Awareness": recruiters were shown 5 examples where the AI had made plausible but incorrect inferences — such as assuming a candidate had "senior-level experience" because they had managed a student society. Recruiters practiced checking the AI's source citations (which paragraph in the CV supported each claim) and applying their domain knowledge. A feedback form was embedded directly in the screening interface, allowing recruiters to flag issues with one click. Adoption was measured through weekly dashboards showing active users, average time-per-vacancy, and the percentage of AI suggestions accepted, modified, or rejected. Result: After 4 weeks, 92% of recruiters were active users. Average time-to-shortlist dropped from 3.5 hours to 50 minutes. The feedback channel received 47 submissions in the first month, of which 12 led to prompt refinements and 3 led to updates in the Knowledge Coupling sources. The Training Materials and the feedback loop became part of the Compliance Dossier, demonstrating that human oversight was not just documented but actively practiced. The Go-live Plan included a phased rollout: 20% of vacancies in week 1, 50% in week 2, and 100% by week 4, with a rollback procedure if adoption fell below 70%.

4. Compliance Dossier

Completing all documentation for laws and regulations. The compliance dossier is the evidence pack that demonstrates regulatory compliance.

Contents:

  • Risk Pre-Scan — initial risk classification under the EU AI Act.
  • Validation Reports — results from the Validation Pilot and ongoing testing.
  • Technical Model Card — complete documentation of the running system.
  • Goal Definition — the system's purpose and Hard Boundaries.
  • Guardian Approvals — ethical and legal sign-offs at each gate.
  • Audit Trail — complete log of changes, validations, and decisions.
  • Incident Response Procedure — documented procedure for handling production incidents.

Steps:

  1. Collect All Artefacts: Gather all documents produced throughout the lifecycle. Verify that each artefact is complete, versioned, and signed off.
  2. Cross-Reference with Requirements: Map each document to the applicable regulatory requirement (EU AI Act articles, GDPR provisions, sector-specific regulations). Identify any gaps.
  3. Guardian Review: Have the Guardian review the complete dossier and confirm that all Hard Boundaries obligations are fulfilled.
  4. Business Sponsor Sign-off: The Business Sponsor signs off the Compliance Dossier, accepting accountability for the system's compliance posture.

👥 RACI

Role Responsibility in Delivery
Implementation Engineer Responsible: Technical connections, security, and stability testing.
AI Product Manager Accountable: Leads adoption, coordinates training, and owns the Go-live plan.
Guardian (Ethicist) Consulted: Verifies Human Oversight protocols and reviews Compliance Dossier.
Business Sponsor Consulted: Signs off the Compliance Dossier and authorises Go-live.
End Users Informed/Consulted: Are trained and provide initial practical feedback.

✅ Exit Criteria (Gate 4 — Go-live)

The Delivery phase activities are complete when:

  • Technical integration is verified with passing stability tests.
  • Human Oversight protocols are documented and tested.
  • User training is completed and feedback channel is active.
  • Compliance Dossier is complete and signed off.
  • Monitoring dashboards and alerts are active.

Collaboration Mode: [Mode X — Name]. Handover protocol specifies human oversight per mode. Required validation for this mode: → See Evidence Standards.

📦 Deliverables

  1. Go-live Plan — deployment sequence and rollback procedure.
  2. Human Oversight Protocol — mode-specific procedures and escalation paths.
  3. Training Materials — workflow guides and quality awareness content.
  4. Compliance Dossier — complete evidence pack.
  5. Operational Handover Checklist — production readiness confirmation.

Next step: Complete the handover checklist and activate the monitoring dashboard. → Use the Gate 3 Checklist as your starting point. → See also: Objectives | Monitoring & Optimisation | Traceability

Version: 1.1 Date: 07 May 2026 Status: Final