Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Incident and Issue Reporting Automation

[YourCompany.com] · Operations Department · Prepared by FullSpec · [Today's Date]

This document defines the full quality assurance process for the Incident and Issue Reporting automation. It covers all test cases across three sequential phases: unit testing of each agent in isolation, integration and edge-case testing of agent handoffs and failure scenarios, and a final end-to-end production run. The FullSpec team executes Phases 1 and 2 entirely in sandbox environments. The process owner joins Phase 3. No phase begins until the previous phase has fully passed. All test outcomes are logged in the FullSpec confirmation record.

01Testing approach

Testing is structured in three sequential phases. No phase begins until every test case in the previous phase has a confirmed pass status. This sequencing prevents integration tests from masking unit-level failures and ensures the production environment is only touched once the full workflow is verified in sandbox conditions.

Phase 1: Unit Testing

Each agent tested in isolation against sandbox credentials

Scope

Incident Triage Agent and Notification and Escalation Agent tested independently. No live handoffs between agents.

Environment

Sandbox only. Test Google Form, test Sheets tab, test Notion workspace, test Slack channel, test Gmail alias, sandbox PagerDuty service.

Credential rule

No production credentials may be used. All OAuth tokens and API keys must point to sandbox or staging equivalents.

Pass condition

All cases T01 through T14 return expected outputs with zero manual intervention required.

Who runs it

FullSpec team only.

Phase 2: Integration and Edge Case Testing

Agent handoffs, connector reliability, and failure-mode coverage

Scope

End-to-end agent handoff from Triage Agent output to Notification Agent trigger. Plus all edge cases: duplicates, missing fields, unavailable tools, and unanswered human tasks.

Environment

Sandbox only. Full connector chain active but pointing to sandbox endpoints for all six tools.

Credential rule

No production credentials. Sandbox PagerDuty service key must be distinct from any live on-call integration.

Pass condition

All cases T15 through T22 and E01 through E08 pass. Any edge case that surfaces unhandled errors blocks progression to Phase 3.

Who runs it

FullSpec team only.

Phase 3: End-to-End Production Test

Full production run with process owner present

Scope

Complete workflow traced through live production tools: Google Forms, Google Sheets, Notion, Slack, Gmail, and PagerDuty. One low-severity and one high-severity run minimum.

Environment

Production. Test submissions use a designated internal test address and are voided from the incident log immediately after sign-off.

Credential rule

Production credentials active. FullSpec confirms all OAuth scopes are granted under the [YourCompany.com] workspace before Phase 3 begins.

Pass condition

All cases T23 through T28 pass, including process owner review and approval recorded in the FullSpec confirmation.

Who runs it

FullSpec team executes; process owner reviews outputs and provides sign-off in the FullSpec confirmation record.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only for all Phase 1 tests. Do not connect any production Google Form, Sheets document, Notion workspace, Slack workspace, Gmail account, or PagerDuty service during this phase. Each agent is triggered independently; the handoff between agents is not under test here.

Incident Triage Agent — unit tests (T01 to T08)

Scenario

Input

Expected output

Pass criteria

T01

Happy path: low-severity submission

Google Form response with all required fields; severity estimate: Low; category: Equipment.

Row created in Sheets with correct incident ID, severity score 1-2, category 'Equipment', assigned owner per routing rubric, and Notion page created.

Sheets row matches all form fields. Notion page URL written to Sheets. No escalation triggered. Completion under 30 seconds.

T02

Happy path: medium-severity submission

Google Form response; severity estimate: Medium; category: Service Outage.

Row created in Sheets with severity score 3-4, category 'Service Outage'. Notion page created. No PagerDuty alert.

Severity score within rubric band 3-4. Owner field populated per routing table. No PagerDuty call made.

T03

Happy path: high-severity submission

Google Form response; severity estimate: High; category: Safety.

Row created in Sheets with severity score 5. Notion page created. PagerDuty escalation flag set to true in record.

Score is 5. PagerDuty flag present in Sheets row. Notion page created. Escalation passed to Notification Agent trigger payload.

T04

Happy path: critical-severity submission

Google Form response; severity estimate: Critical; free-text contains keywords 'injury' and 'immediate'.

Row created with severity score 5 and escalation flag. Category auto-assigned as 'Safety'. Notion page created.

Keyword detection overrides initial estimate if rubric conflict exists. Score 5 confirmed. Category correctly assigned.

T05

Missing description field

Google Form response where incident description field is blank.

Row created with all available fields. Description field in Sheets marked 'Not provided'. Notion page created with placeholder. No crash.

Agent does not halt. Sheets row written with 'Not provided' string. Notion page created. Error not thrown to workflow.

T06

Unrecognised category keyword

Google Form response with free-text description containing no rubric-matching keywords.

Category assigned as 'Uncategorised'. Severity scored using available fields only. Notion page created. Human-review flag set.

Category field reads 'Uncategorised'. Human-review flag present in Sheets row. No silent failure.

T07

Google Sheets write failure

Valid form response. Sheets API returns 503 on first attempt.

Agent retries up to 3 times with exponential back-off. If all retries fail, error is logged and workflow halts with alert.

Retry logic fires. After 3 failures, error record written to error log tab in Sheets (or equivalent). No silent data loss.

T08

Notion API unavailable

Valid form response. Notion API returns 500.

Sheets row is written successfully. Notion creation fails gracefully. Error flag set in Sheets row. Retry queued.

Sheets row present and complete. Notion URL field reads 'Creation failed - retry pending'. Workflow does not halt entirely.

Notification and Escalation Agent — unit tests (T09 to T14)

Scenario

Input

Expected output

Pass criteria

T09

Happy path: standard owner Slack notification

Classified incident record with severity 2, owner mapped to Slack user ID, Notion URL present.

Slack DM sent to owner with incident summary, severity, deadline, and Notion link. Message timestamp recorded in Sheets.

Slack message delivered within 60 seconds. Correct owner receives message. Notion URL renders as hyperlink in message.

T10

Happy path: Gmail reporter acknowledgement

Incident record with reporter email address, incident ID, assigned owner name, and resolution timeframe.

Gmail sent to reporter email confirming receipt, incident ID, owner name, and expected resolution window.

Email delivered to test reporter address. Subject line contains incident ID. All four data fields present in email body.

T11

Happy path: PagerDuty alert on high-severity incident

Classified incident record with severity score 5 and escalation flag set to true.

PagerDuty event fires to sandbox service. Incident title, severity, and Notion link included in payload.

PagerDuty sandbox receives event. Event key matches incident ID. Dedup key present to prevent duplicate alerts.

T12

No PagerDuty alert on low/medium severity

Classified incident record with severity score 2 and escalation flag set to false.

No PagerDuty call made. Slack and Gmail messages sent as normal.

PagerDuty API not called. Confirmed by API log showing no outbound request for this incident ID.

T13

Slack owner not found in routing table

Classified incident record where owner routing returns no matching Slack user ID.

Fallback Slack message sent to the designated fallback channel (e.g. #incidents-unassigned). Error flag set in Sheets row.

Fallback channel receives message. Sheets row has 'Owner not resolved' flag. No silent failure.

T14

Gmail send failure

Reporter acknowledgement triggered. Gmail API returns 429 (rate limit).

Agent retries after back-off delay. If retry succeeds, email sent. If all retries fail, error logged in Sheets row.

Retry logic executes. On persistent failure, Sheets row updated with 'Acknowledgement failed' flag. No silent drop.

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Phase 2 tests the full handoff between the Incident Triage Agent and the Notification and Escalation Agent, and verifies that connector failures and data anomalies are handled without silent data loss. All tests remain in the sandbox environment. IDs continue from the Phase 1 sequence.

Scenario

Input

Expected output

Pass criteria

T15

Full handoff: low-severity end to end in sandbox

Form submission triggers Triage Agent; output payload passed to Notification Agent.

Sheets row created, Notion page created, Slack DM sent, Gmail acknowledgement sent. No PagerDuty alert.

All four outputs verified in sandbox tools. Timing from form submit to final Gmail send under 90 seconds.

T16

Full handoff: high-severity end to end in sandbox

Form submission with severity Critical triggers both agents.

Sheets row, Notion page, Slack DM, Gmail acknowledgement, and PagerDuty alert all created.

All five outputs verified. PagerDuty sandbox alert confirmed. Dedup key matches incident ID.

T17

Triage Agent output payload missing owner field

Triage Agent produces record with no resolved owner (rubric gap for an unusual category).

Notification Agent receives payload, detects missing owner, routes to fallback Slack channel, flags Sheets row, and still sends Gmail to reporter.

Fallback channel receives Slack message. Sheets row has 'Owner unresolved' flag. Gmail still delivered to reporter.

T18

Notification Agent trigger delay over 5 minutes

Triage Agent completes successfully but orchestration layer delays passing the payload by 6 minutes.

Notification Agent receives payload and processes normally. Timestamp in Sheets reflects actual send time, not form submit time.

Both timestamps recorded correctly in Sheets. Late delivery does not cause a duplicate trigger.

T19

Reminder nudge fired when deadline approaches

Incident created with 48-hour resolution deadline. Simulate deadline minus 4 hours.

Scheduled reminder Slack message sent to owner. Message references incident ID and Notion link.

Reminder message delivered to correct Slack user. Incident ID and Notion URL present in message body.

T20

Resolution update propagated back to Sheets

Notion record marked resolved by owner via status field update.

Google Sheets row updated with status 'Resolved', resolution notes, and close timestamp.

Sheets row reflects resolved status within 60 seconds of Notion update. Close timestamp is accurate.

T21

PagerDuty service key invalid or expired

High-severity incident triggers PagerDuty call. API returns 401 Unauthorized.

Error logged. Fallback: Slack alert sent to designated escalation channel as substitute notification. Sheets row flagged.

Slack escalation channel receives fallback message. Sheets row shows 'PagerDuty alert failed'. FullSpec error log entry created.

T22

Multiple concurrent submissions

Three Google Form submissions arrive within 10 seconds of each other, each with different severity levels.

Three independent Sheets rows created with unique incident IDs. Three Notion pages created. Correct notifications sent per incident.

No rows overwrite each other. Incident IDs are sequential and unique. All three notification chains complete independently.

Edge case tests (E01 to E08)

Scenario

Input

Expected output

Pass criteria

E01

Duplicate form submission: same reporter, same description within 2 minutes

Two identical form responses submitted in rapid succession.

Second submission is flagged as a potential duplicate in Sheets. Duplicate flag visible. Only one Notion page created, or second page labelled 'Potential duplicate'.

Deduplication logic fires on second submission. Sheets row has 'Duplicate suspected' flag. Only one owner notification sent.

E02

Missing reporter email address

Form response with reporter email field blank.

Incident logged and classified. Gmail acknowledgement step skipped. Sheets row flagged 'Reporter email missing'.

Workflow does not crash. Slack owner notification still sent. Sheets row records missing email flag.

E03

Missing incident category selection

Form response where the category dropdown was left empty.

Category auto-assigned as 'Uncategorised'. Human-review flag set. All other outputs proceed normally.

'Uncategorised' appears in Sheets and Notion. Human-review flag present. No block on notification step.

E04

Google Sheets permanently unavailable

Triage Agent attempts to write row. Sheets API returns 503 for all 3 retry attempts.

Workflow halts after final retry. Error written to a secondary error log (separate tab or file). Alert sent to FullSpec error monitoring channel.

Error log entry created with incident payload preserved. FullSpec monitoring channel receives alert. No silent data loss.

E05

Notion permanently unavailable at record creation

Triage Agent attempts Notion page creation. API returns 500 for all retries.

Sheets row created successfully. Notion URL field set to 'Creation failed'. Error flag in Sheets. Incident ID preserved for manual Notion creation.

Sheets row complete and accurate. 'Creation failed' string in Notion URL column. Workflow continues to notification step.

E06

Assigned owner has left the organisation: Slack user deactivated

Routing table maps incident category to a Slack user who is no longer active.

Slack DM fails. Fallback message sent to #incidents-unassigned channel with incident ID and full summary.

Fallback channel receives message within 90 seconds. Sheets row shows 'Owner Slack inactive' flag. No silent failure.

E07

Human review timeout: incident flagged for manual classification but no action taken within 24 hours

Incident lands in 'Uncategorised' state with human-review flag. No manager action for 24 hours.

Automated reminder sent to designated reviewer via Slack after 24-hour threshold. Reminder repeats at 48 hours if still unresolved.

First reminder delivered at 24-hour mark. Second reminder at 48 hours. Sheets row updated with each reminder timestamp.

E08

Form submission arrives outside business hours: PagerDuty configured with business-hours-only on-call schedule

High-severity incident submitted at 11pm local time. PagerDuty on-call schedule has no active responder.

PagerDuty alert fires regardless. PagerDuty schedule handling determines escalation path. Sheets row and Notion page created normally.

PagerDuty receives alert payload. Response routing is handled by PagerDuty schedule configuration, not by the automation. Sheets and Notion records confirmed present.

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: All submissions must use a designated internal test email address (e.g. qa-test@[YourCompany.com]) as the reporter address. The test Slack recipient must be a FullSpec test user or a nominated internal reviewer, not an active on-call owner. The PagerDuty test alert must target the sandbox service key, not the live on-call integration, unless the process owner has explicitly approved a live PagerDuty test. Every test record created in production Sheets and Notion must be voided (deleted or clearly marked 'QA TEST - VOID') immediately after the process owner confirms sign-off. Failure to void test records will corrupt the live incident log.

Phase 3 proceeds only after all Phase 1 and Phase 2 cases have passed. The FullSpec team coordinates the production test session with the process owner. The owner must be present (or available by message) for the duration of Phase 3 to review live outputs and confirm the final approval case.

Scenario

Input

Expected output

Pass criteria

T23

Production run: low-severity incident, full chain

Internal staff member (FullSpec tester) submits a test Google Form response with severity Low, category Equipment, and test reporter email.

Production Sheets row created with unique incident ID. Production Notion page created. Slack DM sent to designated test owner. Gmail acknowledgement sent to test reporter address. No PagerDuty alert.

All four outputs verified in live production tools within 90 seconds. Incident ID is sequential with any existing production records. Sheets row schema matches configured column headers exactly.

T24

Production run: high-severity incident, full chain including PagerDuty

Internal tester submits form with severity Critical and category Safety. PagerDuty configured to point to sandbox service key for this test.

Production Sheets row and Notion page created. Slack DM and Gmail sent. PagerDuty sandbox receives alert.

All five outputs present. PagerDuty alert payload contains correct incident ID, severity label, and Notion URL. Dedup key confirmed unique.

T25

Reminder cadence verification in production

Low-severity incident created in production with deadline set 4 hours from test start using a back-dated deadline entry.

Scheduled Slack reminder fires to test owner at the deadline-minus-threshold interval.

Reminder message received by test Slack user. Message body contains correct incident ID, deadline time, and Notion link. Timestamp on Sheets row updated.

T26

Resolution close-out in production

Test owner marks the T23 Notion record as resolved, adds resolution notes.

Production Sheets row updated with status 'Resolved', resolution notes from Notion, and accurate close timestamp.

Sheets row reflects resolved status within 60 seconds. Resolution notes match what was entered in Notion. Close timestamp is within 5 seconds of actual update.

T27

Error log verification

FullSpec team reviews the error log (Sheets error tab or equivalent) after T23 through T26 have run.

Error log tab shows no unexpected entries for the test runs. Any retries logged during T23-T26 are explained and closed.

Zero unexplained error entries. If any retry events appear, each has a resolved status and did not result in data loss or duplicate records.

T28

Process owner review and approval

Process owner reviews all production outputs from T23 through T27: Sheets log, Notion pages, Slack messages (via screenshot or direct review), Gmail acknowledgement, and PagerDuty sandbox alert.

Process owner confirms all outputs are correct, the severity classification matches expectation, notifications arrived in the correct format, and the Sheets schema is audit-ready. Approval is recorded in the FullSpec confirmation document.

Process owner provides explicit written approval in the FullSpec confirmation record. No outstanding issues remain open. All test records in production Sheets and Notion are marked 'QA TEST - VOID' before the session closes. Phase 3 status set to 'Passed' in the FullSpec project log.

Once T28 is confirmed, the FullSpec team updates the project status to 'Ready for go-live' and issues the handover summary to the process owner. Any issues raised during T28 review are logged as pre-launch defects and must be resolved before the go-live date. Contact support@gofullspec.com for any questions arising from this QA plan.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process