Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

End-of-day / Shift Handover Automation

[YourCompany.com] · Operations Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance programme for the End-of-day / Shift Handover automation. It is written for the FullSpec build team and covers every test case required before the workflow goes live in production. Testing runs in three sequential phases: unit tests against isolated agent logic in a sandbox environment, integration and edge-case tests against live tool connections in staging, and a final end-to-end run in production with the process owner present. No phase begins until every test case in the previous phase reaches a passing state.

01Testing approach

All testing follows three phases in strict sequence. No phase begins until the previous phase fully passes. Each phase has a defined scope, environment, credential rule, pass condition, and a named responsible party. The FullSpec team runs Phases 1 and 2 independently. The process owner joins Phase 3 for the production run and final approval.

Phase 1: Unit Testing

Isolated agent logic; sandbox environment only

Scope

Each agent tested in isolation: trigger handling, form dispatch, Sheets log write, AI compilation logic, report formatting, Notion filing, Gmail send, Slack post, and PagerDuty alert creation.

Environment

Sandbox only. No production accounts, live channels, or real shift data are touched at this phase.

Credential rule

All tool connections must use sandbox or test credentials. Production API keys and OAuth tokens must not be loaded into the test environment at this phase.

Pass condition

All test cases T01 through T14 return the expected output with no unhandled errors. Any failure blocks progression to Phase 2.

Who runs it

FullSpec team only. Process owner not required.

Phase 2: Integration and Edge Case Testing

Agent handoffs and failure-mode validation; sandbox to staging

Scope

Agent-to-agent handoffs, tool API responses under realistic load, and all identified edge cases including missing data, duplicate submissions, unavailable tools, and human timeout scenarios.

Environment

Sandbox for edge cases; staging connections to live tool sandboxes (Slack test workspace, Sheets staging tab, Notion test database, PagerDuty sandbox service, Gmail alias).

Credential rule

Staging credentials only. OAuth scopes validated at this phase but against test accounts. No production workspaces or live channels used.

Pass condition

All integration test cases T15 through T22 and all edge-case tests E01 through E06 pass. All handoff payloads match the schema defined in the Integration and API Spec.

Who runs it

FullSpec team only. Process owner not required.

Phase 3: End-to-End Production Testing

Full production run with process owner sign-off

Scope

One complete shift handover cycle executed in the live production environment, traced from the shift-end trigger through to report delivery, Notion filing, and PagerDuty alert (if applicable).

Environment

Production. All live credentials active. Real Slack channels, Gmail accounts, Notion workspace, and PagerDuty service used under controlled conditions.

Credential rule

Production credentials loaded. The FullSpec team verifies that no sandbox keys remain in the environment before the test run begins.

Pass condition

All production test cases T23 through T26 pass. Process owner reviews the output and records approval in the FullSpec confirmation log. This constitutes the formal go-live sign-off.

Who runs it

FullSpec team executes; process owner (Operations Manager) reviews output and provides written approval via the FullSpec confirmation log.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only. Before running any test in this phase, confirm that all tool connections reference test API keys, sandbox OAuth tokens, and staging sheet IDs. Do not load production credentials. Any test that inadvertently posts to a live Slack channel or writes to the production Sheets log must be treated as a Phase 1 failure and the credential configuration reviewed before retesting.

Handover Collector Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T01

Scheduled shift-end trigger fires at correct time

Shift-end time reached in scheduler; no manual override present

Workflow initiates; Handover Collector Agent activates within 60 seconds of trigger time

Agent activation timestamp is within 60 s of the scheduled trigger time; no duplicate triggers fired

T02

Manual shift-close event logged in Google Sheets overrides scheduled trigger

Cell value 'CLOSED' written to the shift-status column in Google Sheets

Workflow initiates immediately on Sheets event; scheduled trigger suppressed for that shift

Workflow fires within 30 s of the Sheets write; scheduler does not fire a second instance for the same shift ID

T03

Slack message dispatched to all outgoing staff

Shift roster contains three staff members; all have valid Slack user IDs

One structured Slack DM sent to each staff member with the Google Form link and a ten-minute response window statement

Three messages sent; each contains the correct form URL; message timestamp logged in the Sheets audit column

T04

Google Form response appended to Sheets log correctly

One staff member submits a completed form with task status 'Done', no incidents, and a single equipment note

New row appended to the Sheets log with correct staff ID, timestamp, task status, incident flag (FALSE), and equipment note text

Row count increments by one; all five mapped columns populated; no null values in mandatory fields

T05

Response window closes with all staff responded

Three form submissions received within the ten-minute window

Agent sets window-status to 'COMPLETE' in the Sheets control tab; triggers Handover Compiler Agent

Window-status value is 'COMPLETE'; compiler trigger event fires within 10 s; no further Slack prompts sent

T06

Response window closes with one staff member missing

Two of three form submissions received; window expires

Agent sets window-status to 'PARTIAL'; missing staff member flagged in the Sheets log with 'NO_RESPONSE' in the status column; compiler trigger fires

Window-status is 'PARTIAL'; exactly one row contains 'NO_RESPONSE'; compiler trigger fires; no further reminders sent beyond initial message

T07

Slack API returns a rate-limit error during message dispatch

Slack API responds with HTTP 429 on the second of three DMs

Agent retries the failed message after a 60-second backoff; retries up to three times; logs the error to the Sheets audit tab

Retry attempt is logged; message is eventually delivered within three attempts; workflow does not abort

T08

Google Sheets write fails due to permission error

Service account token lacks write access to the target sheet

Agent logs a 'SHEETS_WRITE_FAILED' error to the workflow error log; workflow halts and sends an alert to the FullSpec monitoring channel

Error entry exists in the workflow error log with the correct error code; monitoring alert fired; no partial row written to the Sheets log

Handover Compiler Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T09

Agent reads completed Sheets log and generates structured report

Sheets log contains three rows: two 'Done' tasks, one equipment fault, zero incidents

Formatted handover report produced with sections: Tasks Completed, Equipment Notes, Incidents (none), Open Items (none)

All four sections present in output; task count matches input rows; equipment note text matches source; incident section reads 'None this shift'

T10

Report correctly flags a high-priority unresolved incident

Sheets log contains one row with incident_flag TRUE, incident_severity 'HIGH', description 'Forklift hydraulic fault'

Report marks the incident in the Incidents section with a HIGH label; PagerDuty alert flag set to TRUE in the compiler output payload

Incident appears under the correct section; severity label is 'HIGH'; PagerDuty trigger flag is TRUE in the output JSON

T11

Report produced when one staff member has 'NO_RESPONSE' status

Sheets log contains one 'NO_RESPONSE' row alongside two completed rows

Report includes an 'Incomplete Responses' section listing the non-responding staff member by name; all other sections populated from available data

'Incomplete Responses' section present; non-responder name matches the 'NO_RESPONSE' row; report is not aborted due to missing data

T12

Slack post of compiled report to operations channel succeeds

Compiler output payload ready; target Slack channel ID configured

Report posted to the operations Slack channel as a formatted message block; thread reply confirms delivery

Message appears in the correct channel within 30 s of compiler completion; message contains all four report sections; no duplicate posts

T13

Gmail email sent to incoming manager and on-call contacts

Compiler output ready; recipient list contains two addresses

One email sent to each recipient via Gmail with the report as the message body and subject line 'Shift Handover Report — [Date] [Shift Label]'

Two emails delivered to sandbox addresses; subject line matches the expected format; body content matches the compiled report text

T14

Notion filing fails due to expired OAuth token

Notion integration token is expired; compiler attempts to create a database page

Agent logs 'NOTION_AUTH_FAILED' to the error log; report is still delivered via Slack and Gmail; a retry is queued for the Notion step after token refresh

Error log entry present with correct code; Slack and Gmail delivery not blocked by the Notion failure; retry queue entry created

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration Tests — Agent Handoffs (IDs continue from Phase 1)

Scenario

Input

Expected Output

Pass Criteria

T15

Handover Collector Agent passes a correctly formed trigger payload to Handover Compiler Agent

Collector completes the response window with three rows in the Sheets log; window-status set to 'COMPLETE'

Compiler receives a trigger event containing the correct sheet ID, shift ID, and row count; begins compilation within 15 s

Compiler trigger event logged with shift_id, sheet_id, and row_count matching the collector output; compilation starts within 15 s

T16

Handover Collector Agent passes a 'PARTIAL' payload and compiler handles missing rows gracefully

Collector sets window-status to 'PARTIAL'; trigger payload includes a non-responder list with one entry

Compiler reads the non-responder list, includes the 'Incomplete Responses' section, and does not fail or skip the report

Compiled report contains the non-responder name; report delivery proceeds normally; no unhandled exceptions in compiler logs

T17

PagerDuty alert raised correctly when compiler flags a high-priority incident

Compiler output payload contains pagerduty_trigger: TRUE and incident details including severity HIGH and description text

PagerDuty event API receives a POST request with routing_key, payload.severity 'critical', and the incident description in payload.summary

PagerDuty sandbox receives the event within 20 s of compiler completion; event status is 'triggered'; severity and summary fields match the compiler output

T18

No PagerDuty alert raised when compiler finds no unresolved incidents

Compiler output payload contains pagerduty_trigger: FALSE; all tasks marked as resolved

No PagerDuty API call made; workflow proceeds directly to Slack post and Gmail send

PagerDuty API call count is zero for this run; Slack and Gmail steps execute normally; compiler log shows pagerduty_trigger evaluated as FALSE

T19

Notion database page created with correct shift-date title and all report sections

Compiler output ready; Notion integration token valid; target database ID configured

New Notion page created in the shift archive database with title matching 'Handover — [YYYY-MM-DD] [Shift]'; all report sections written as toggle blocks

Page exists in the Notion sandbox database; title format matches expected pattern; toggle block count equals four (Tasks, Equipment, Incidents, Open Items)

T20

Full end-to-end handoff: collector to compiler to all four delivery channels

Three staff respond to the form; one low-priority equipment note present; no incidents

Collector writes log, triggers compiler; compiler posts to Slack, sends Gmail, files to Notion, does not trigger PagerDuty

All three delivery channels confirmed in staging within 90 s of trigger; PagerDuty call count is zero; Notion page created; no errors in workflow log

T21

Collector agent handles a Google Forms webhook delivery delay of more than two minutes

Form submitted at T+0; webhook delivery delayed to T+2m 15s due to simulated latency

Agent detects the delayed response, accepts it within the extended tolerance window, and appends the row correctly to Sheets

Row is appended correctly; no duplicate row created; agent does not mark the staff member as 'NO_RESPONSE' before the tolerance window expires

T22

Compiler agent produces a consistent report format regardless of which staff member submitted data

Two separate runs with different staff subsets providing different combinations of completed and incomplete tasks

Both compiled reports follow the identical section structure with the same heading labels and field order

Section headers in both reports match exactly; field order within each section is identical; no format drift between runs

Edge Case Tests

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate Google Form submission from the same staff member

Staff member submits the form twice within the response window; both submissions have the same staff_id and shift_id

Second submission detected as a duplicate; only the first submission is written to the Sheets log; a deduplication note is appended to the audit tab

Sheets log contains exactly one row for that staff_id in that shift; audit tab contains a 'DUPLICATE_SUBMISSION' entry referencing the staff_id

E02

Form submission missing a mandatory field (task_status left blank)

Staff member submits the form with the task_status field empty; all other fields populated

Row written to Sheets with task_status marked as 'MISSING'; compiler includes the row in the report with a 'Missing status' note against that staff member

Row present in Sheets log with 'MISSING' in the task_status column; compiler report includes the staff member name with 'Missing status' note; workflow does not abort

E03

Slack API unavailable during the staff message dispatch step

Slack API returns HTTP 503 on all three dispatch attempts

Agent retries three times with exponential backoff (30 s, 60 s, 120 s); after three failures, logs 'SLACK_UNAVAILABLE' and sends an alert to the FullSpec monitoring webhook

Three retry attempts logged with correct backoff intervals; 'SLACK_UNAVAILABLE' error entry present; monitoring alert fired; workflow is paused, not aborted

E04

PagerDuty API returns an authentication error during alert creation

PagerDuty routing key is invalid; API returns HTTP 403

Agent logs 'PAGERDUTY_AUTH_FAILED'; Slack and Gmail delivery are not blocked; an alert is sent to the FullSpec monitoring channel with the error details

Error log entry present with code and routing key reference; Slack and Gmail steps complete normally; monitoring channel receives the alert within 60 s

E05

No staff respond to the form within the response window

Response window expires with zero form submissions; all staff Slack DMs were delivered

Agent sets window-status to 'PARTIAL'; all staff marked 'NO_RESPONSE'; compiler generates a report noting zero responses received; report delivered via Slack and Gmail

Window-status is 'PARTIAL'; all staff rows contain 'NO_RESPONSE'; compiled report contains 'Incomplete Responses' section listing all staff; delivery confirmed to Slack and Gmail

E06

Shift-end trigger fires twice for the same shift ID due to a scheduling misconfiguration

Two trigger events fired with the same shift_id within five minutes of each other

Second trigger detected as a duplicate by the idempotency check on shift_id; second workflow instance is suppressed; a deduplication log entry is created

Only one workflow instance runs for the shift_id; deduplication log entry present referencing the suppressed trigger event timestamp

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use a dedicated internal test address for the incoming manager email recipient (e.g. a FullSpec-managed alias) so that the test report is not delivered to live operational staff. Post the Slack report to a designated test channel rather than the active operations channel. After the test run is confirmed as passing, void the test shift record by marking it with a 'TEST' flag in the Sheets log and archiving the Notion page out of the main shift database. The FullSpec team and the process owner must both confirm these steps are complete before the workflow is cleared for unsupervised production use.

Scenario

Input

Expected Output

Pass Criteria

T23

Full production run: shift-end trigger to report delivery across all channels

Production shift-end trigger fires at the scheduled time; three staff members respond via the Google Form within the response window; one equipment note logged; no incidents

Handover Collector Agent fires on schedule; Sheets log populated with three rows; Handover Compiler Agent produces a formatted report; report posted to the test Slack channel; email sent to the internal test address; Notion page created in the production database (test-flagged); PagerDuty not triggered

All four delivery channels confirmed within 120 s of trigger; Sheets log contains exactly three rows; Notion page exists with correct title; PagerDuty event count is zero; no errors in the production workflow log

T24

Production run with an unresolved high-priority incident: PagerDuty alert raised

One staff member submits a form response with incident_flag TRUE and incident_severity 'HIGH'; description: 'Conveyor belt motor failure — unit offline'

Compiler flags the incident; PagerDuty event created in the production service with severity 'critical'; incident appears prominently in the Slack and Gmail report; Notion page records the incident under the Incidents section

PagerDuty production event exists with correct routing key and severity; incident text in Slack message and Gmail body matches the form submission description exactly; Notion page Incidents section is not empty

T25

Error log verification: confirm all workflow steps are logged correctly in production

Full production run from T23 completed successfully

Workflow execution log contains an entry for every step: trigger received, Slack dispatch (per staff member), Sheets write (per row), compiler start, compiler complete, Slack post, Gmail send, Notion create, PagerDuty evaluation (skipped), run complete

Log entry count matches the expected step count for a three-staff, zero-incident run; each entry contains a timestamp, step name, status ('SUCCESS'), and duration in milliseconds; no entries with status 'ERROR' or 'SKIPPED' outside of the PagerDuty evaluation step

T26

Process owner review and approval: final Phase 3 sign-off

Outputs from T23 and T24 presented to the Operations Manager (process owner) for review: Slack message, Gmail copy, Notion page, PagerDuty alert, and workflow error log

Process owner confirms the report format is correct, the content matches the test input data, the Notion archive is accurate, and the PagerDuty alert meets the agreed escalation threshold. Approval is recorded in the FullSpec confirmation log with the owner's name, date, and a written confirmation statement.

FullSpec confirmation log contains a timestamped entry with the process owner's name, the statement 'Phase 3 approved for production go-live', and the date. No unresolved comments or amendment requests are outstanding. The test shift record is voided in Sheets and the Notion test page is archived. The workflow is cleared for unsupervised production operation.

Once T26 is recorded in the FullSpec confirmation log, the automation is approved for unsupervised production use. Any issues discovered after go-live should be reported to support@gofullspec.com with the workflow run ID and a description of the unexpected behaviour. The FullSpec team will triage and resolve within the agreed support window.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process