Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Team Performance Dashboard

[YourCompany.com] · Management Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance plan for the Team Performance Dashboard automation. It covers every test case the FullSpec team runs before and during go-live, organised across three sequential phases: unit testing of individual agents, integration and edge-case testing of agent handoffs and failure scenarios, and a final end-to-end production run. No phase begins until the previous phase has fully passed. The process owner joins the FullSpec team for Phase 3 only, to review and formally approve the live output before the automation is handed over.

01Testing approach

Testing is structured across three phases that must be completed in sequence. A phase may not begin until every test case in the preceding phase carries a pass status and any defects found have been resolved and re-tested. This sequencing ensures that integration and production tests are never run against code that has not first passed in isolation.

Phase 1: Unit Testing

Each agent tested in isolation against sandbox credentials

Scope

Individual agent logic: scheduled trigger accuracy, HubSpot API call, Jira API call, Google Sheets write, Slack reminder dispatch, validation rules, anomaly detection, Notion commentary draft

Environment

Sandbox only. No production credentials are used at this phase.

Credential rule

HubSpot sandbox account, Jira sandbox project, Google Sheets test workbook, Notion test page, Slack test workspace. All credentials stored in the automation platform's secret manager under a sandbox prefix.

Pass condition

All cases T01 through T14 return expected output with no unhandled errors.

Who runs it

FullSpec team. Process owner is not required at this phase.

Phase 2: Integration and Edge-Case Testing

Agent handoffs and failure-mode coverage in sandbox

Scope

Cross-agent handoff from Data Collection Agent to Validation and Commentary Agent; edge cases including duplicate rows, missing mandatory fields, tool unavailability, and human submission timeout

Environment

Sandbox only. Simulated failure states injected via test fixtures.

Credential rule

Same sandbox credentials as Phase 1. No production API keys are introduced.

Pass condition

All cases T15 through T20 and E01 through E06 return expected output or expected error-handling behaviour with no silent failures.

Who runs it

FullSpec team. Process owner is not required at this phase.

Phase 3: End-to-End Production Test

Full live run with real credentials and process owner sign-off

Scope

Complete workflow from Monday 7:00 AM trigger through HubSpot and Jira API calls, Google Sheets population, Looker Studio refresh, Notion commentary draft, manager approval gate, and final Slack distribution

Environment

Production. Live HubSpot, Jira, Google Sheets, Looker Studio, Notion, and Slack credentials. Test records are voided after each run.

Credential rule

Production API keys used for the first time. FullSpec team confirms keys are scoped to read-only for HubSpot and Jira. Slack post is directed to a private test channel, not the live management channel, until T23 is passed.

Pass condition

All cases T21 through T24 pass, including process owner review and approval recorded in the FullSpec confirmation log.

Who runs it

FullSpec team executes; process owner joins to complete the approval gate in T24.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only: All Phase 1 tests must be executed against sandbox accounts for HubSpot, Jira, Google Sheets, Notion, and Slack. Do not connect any production API key during this phase. If a production key is accidentally used, rotate it immediately and notify support@gofullspec.com.

Data Collection Agent — Estimated build time: 12 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T01

Scheduled trigger fires at correct time

System clock reaches Monday 07:00 AM

Workflow execution log shows trigger fired within 60 seconds of 07:00 AM

Timestamp in execution log is 07:00:00 to 07:01:00 UTC on a Monday

T02

HubSpot API: successful metric retrieval

Sandbox HubSpot account with 3 deals in current period

Pipeline value, deal count, and revenue figures written to Google Sheets staging row columns B2:D2

All three values match HubSpot sandbox records exactly; no null cells

T03

HubSpot API: authentication failure

Invalid or expired HubSpot API key injected

Workflow catches 401 response, writes error flag to staging sheet column Z2, and logs the failure

Error flag present; no silent continuation; execution log shows 401 error detail

T04

HubSpot API: rate limit response

HubSpot sandbox configured to return 429 on first call

Workflow retries after 30 seconds, up to 3 attempts, then flags if all retries fail

Retry logic visible in execution log; successful retry on attempt 2 or 3 resolves T02 scenario

T05

Jira API: successful sprint metric retrieval

Sandbox Jira project with sprint velocity 42, 7 open issues, cycle time 3.2 days

Three metric values written to Google Sheets staging row columns E2:G2

All three values match Jira sandbox data exactly; no null cells

T06

Jira API: authentication failure

Invalid Jira API token injected

Workflow catches 401, writes Jira error flag to staging sheet column AA2, logs failure

Error flag present in correct column; execution log shows Jira 401 detail

T07

Google Sheets write: correct row appended

Full set of HubSpot and Jira sandbox metrics available

Single new row appended to the staging sheet with ISO week number in column A and all metric values in correct columns

Row count increments by 1; week number matches current ISO week; no existing row overwritten

T08

Google Sheets write: duplicate row prevention

Trigger fires twice in same Monday window (simulated re-run)

Second run detects existing row for current week and does not append a duplicate

Row count remains unchanged after second run; execution log confirms duplicate skip

T09

Slack reminder: sent to correct team lead

One manual-input metric flagged as missing (column H2 empty) in staging sheet

Slack DM sent to the team lead mapped to that metric with a direct link to their Google Sheets input cell

Test Slack workspace shows DM from bot to correct test user; link resolves to correct cell

T10

Slack reminder: no message sent when all fields populated

All manual-input columns in staging row populated before agent runs

No Slack message dispatched

Test Slack workspace shows zero new DMs from bot in this run

Validation and Commentary Agent — Estimated build time: 14 hours. Complexity: Complex.

Scenario

Input

Expected output

Pass criteria

T11

Validation: all fields present, no anomalies

Google Sheets staging row fully populated; all values within 25% of previous week

Validation status written as PASS to column AB2; no anomaly flags; Notion draft created

Column AB2 reads PASS; Notion test page contains a draft commentary block with no anomaly section

T12

Validation: anomaly detection triggers on large movement

One metric value is 40% higher than the prior week value in the staging sheet

Anomaly flag written to column AC2; Notion draft includes a flagged-items section naming the metric and the delta

Column AC2 is non-empty; Notion draft contains the word 'flagged' and the correct metric name

T13

Validation: missing mandatory field after 90-minute wait

One mandatory staging column remains empty; 90-minute timer elapses

Agent proceeds with available data; missing field noted in Notion draft commentary; Slack reminder re-sent to relevant team lead

Notion draft contains a missing-data note; Slack shows a second DM to the same test user

T14

Notion commentary draft: correct content and structure

Full validated staging row with two anomaly flags

Notion page for current week contains: headline summary sentence, metric table, anomaly section listing two items, and a status of 'Awaiting manager approval'

All four structural sections present in Notion test page; status field reads 'Awaiting manager approval'

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests confirm that the two agents hand off correctly and that the Looker Studio refresh and Slack distribution steps connect to the validated data as expected. All tests in this phase remain in the sandbox environment.

Scenario

Input

Expected output

Pass criteria

T15

Handoff: Data Collection Agent to Validation and Commentary Agent triggers correctly

Data Collection Agent completes with all mandatory fields populated in staging sheet

Validation and Commentary Agent starts within 2 minutes of Data Collection Agent completion event

Execution log shows Validation Agent start timestamp within 120 seconds of Data Collection Agent end timestamp

T16

Handoff: 90-minute wait path triggers Validation Agent on timeout

Data Collection Agent completes; one mandatory field remains empty; 90-minute timer set to 5 minutes in sandbox for speed

Validation and Commentary Agent fires at timeout with partial data and writes missing-field note to Notion

Execution log shows timer path taken; Notion draft contains missing-field note; no unhandled exceptions

T17

Looker Studio refresh triggered after Google Sheets write

Validated data row present in Google Sheets staging sheet

Looker Studio sandbox data source refreshes and reflects the new row values in the test dashboard

Looker Studio sandbox report shows updated metric values matching the staging sheet row

T18

Manager approval gate: approved path sends Slack post

Manager approval status set to APPROVED in Notion test page

Automation detects approval and posts the Looker Studio dashboard link and Notion summary to the Slack test channel

Slack test channel receives a message containing the dashboard URL and the first two lines of the Notion summary

T19

Manager approval gate: rejected path returns to Notion without Slack post

Manager approval status set to CHANGES REQUESTED in Notion test page

Automation detects rejection, does not post to Slack, and logs awaiting revision status in execution log

No new Slack message in test channel; execution log shows CHANGES REQUESTED state recorded

T20

Full sequential run across both agents end to end in sandbox

Trigger fires, Data Collection Agent runs to completion, Validation Agent runs to completion, manager approval simulated

Staging sheet populated, Looker Studio refreshed, Notion draft created, Slack post delivered to test channel

All four output artifacts present and accurate; total elapsed time under 15 minutes from trigger to Slack post

Edge-case tests cover failure modes and boundary conditions that sit outside the happy path. These are run after T15 through T20 have passed.

Scenario

Input

Expected output

Pass criteria

E01

Duplicate trigger: workflow fires twice in the same Monday window

Scheduled trigger invoked manually a second time while the first run is still active

Second execution is blocked or queued; no duplicate staging row is written; no duplicate Slack messages sent

Execution log shows only one active run; staging sheet has a single row for the current week

E02

Missing mandatory HubSpot field: API returns null for deal revenue

HubSpot sandbox deal record has revenue field set to null

Null value written to staging sheet with a MISSING flag; anomaly note added to Notion draft; Slack reminder sent to HubSpot data owner

Staging cell shows null with MISSING flag; Notion draft flags the field; no unhandled null pointer error in execution log

E03

Jira API unavailable: service returns 503

Jira sandbox configured to return 503 for 3 consecutive calls

Workflow retries 3 times with exponential backoff; after all retries fail, writes JIRA_UNAVAILABLE to staging sheet and continues with HubSpot data only

Execution log shows 3 retry attempts with increasing delays; staging sheet contains JIRA_UNAVAILABLE flag; workflow does not halt entirely

E04

Notion API unavailable when commentary draft is attempted

Notion sandbox returns 503 on first two attempts

Workflow retries twice then succeeds on third attempt, or if all retries fail, logs NOTION_WRITE_FAILED and sends alert to Slack test channel

Retry logic visible in execution log; either Notion page created on retry or NOTION_WRITE_FAILED alert appears in Slack test channel

E05

Team lead does not respond to Slack submission reminder within 90 minutes

Manual input cell remains empty after Slack DM sent; 90-minute timer elapses (set to 5 minutes in sandbox)

Agent proceeds with partial data, notes the missing submission in the Notion draft, and does not block the rest of the workflow

Notion draft contains a named missing-submission note; workflow reaches Looker Studio refresh step without error

E06

Anomaly threshold: metric at exactly 25% week-on-week movement

Previous week value 100, current week value 125 in staging sheet

Boundary value of exactly 25% does not trigger an anomaly flag; only values above 25% are flagged

No anomaly flag in staging sheet column AC2; Notion draft has no flagged-items section for this metric

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Direct the initial Slack distribution step to a private internal test channel, not the live management channel. Use a dedicated internal test email address for any notification recipients. After each production test run, void the test staging row in Google Sheets and archive the Notion test page to prevent test data from appearing in the live dashboard or reports. Confirm these steps are complete before approving T24.

Phase 3 uses live production credentials for the first time. The FullSpec team conducts the run and the process owner is present to complete the approval gate. Every output artifact is verified against the production systems before the process owner confirms acceptance. Test records are voided immediately after each case.

Scenario

Input

Expected output

Pass criteria

T21

Full production run: trigger to Google Sheets population

Production Monday 7:00 AM trigger fires (or manually advanced trigger for test); live HubSpot and Jira production API keys active

Live HubSpot pipeline metrics and Jira sprint metrics written to the production Google Sheets staging sheet; Slack DMs sent to any team lead with an outstanding manual input

Staging sheet contains a correctly formatted row with live metric values; values cross-checked against HubSpot and Jira dashboards manually by FullSpec team; Slack DMs present in production workspace

T22

Production Looker Studio refresh and Notion commentary creation

Populated staging row from T21; Validation and Commentary Agent runs against live data

Looker Studio live dashboard reflects current week data; Notion production page contains a draft commentary with correct metric values and any anomaly flags

Looker Studio dashboard values match staging sheet; Notion page shows status 'Awaiting manager approval'; at least one correctly formatted metric table present in the draft

T23

Error log verification: no unhandled exceptions in production run

Execution logs from T21 and T22 reviewed by FullSpec team

Zero unhandled exceptions; all retries resolved successfully; no silent failures in any API call or write step

Execution log shows all steps completed with status SUCCESS or handled-retry; no ERROR or UNHANDLED_EXCEPTION entries

T24

Process owner review and approval: final production acceptance

Process owner reviews the Notion commentary draft and Looker Studio dashboard produced in T21 and T22; approves the summary in Notion

Automation detects approval, posts the Looker Studio dashboard link and approved Notion summary to the Slack private test channel; FullSpec team records the approval event in the FullSpec confirmation log

Process owner confirms commentary is accurate and dashboard reflects correct data; Slack test channel receives the formatted distribution message; FullSpec confirmation log entry timestamped and associated with the process owner's name. This case constitutes the process owner's formal acceptance of the automation. No separate signature block is required.

Once T24 is logged as passed in the FullSpec confirmation system, the automation is cleared for live operation. The scheduled Slack distribution channel is then switched from the private test channel to the live management channel, and the Monday 7:00 AM trigger is confirmed as active. Any questions after go-live should be directed to support@gofullspec.com.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process