FS-DOC-06Quality
Test and QA Plan
Capacity Planning Automation
[YourCompany.com] · Operations Department · Prepared by FullSpec · [Today's Date]
This document defines the complete quality assurance process for the Capacity Planning automation, covering all three agents: Data Collection, Capacity Analysis, and Reporting and Notification. It is intended for use by the FullSpec build team during development and QA, with the process owner joining Phase 3 to review and approve the live production run. Testing proceeds in strict sequence: no phase begins until the previous phase fully passes. All test cases are numbered sequentially across phases.
01Testing approach
Testing is structured across three phases in sequence. No phase begins until the previous phase fully passes. Phase 1 validates each agent in isolation using sandbox credentials and synthetic data. Phase 2 validates the handoffs between agents and stress-tests edge cases including missing data, duplicate triggers, and tool unavailability. Phase 3 runs a single end-to-end trace in the live production environment with the process owner present to review outputs and record approval.
1Phase 1: Unit Testing
Each agent tested in isolation before integration begins
Scope
Individual agent behaviour: Data Collection Agent, Capacity Analysis Agent, Reporting and Notification Agent
Environment
Sandbox only. No production credentials used at any point in Phase 1.
Credential rule
Sandbox OAuth tokens for Asana, Google Calendar, HubSpot, Google Sheets, Slack, and Notion. Production tokens must not be loaded into the environment during Phase 1.
Pass condition
All test cases T01 through T15 return expected outputs with no unhandled errors. Any failure blocks progression to Phase 2.
Who runs it
FullSpec team only. Process owner not required for Phase 1.
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure mode validation
Scope
Cross-agent data handoffs, pipeline trigger scenarios, and all defined edge cases including duplicates, missing fields, and tool downtime
Environment
Sandbox environment with simulated tool failures injected at the orchestration layer. No production data.
Credential rule
Sandbox credentials only. Edge case simulations use mocked API responses rather than real tool outages.
Pass condition
All integration test cases T16 through T22 pass, and all edge case tests E01 through E06 resolve to the expected fallback or error state without crashing the workflow.
Who runs it
FullSpec team only. Process owner not required for Phase 2.
3Phase 3: End-to-End Production Test
Full live run with process owner sign-off
Scope
One complete production run traced from trigger through all three agents to Slack delivery and Notion page publication
Environment
Live production environment using real credentials and real connected accounts (Asana, Google Calendar, HubSpot, Google Sheets, Slack, Notion)
Credential rule
Production OAuth tokens loaded. Test records must use a designated internal test address and be voided in all destination systems after the run completes.
Pass condition
All production test cases T23 through T26 pass. Final case T26 is the process owner review and written approval, recorded in the FullSpec confirmation log.
Who runs it
FullSpec team executes; process owner joins to review outputs and record approval for T26.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only for all Phase 1 tests. Do not load production OAuth tokens, API keys, or workspace credentials into the test environment at any point during this phase. Use dedicated sandbox accounts for Asana, Google Calendar, HubSpot, Google Sheets, Slack, and Notion. Any test that inadvertently writes to a production system must be flagged and the record voided before Phase 2 begins.
Data Collection Agent (Complexity: Moderate. Estimated build time: 14 hours)
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: scheduled Monday trigger
Monday 07:00 cron fires; sandbox Asana has 5 team members with tasks assigned for the coming week
Structured dataset containing assignee, allocated hours, and due dates for all 5 members
Dataset fields: assignee_id, allocated_hours, task_count, week_start all populated with no null values
T02
Happy path: HubSpot Closed Won webhook trigger
HubSpot deal moves to Closed Won stage; webhook fires with deal_id and estimated_resource_hours field populated
Agent triggers immediately; deal resource hours appended to pipeline demand column in dataset
Trigger latency under 60 seconds; pipeline_demand field non-zero in output dataset
T03
Google Calendar leave detection: leave event present
Team member has a calendar event titled 'Annual Leave' spanning 3 working days in the coming week
Available hours for that member reduced by 3 x daily hours (e.g. 24 hours deducted from a 40-hour week)
available_hours field reflects deduction; leave_flag boolean set to true for affected member
T04
Asana task pull: member has no tasks assigned
One team member has zero tasks in Asana for the coming week
Member included in dataset with allocated_hours = 0 and task_count = 0
Member row present; no null; allocated_hours = 0; no error thrown
T05
Asana API returns 401 Unauthorized
Sandbox Asana token deliberately expired or revoked
Agent logs authentication error; workflow halts gracefully; error event written to error log
Workflow does not proceed to Capacity Analysis Agent; error_code 401 captured in log; no partial dataset written
T06
HubSpot resource estimate field missing
Closed Won deal has no value in the estimated_resource_hours custom field
Agent defaults pipeline_demand to 0 for that deal; warning flag written to log; dataset still complete
pipeline_demand = 0; warning logged with deal_id; no crash; dataset passed downstream
T07
Google Calendar API rate limit hit (429)
Sandbox mock returns HTTP 429 on the third calendar request
Agent retries with exponential backoff (3 attempts, 5s / 15s / 45s delays); succeeds on retry or logs failure after third attempt
Retry behaviour logged; if all retries fail, error written and workflow halts; no infinite loop
Capacity Analysis Agent (Complexity: Moderate. Estimated build time: 12 hours)
ID
Scenario
Input
Expected Output
Pass Criteria
T08
Happy path: mixed utilisation across team
Dataset with 5 members: 2 at 85% utilisation, 1 at 110%, 1 at 45%, 1 at 72%
Utilisation table written to Google Sheets; member at 110% flagged overload; member at 45% flagged underload
Google Sheets row per member; status_flag = 'overload' for 110% member; status_flag = 'underload' for 45% member; remaining 3 = 'ok'
T09
All staff within normal range (60-100%)
Dataset with 5 members all between 62% and 98% utilisation
All rows written with status_flag = 'ok'; no alert trigger sent to Reporting Agent
No overload or underload flags; alert_required boolean = false passed to next agent
T10
Pipeline pressure adds demand
Base dataset shows team at 80% average; HubSpot pipeline adds 30 hours of incoming demand
Recalculated utilisation reflects pipeline demand; at least one member pushed above 100% threshold
pipeline_pressure_flag = true in output; affected member status updated to 'overload' in Sheets
T11
Google Sheets write fails (503 Service Unavailable)
Sandbox Sheets API returns 503 on write attempt
Agent retries up to 3 times; if all fail, error logged and workflow halts before Reporting Agent is called
No partial row written; error_code 503 in log; Reporting Agent not triggered; run marked failed
T12
Upstream dataset is empty
Data Collection Agent passes empty dataset (zero rows)
Capacity Analysis Agent detects empty input; logs warning; skips calculation; workflow halts
No Sheets update; no Reporting Agent call; warning message in error log with run_id reference
Reporting and Notification Agent (Complexity: Simple. Estimated build time: 8 hours)
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Happy path: Slack alert delivered with overload flags
Capacity analysis output includes 1 overloaded member and 1 underloaded member; alert_required = true
Slack message sent to ops manager channel listing both members, utilisation percentages, pipeline pressure status, and link to Google Sheet
Slack message received in sandbox channel within 30 seconds; all required fields present; Sheet link resolves
T14
Happy path: Notion summary page created
Full utilisation table passed to agent; no prior Notion page for this week's date
New Notion page created under the capacity planning parent with week_start date as title; all members listed with utilisation and status
Notion page exists post-run; page_title matches week_start; member count in page matches dataset row count
T15
Slack API returns 404 (channel not found)
Sandbox Slack channel ID configured incorrectly
Agent logs delivery failure with channel_id and error code; Notion write still attempted and completed
Error logged with channel_id; Notion page successfully created regardless; run marked partial-fail not full-fail
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Integration tests validate the data handoff points between agents and confirm that the end-to-end chain behaves correctly when agents pass outputs to one another. All tests run in sandbox. IDs continue from Phase 1.
ID
Scenario
Input
Expected Output
Pass Criteria
T16
Data Collection to Capacity Analysis handoff: full dataset
Data Collection Agent completes successfully; structured dataset with 5 members passed to Capacity Analysis Agent
Capacity Analysis Agent receives dataset without manual intervention; calculation begins within 10 seconds of Data Collection completing
Capacity Analysis Agent run log shows trigger_source = 'DataCollectionAgent'; no data loss between agents
T17
Capacity Analysis to Reporting handoff: alert required
Capacity Analysis Agent writes results to Sheets with at least one overload flag; alert_required = true
Reporting Agent triggers automatically; Slack message and Notion page both produced in the same run
Both Slack and Notion delivery confirmed in run log; no manual trigger required; end-to-end latency under 3 minutes
T18
Capacity Analysis to Reporting handoff: no alert required
All staff within 60-100% range; alert_required = false
Reporting Agent still publishes Notion summary page; Slack alert is suppressed
Notion page created; Slack message not sent; log entry confirms alert_suppressed = true with reason
T19
Dual trigger: scheduled run and Closed Won fire within same hour
Monday 07:00 cron fires; HubSpot Closed Won webhook also fires at 07:22 the same morning
First trigger initiates a full run; second trigger detected as duplicate within a 60-minute deduplication window and suppressed
Only one complete run executed; second trigger logged as deduplicated with original run_id reference; no double-write to Sheets
T20
Asana data refresh mid-run: task added after pull starts
Task assigned to a team member in Asana 30 seconds after Data Collection Agent begins its pull
Late task not included in current run; included in next scheduled run
Dataset reflects Asana state at pull start time only; no partial row errors; log records pull_timestamp
T21
Google Sheets row already exists for current week
Capacity Analysis Agent attempts to write to a Sheets row where week_start date already has an entry from a previous run
Agent overwrites the existing row rather than appending a duplicate
Sheet contains exactly one row per team member per week_start date after run completes; no duplicate rows
T22
Notion page already exists for current week
Reporting Agent attempts to create a Notion page for a week_start date where a page already exists
Agent updates the existing page content rather than creating a second page
Single Notion page per week_start date; page last_edited timestamp updated; no duplicate pages in parent
Edge case tests below cover failure modes, missing data, unavailable tools, and human timeout scenarios. These use injected mock responses and sandbox credentials throughout.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate HubSpot webhook: same deal fires twice
Closed Won webhook fires for deal_id 98321 at 09:00 and again at 09:04 (network retry)
Deduplication logic identifies matching deal_id within 10-minute window; second event suppressed
Only one pipeline demand entry per deal_id per run; second event logged as duplicate with original event_id
E02
Missing assignee field in Asana task
One Asana task in the dataset has a null assignee (task created without assignment)
Task excluded from individual utilisation calculation; logged as unassigned_task with task_id; team total hours unaffected
No crash; unassigned task logged with task_id; no null propagated into Sheets; warning in run log
E03
Team member has no Google Calendar access (permission denied)
Calendar API returns 403 Forbidden for one team member's calendar
Agent records available_hours for that member as full_week_hours (no leave detected); leave_flag = false; warning logged
Dataset row present for member with leave_flag = false and warning_code = 'calendar_403'; no crash; other members unaffected
E04
Slack unavailable at time of Reporting Agent run (tool outage)
Slack API returns 503 for all message attempts during a simulated outage window
Agent retries 3 times with backoff; after all retries exhausted, logs Slack delivery failure; Notion write still completes
Notion page created; Slack failure recorded in error log with retry_count = 3; run marked partial-fail; no silent data loss
E05
Utilisation threshold field missing from Sheets configuration
Capacity Analysis Agent reads Sheets config tab; overload_threshold cell is blank
Agent falls back to hardcoded defaults (overload = 100%, underload = 60%); logs warning that config was missing
Calculation proceeds using defaults; warning logged with config_field = 'overload_threshold'; no crash; fallback values recorded in run log
E06
Human review step times out: ops manager does not act on Slack alert within 48 hours
Slack alert sent Monday 07:30; no action recorded by Wednesday 07:30 (48-hour window)
Automation sends a single follow-up Slack reminder to the ops manager; no further automated action taken; reallocation decision remains human
One follow-up message sent at 48-hour mark; no second reminder sent; reminder logged with original alert run_id; no automated reallocation triggered
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: Use a designated internal test address and internal Slack channel for all notifications during this run. Do not send outputs to the live ops manager channel or publish to the primary Notion leadership space until the test is confirmed clean. After the run completes, void the test Sheets row, archive the Notion test page, and confirm with FullSpec that no test data remains in any production system before the automation is handed over for live use. Production OAuth tokens will be active during this phase: treat all credentials with the same care as the live environment.
Phase 3 runs one complete production trace from trigger to delivery. The process owner must be present for the final test case (T26). All cases below use live production credentials and real data from connected systems. IDs continue from Phase 2.
ID
Scenario
Input
Expected Output
Pass Criteria
T23
Full production run: Monday schedule trigger
Monday 07:00 production cron fires; live Asana workspace has current week's tasks; live Google Calendar has at least one leave event; HubSpot has at least one late-stage deal
Data Collection Agent produces a complete dataset within 2 minutes; Capacity Analysis Agent writes utilisation table to production Google Sheet within 4 minutes of trigger; Reporting Agent delivers Slack message and Notion page within 6 minutes of trigger
All three agents complete without error; dataset row count matches team member count; Sheets row for current week_start present; Slack message delivered to internal test channel; Notion page created in test section; total run time under 8 minutes
T24
Error log verification
Phase 3 run completes (T23 pass); error log reviewed immediately after
Error log contains zero unhandled exceptions; any warnings (e.g. missing HubSpot resource field) are present as warning-level entries, not errors
error_count = 0 in run log; warning entries present only if applicable; each warning includes run_id, agent_name, and field reference; no silent failures
T25
HubSpot Closed Won trigger in production
A test deal (clearly labelled 'QA Test Deal - Do Not Close') is moved to Closed Won in the production HubSpot sandbox account during the Phase 3 session
Automation triggers within 60 seconds; pipeline demand column updated in Sheets; deal voided in HubSpot after test confirms trigger
Trigger latency under 60 seconds confirmed in run log; pipeline_demand field updated for the test deal; deal returned to prior stage or deleted in HubSpot within 5 minutes of test completion
T26
Process owner review and approval
T23 through T25 all pass; process owner (Operations Manager) reviews the Slack alert content, the Google Sheets utilisation table, and the Notion summary page produced during the Phase 3 run
Process owner confirms: (1) Slack alert contains correct member names, utilisation percentages, and Sheet link; (2) Google Sheets reflects accurate team data with correct status flags; (3) Notion page is readable and matches the agreed format; (4) no unexpected data appears in any production system
Process owner records written approval in the FullSpec confirmation log. Approval must reference run_id from T23. No separate signoff block is required: this case entry and the FullSpec confirmation log constitute the formal record of acceptance.
Once T26 approval is recorded in the FullSpec confirmation log, the automation is cleared for go-live. The FullSpec team will schedule the cutover, retire the manual workflow steps, and confirm the first live unattended run with you. Any issues found after go-live should be reported to support@gofullspec.com with the run_id from the affected run.
Test and QA PlanPage 4 of 4