FS-DOC-06Quality
Test and QA Plan
Board and Investor Reporting Automation
[YourCompany.com] · Management Department · Prepared by FullSpec · [Today's Date]
This document defines the complete testing and quality assurance programme for the Board and Investor Reporting automation. It covers three sequential phases: unit testing of each individual agent in a sandbox environment, integration and edge-case testing of agent handoffs and failure scenarios, and a final end-to-end production run against a live reporting period. The FullSpec team executes Phases 1 and 2 in full; the process owner joins Phase 3 to confirm outputs and record sign-off. No phase begins until every test case in the preceding phase achieves a passing result.
01Testing approach
Testing is structured in three phases that must be completed in strict sequence. Phase 1 validates each agent in isolation using sandbox credentials and synthetic data. Phase 2 tests the handoffs between agents and exercises edge cases including missing data, duplicate records, and tool unavailability. Phase 3 is a single traced end-to-end run in the production environment with real credentials, real data from a prior closed period, and process owner involvement. No phase begins until every test case in the previous phase carries a confirmed pass result, verified and signed off by the FullSpec team.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox credentials
Scope
Individual agent logic: Data Collector Agent, KPI Collection Agent, Pack Assembly Agent. Each tested independently with no dependency on other agents.
Environment
Sandbox only. Xero demo company, HubSpot sandbox portal, Google Sheets test workbook, Google Drive test folder, Slack test workspace.
Credential rule
No production API tokens may be used. All credentials must reference sandbox or test accounts. Sandbox OAuth tokens must be stored in the test credential vault, separate from production secrets.
Pass condition
All test cases T01 through T14 return the expected output with zero blocking errors. Any failure halts progression to Phase 2.
Who runs it
FullSpec team only. No process owner involvement required at this phase.
2Phase 2: Integration and Edge Case Testing
Agent handoffs, boundary conditions, and failure modes
Scope
End-to-end agent handoff chain in sandbox, plus isolated edge-case scenarios covering duplicate data, missing inputs, API unavailability, and KPI submission timeouts.
Environment
Sandbox environment for all integration tests. Edge cases may use mocked API responses to simulate rate limits and outages without touching production systems.
Credential rule
Sandbox credentials only, consistent with Phase 1. Mocked responses must be clearly flagged in test logs. No real financial data may be used.
Pass condition
All integration test cases T15 through T22 pass, and all edge-case tests E01 through E08 either pass or produce the specified graceful failure output with correct error logging.
Who runs it
FullSpec team only. Results are documented and shared with the process owner before Phase 3 is scheduled.
3Phase 3: End-to-End Production Test
Single traced run in production with process owner sign-off
Scope
A single complete workflow run using real production credentials, a prior closed reporting period, and a nominated internal test recipient list. Every agent fires in sequence and all outputs are inspected against source data.
Environment
Production. Real Xero organisation, real HubSpot portal, live Google Sheets, live Google Drive board folder, real Slack workspace. Test addresses used for board distribution step.
Credential rule
Production OAuth tokens and API keys are used for the first time in this phase. FullSpec confirms all credentials are stored in the production secrets manager before the run begins.
Pass condition
Test cases T23 through T26 all pass. The process owner reviews outputs, confirms figures match the source period, and records approval in the FullSpec confirmation log. This approval constitutes the final test case T26.
Who runs it
FullSpec team executes the run. The process owner (CFO or nominated delegate) joins to review outputs and record sign-off at the conclusion of Phase 3.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only for all Phase 1 tests. No production API tokens, real Xero organisations, real HubSpot portals, or live Google Sheets workbooks may be referenced during this phase. All OAuth tokens must point to sandbox or demo accounts. Any accidental use of production credentials must be reported to the FullSpec team immediately and the affected credential rotated before testing continues.
Data Collector Agent — fetches financial data from Xero and pipeline data from HubSpot, then writes structured results to the Google Sheets staging workbook. Estimated build time: 14 hours. Complexity: Moderate.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: Xero financial pull
Valid OAuth token, closed period (e.g. May), demo Xero company with P&L, cash flow, and balance sheet data present
Profit and loss, cash position, and balance sheet figures written to designated columns in the staging Google Sheet with correct period label
All three report types present in sheet; no null values in mandatory fields; period label matches input parameter
T02
Happy path: HubSpot pipeline pull
Valid HubSpot sandbox OAuth token, deals present in at least three pipeline stages
Pipeline by stage, weighted forecast total, and closed-won revenue appended to the same Google Sheet row as Xero figures
All three HubSpot metrics present; closed-won figure matches sandbox CRM total; no overwrite of Xero columns
T03
Xero authentication failure
Expired or revoked Xero OAuth token
Workflow halts; error entry written to a dedicated error log tab in Google Sheets; Slack alert sent to the nominated admin channel
No partial data written to staging sheet; error log entry includes timestamp, agent name, and error code; Slack message delivered within 60 seconds
T04
HubSpot authentication failure
Expired HubSpot API key
Workflow halts at HubSpot step; error logged; Slack alert fired; Xero data already written is preserved
Xero columns in staging sheet unchanged; HubSpot columns remain blank with a 'fetch_failed' flag; error log entry present
T05
Xero period with no transactions
Valid token, period selected that has zero activity in demo company
Agent writes zero values to financial columns and flags the period as 'zero activity' in a status column
Status column contains 'zero_activity'; downstream agents receive the flag and do not treat zeros as missing data
T06
Google Sheets write failure
Valid Xero and HubSpot data fetched; Google Sheets API returns a 503 error on write
Agent retries up to three times with exponential backoff; on persistent failure, error is logged and Slack alert sent
Retry attempts visible in execution log; after three failures, error log entry present; no silent data loss
KPI Collection Agent — sends structured Slack prompts to department heads, tracks responses against a deadline, fires one reminder for non-responses, and writes confirmed KPI values to Google Sheets. Estimated build time: 10 hours. Complexity: Moderate.
ID
Scenario
Input
Expected Output
Pass Criteria
T07
Happy path: all KPIs received before deadline
Three test Slack user IDs (sandbox workspace), KPI prompt template configured, deadline set 2 minutes from trigger for test speed
Structured KPI prompt delivered to each user; all three respond with valid values; figures written to correct Google Sheet columns with username and timestamp
All three Slack messages delivered; response values appear in sheet within 30 seconds of submission; no values missing or misallocated to wrong department column
T08
One non-responder: reminder fired
Two test users respond before deadline; one test user does not respond
Reminder Slack message sent to the non-responder after deadline passes; response window extended by configured grace period
Reminder message delivered to correct user only; no duplicate reminder to already-responding users; reminder timestamp logged in sheet
T09
All non-responders after reminder
No test users respond to initial prompt or reminder
After reminder deadline passes, agent marks KPI collection as 'incomplete'; Slack alert sent to CFO channel; workflow pauses and does not proceed to Pack Assembly Agent
CFO channel receives alert with list of non-responding departments; Pack Assembly Agent not triggered; status column in sheet reads 'collection_incomplete'
T10
Invalid KPI value submitted
Test user submits a text string where a numeric value is expected (e.g. 'TBC' in a revenue field)
Agent flags the response as invalid; re-prompts the user with a validation error message; does not write the invalid value to the sheet
Invalid value not written to sheet; user receives re-prompt within 60 seconds; re-prompt message references the specific field that failed validation
T11
Slack API unavailable during prompt send
Slack API returns 503 on the initial prompt send
Agent retries up to three times; on persistent failure, error logged and fallback alert sent via Google Sheets notification to CFO
Retry log entries present; after three failures, fallback notification delivered; no prompts silently dropped
Pack Assembly Agent — reads the completed Google Sheet, populates the board pack template in Google Drive, flags data anomalies, exports the final PDF, and notifies board members. Estimated build time: 14 hours. Complexity: Moderate.
ID
Scenario
Input
Expected Output
Pass Criteria
T12
Happy path: full pack assembly
Completed Google Sheet with all financial, pipeline, and KPI fields populated; board pack template present in Google Drive test folder
All figure references in the template updated with sheet values; charts reflect new data; a draft document created in the test Drive folder with correct filename including period label
Zero unchanged placeholder values in draft; chart data sources verified against sheet figures; filename format matches configured naming convention
T13
Missing data field in staging sheet
Google Sheet has one mandatory field left blank (e.g. gross margin is null)
Agent flags the specific missing field in a summary note appended to the draft document; affected section highlighted in amber; CEO notified via Slack with field name
Draft document produced with flag visible; no silent blank cell in the output; Slack notification references the exact missing field name
T14
Value outside expected range
A revenue figure in the sheet is 400% above the prior period benchmark stored in the template configuration
Agent inserts an out-of-range warning annotation adjacent to the figure in the draft; Slack alert sent to CFO noting the anomalous figure and its source cell
Warning annotation present in draft at correct location; Slack alert delivered; figure is not suppressed or altered by the agent
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Integration tests verify that agents hand off correctly to one another in sequence, that the trigger-to-completion chain fires as designed, and that status flags passed between agents are interpreted correctly. All tests in this section continue the T-sequence from Phase 1.
ID
Scenario
Input
Expected Output
Pass Criteria
T15
Full handoff: Data Collector to KPI Collection Agent
Data Collector Agent completes successfully; staging sheet status column set to 'data_ready'
KPI Collection Agent triggers automatically within the configured polling interval (max 5 minutes); Slack prompts fire to all configured department head user IDs
KPI Collection Agent start timestamp logged within 5 minutes of Data Collector completion; all Slack prompts delivered; no manual trigger required
T16
Handoff blocked: Data Collector fails
Data Collector Agent encounters a Xero API error and writes 'fetch_failed' to the status column
KPI Collection Agent does not start; workflow pauses; Slack error alert sent to CFO channel referencing the failed upstream agent
KPI Collection Agent shows no execution record after the Data Collector failure; CFO channel alert present; no partial Slack prompts sent to department heads
T17
Full handoff: KPI Collection to Pack Assembly Agent
All KPI responses confirmed; sheet status column set to 'kpis_complete'
Pack Assembly Agent triggers within the configured polling interval; template population begins immediately
Pack Assembly Agent start timestamp logged within 5 minutes of KPI completion; no manual trigger required; correct period label passed to the assembly step
T18
Handoff blocked: KPI collection incomplete
KPI status column reads 'collection_incomplete' after all reminders exhausted
Pack Assembly Agent does not fire; orchestration layer holds the workflow in a paused state; CFO receives a consolidated summary alert listing outstanding departments
Pack Assembly Agent shows no execution record; paused state visible in workflow run log; CFO alert lists each department with missing submission
T19
Schedule trigger fires on correct date and time
Automation platform configured with a cron-equivalent trigger set to the third business day after month end; test date matches the configured schedule
Data Collector Agent fires within 2 minutes of the scheduled time with no manual intervention
Execution log shows trigger firing within the 2-minute window; no duplicate executions within the same run cycle
T20
PDF export and Drive upload
Pack Assembly Agent completes draft; CEO approval flag set in the configured approval field
PDF exported from the Google Drive draft; uploaded to the board folder with correct filename; board member email notifications sent to test addresses
PDF present in board folder within 5 minutes of approval flag; filename includes period label and 'FINAL' suffix; all test recipient addresses receive notification email
T21
Duplicate trigger within same period
Schedule trigger fires a second time within the same reporting period (simulated via manual re-trigger in sandbox)
Workflow detects the period has already been processed via a run-log check; second execution is suppressed; Slack alert sent to admin noting the duplicate attempt
Only one completed run record exists for the period; second trigger does not overwrite the staging sheet or create a duplicate draft document
T22
Google Drive template missing or renamed
Pack Assembly Agent attempts to locate the board pack template by its configured file ID; file has been deleted or moved in the test Drive folder
Agent returns a specific 'template_not_found' error; workflow halts; Slack alert sent to CFO and admin channel with the missing file ID and Drive folder path
No partial document created; error message includes the file ID; alert delivered to both CFO and admin channels within 60 seconds
Edge case tests cover boundary and failure conditions that fall outside the normal happy path. These scenarios are designed to confirm that the system fails gracefully, logs accurately, and never silently drops data or produces an incorrect output.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate KPI submission from same user
A department head submits their KPI values twice via Slack (e.g. resubmits after editing a figure)
Agent retains only the most recent submission; earlier value overwritten in the sheet with a 'resubmitted' flag and updated timestamp; no duplicate rows created
Sheet contains exactly one row per department; resubmit flag present; audit log records both submission timestamps and the value used
E02
Missing mandatory financial field from Xero
Xero API returns a response with the balance sheet section absent (e.g. API scope does not include balance sheet endpoint)
Agent writes available fields; marks balance sheet columns as 'scope_missing'; Slack alert to CFO specifying the missing scope; workflow does not halt but Pack Assembly Agent flags the gap
Available fields written correctly; 'scope_missing' flag visible in sheet; Slack alert references the specific OAuth scope needed; pack draft produced with annotation in the balance sheet section
E03
HubSpot rate limit hit mid-pull
HubSpot API returns HTTP 429 after partial pipeline data is retrieved
Agent pauses and retries after the retry-after interval specified in the 429 response header; on success, completes the pull without duplicating already-fetched deal records
Final sheet data contains no duplicate deal records; retry logged with timestamp and wait duration; total pull completes within a 10-minute window
E04
Slack workspace connectivity lost mid-KPI collection
Slack API becomes unreachable after two of three department heads have responded
Agent queues the pending reminder for the third department head; retries when connectivity is restored; does not re-send prompts to departments that already submitted
No duplicate Slack messages to responded departments; pending reminder delivered on reconnection; execution log shows the connectivity gap and recovery
E05
Department head submits KPI after workflow timeout
A department head submits their values after the KPI collection timeout period has elapsed and the 'collection_incomplete' status has been set
Late submission is ignored by the agent; sheet status remains 'collection_incomplete'; a log entry records the late submission with timestamp and user ID for CFO review
Sheet data unchanged after timeout; log entry present; CFO alert already sent before the late submission is not re-triggered
E06
Google Sheets API quota exceeded during data write
Staging sheet write is attempted when the Google Sheets API daily write quota has been reached
Agent logs a 'quota_exceeded' error; retries the following minute using exponential backoff; does not partially write data
Either a clean retry succeeds and all fields are written atomically, or the error is logged with no partial row written; Slack alert sent if retry fails after five attempts
E07
Board pack template structure changed
A named range or cell reference that the Pack Assembly Agent uses to inject a financial figure has been renamed or deleted in the Google Drive template
Agent returns a 'field_reference_not_found' error for the affected cell; flags the specific missing reference in the error log; produces a partial draft with a visible placeholder where the figure should appear
Error log names the missing cell reference exactly; partial draft is saved to the test folder, not overwritten; CFO Slack alert specifies which section of the pack is affected
E08
Human KPI submission timeout with no reminder response
KPI Collection Agent sends initial prompt and one reminder; the department head does not respond to either within the configured grace period
After the grace period expires, agent marks the specific department's KPI row as 'no_response'; overall status set to 'collection_incomplete'; CFO alert lists the non-responding department by name; Pack Assembly Agent does not start
'no_response' flag present in the correct department row; CFO alert names the department; workflow remains paused; no pack assembly attempted with incomplete data
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: This phase uses production credentials and live data sources. You must select a prior closed reporting period (not the current open period) so that no live board pack cycle is disrupted. Configure the board member distribution step to send to an internal test address list only — do not include actual board member or investor email addresses. After the test run is complete, void the test record by archiving the draft document in a clearly labelled test subfolder in Google Drive and removing the test data rows from the staging Google Sheet. The FullSpec team will confirm this clean-up before the first live cycle begins.
Phase 3 consists of a single complete workflow execution traced from the schedule trigger through to board member notification, using a prior closed period as the data source. Every output is verified against the source data in Xero and HubSpot. Error logs are inspected to confirm no silent failures occurred. The process owner reviews the assembled draft and records approval to conclude the test programme.
ID
Scenario
Input
Expected Output
Pass Criteria
T23
Full production run: Data Collector and KPI Collection in sequence
Production Xero OAuth token, production HubSpot API key, prior closed period selected (e.g. the most recently completed month), all department head Slack user IDs configured in the production workspace
Data Collector Agent pulls Xero and HubSpot data for the selected period and writes all figures to the production staging Google Sheet. KPI Collection Agent fires automatically; Slack prompts delivered to all department heads in the production workspace; responses recorded in the sheet with correct names and timestamps
All financial and pipeline fields populated in the production sheet with non-null values; figures match Xero and HubSpot source exports verified manually by the FullSpec team; all Slack prompts delivered; KPI rows show correct department labels
T24
Full production run: Pack Assembly Agent and draft output
Completed production staging sheet from T23; production Google Drive board pack template at the configured file ID
Pack Assembly Agent populates all figure references in the template; all charts update to reflect the selected period's data; draft document saved in the production Drive folder with the correct filename; no 'template_not_found' or 'field_reference_not_found' errors in the execution log
Zero placeholder values remain in the draft; chart figures match the staging sheet values; filename includes the correct period label; execution log shows clean completion with no error entries
T25
Error log verification across the full run
Complete execution log from T23 and T24
Error log tab in Google Sheets contains zero entries for the production run. Workflow run log in the automation platform shows all steps completed with success status. No Slack error alerts were generated during the run.
Error log tab is empty; all workflow steps show green/success status in the run log; no unintended Slack alerts delivered to CFO or admin channels during the test
T26
Process owner review and approval
Draft board pack produced in T24; process owner (CFO or nominated delegate) provided with the Drive link and the staging sheet for cross-reference
Process owner confirms: figures match the selected prior period source data; pack layout and chart formatting are correct; no anomaly flags require escalation; owner records approval in the FullSpec confirmation log with their name, role, and date
FullSpec confirmation log entry present with process owner name, role, and sign-off date. This entry constitutes the final test case pass and authorises progression to the first live reporting cycle.
Once T26 is recorded in the FullSpec confirmation log, the test and QA programme is formally closed. The FullSpec team will archive all test records, rotate any sandbox credentials that were used during Phases 1 and 2, and confirm to the process owner in writing via support@gofullspec.com that the automation is cleared for the first live cycle. Any issues identified during Phase 3 that do not block go-live are logged as known observations and tracked to resolution in the post-launch monitoring period.
Test and QA PlanPage 4 of 4