Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Financial Reporting Automation

[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]

This document is the authoritative quality assurance reference for the Financial Reporting automation. It defines every test case the FullSpec team runs before any component reaches production, and it specifies the exact conditions under which the process owner joins testing. Testing is structured in three sequential phases: unit tests against each agent in isolation, integration and edge-case tests across agent handoffs and failure scenarios, and a final end-to-end run in the production environment. No phase begins until every test case in the previous phase achieves a passing result.

01Testing approach

The three phases run in strict sequence. A single failing test case blocks progression to the next phase. All remediation work is completed and the full failing phase is re-run from T01 of that phase before advancement. This prevents integration or production issues that are caused by unresolved unit-level defects.

Phase 1: Unit Testing

Each agent tested in isolation against sandbox credentials

Scope

Data Fetch Agent, Variance Analysis Agent, and Distribution Agent tested independently. No cross-agent handoffs exercised in this phase.

Environment

Sandbox only. Xero demo company, HubSpot sandbox portal, Google Sheets test workbook, Google Drive test folder, Slack test channel.

Credential rule

Sandbox API tokens only. Production OAuth tokens and production Slack webhooks must not be loaded in the automation platform during Phase 1.

Pass condition

All cases T01 through T12 return the expected output with no manual intervention. Zero unhandled exceptions in the run log.

Who runs it

FullSpec team only. Process owner is not required for Phase 1.

Phase 2: Integration and Edge-Case Testing

Agent handoffs and failure scenarios tested end-to-end in sandbox

Scope

Full agent chain from schedule trigger through to Slack delivery. Edge cases including duplicate runs, missing data fields, unavailable API endpoints, and human approval timeout.

Environment

Sandbox for all tool connections. Deliberate fault injection used for unavailability and timeout scenarios.

Credential rule

Sandbox credentials only. No production tokens at any point in Phase 2.

Pass condition

Integration cases T13 through T20 and edge cases E01 through E06 all pass. Error handling paths produce the correct alert or fallback behaviour with no silent failures.

Who runs it

FullSpec team only. Process owner is not required for Phase 2.

Phase 3: End-to-End Production Test

Single supervised run in the live environment; process owner participates

Scope

One complete report cycle executed in production: schedule trigger fires, all three agents run in sequence, finance manager reviews and approves, PDF is saved to Drive, and Slack notification is delivered to the live finance channel.

Environment

Production. Live Xero organisation, live HubSpot portal, production Google Sheets report template, production Drive folder, live Slack channel. Test record is voided after the run.

Credential rule

Production OAuth tokens and webhooks are loaded for this phase only. Confirm sandbox credentials have been replaced in all environment variables before the run begins.

Pass condition

Cases T21 through T24 all pass, including the final process owner sign-off case (T24). FullSpec records approval confirmation internally.

Who runs it

FullSpec team leads the run. The process owner (Finance Manager) joins for the review gate step (T23) and the final approval case (T24).

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only. Before running any Phase 1 case, confirm that all environment variables point to the Xero demo company, the HubSpot sandbox portal, and the Google Sheets test workbook. Production tokens must not be present in the automation platform configuration at this stage. Any test that inadvertently writes to a production sheet must be treated as a blocker and the credential configuration reviewed before testing continues.

Data Fetch Agent — unit tests

Scenario

Input

Expected output

Pass criteria

T01

Happy path: full data fetch for a complete period

Scheduled trigger fires for a closed monthly period with all Xero and HubSpot data present

P&L, balance sheet, and cash-flow figures written to correct cells in the Google Sheets test workbook; HubSpot pipeline totals written to the designated pipeline section; sheet formulas recalculate without error

All target cells populated; formula totals match manually verified source figures; run log shows success with no warnings

T02

Xero OAuth token valid; correct scopes granted

API call to Xero with a token scoped to accounting.reports.read and accounting.transactions

Xero returns HTTP 200 with report payload; agent parses P&L, balance sheet, and cash-flow blocks correctly

Three data blocks retrieved; field count matches expected schema; no 401 or 403 error in log

T03

Xero API returns 401 Unauthorized (expired token)

Trigger fires but the stored Xero OAuth token has expired

Agent logs a credential error, does not write partial data to the sheet, and raises an alert to the configured error notification channel

No cells written; error logged with code 401; alert dispatched; run marked as failed, not completed

T04

Xero returns incomplete period data (period not yet closed)

Trigger fires for a period where Xero has not yet finalised the month-end figures

Agent detects that the balance sheet total does not match the P&L net position; logs a data-integrity warning and halts before writing to the sheet

No cells written; warning entry in run log identifying the mismatch; downstream agents not triggered

T05

HubSpot API call: closed-won and pipeline totals for the period

API call to HubSpot Deals endpoint filtered by close date and deal stage

Closed-won revenue total and open pipeline total returned; written to the two designated HubSpot cells in the test workbook

Both cells populated with numeric values; values match the manually exported HubSpot report for the same period

T06

HubSpot returns zero records (no deals closed in period)

HubSpot API returns an empty deals array for the reporting period

Agent writes $0 to the closed-won cell and $0 to the pipeline cell; run log records zero-result warning; does not halt the run

Cells show $0; run continues to Variance Analysis Agent; zero-result warning present in log

T07

Google Sheets write: all target cells populated and formulas recalculate

Complete data payload from T01 submitted to the Sheets API

All mapped cells updated; sheet-level formula recalculation triggered; no formula errors (#REF, #DIV/0, #VALUE) present after write

Sheets API returns HTTP 200 for all write operations; formula audit shows zero errors; summary totals match source data

T08

Google Sheets API returns 403 (insufficient scope on service account)

Agent attempts to write to the sheet with a service account missing the spreadsheets scope

Agent logs a permission error, does not write partial data, and raises an alert

No cells modified; error logged with scope detail; alert dispatched

Variance Analysis Agent — unit tests

Scenario

Input

Expected output

Pass criteria

T09

Happy path: variances detected above threshold and commentary written

Populated test workbook where three line items exceed the configured 10% variance threshold

Variance Analysis Agent identifies the three flagged lines, writes a plain-language commentary block to the report summary section, and highlights the flagged rows

Commentary block present in the summary cell range; three line items flagged; commentary references correct account names and variance percentages

T10

No variances above threshold

Populated test workbook where all actual figures are within 10% of budget

Agent writes a standard no-material-variances statement to the commentary block; no lines flagged; run continues to Distribution Agent

Commentary block present; no lines highlighted; run log shows agent completed with zero flags

T11

Sheet not yet populated when Variance Analysis Agent fires (upstream failure)

Agent fires but the Data Fetch Agent failed silently and target cells are empty

Variance Analysis Agent detects empty cells in the data range, logs a dependency error, and halts without writing commentary

No commentary written; error logged identifying empty cell range; downstream Distribution Agent not triggered

T12

Threshold boundary: variance exactly at the configured 10% limit

One line item with an actual figure exactly 10.0% above budget

Agent flags the line item (threshold is greater than or equal to 10%); commentary includes the line

Line is flagged; commentary references the item; boundary condition correctly handled as a flag, not ignored

Distribution Agent — unit tests

For T13 and beyond, verify that the Slack webhook URL is pointed at the sandbox test channel and that the Google Drive folder ID references the test folder, not the production reporting folder.

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests — agent handoffs

Scenario

Input

Expected output

Pass criteria

T13

Distribution Agent: happy path PDF export to Google Drive

Completed and approved Google Sheets report in the test workbook

Agent exports the sheet as a PDF using the Sheets API export endpoint; PDF saved to the test Drive folder with the correct filename format (YYYY-MM_FinancialReport.pdf)

PDF present in test Drive folder; filename matches the naming convention; file size greater than 50 KB; no Drive API errors in log

T14

Distribution Agent: Slack notification delivered with correct link and summary

PDF saved to test Drive folder (output of T13); variance commentary available

Agent posts a Slack message to the test channel containing the Drive file link, the report period, and the top two variance highlights from the commentary block

Slack message appears in test channel within 30 seconds; link resolves to the correct Drive file; variance highlights match the commentary written by the Variance Analysis Agent in T09

T15

Full chain: schedule trigger through to Slack delivery with no variances

Trigger fires for a period where all figures are within threshold (as per T10 input)

Data Fetch Agent populates sheet; Variance Analysis Agent writes no-material-variance statement; Distribution Agent exports PDF and posts Slack message; approval gate fires automatically after review window (simulated as 1 minute for sandbox)

All three agents complete without error; Slack message delivered; PDF in Drive; run log shows sequential agent completion times with no gaps or retries

T16

Full chain: schedule trigger through to Slack delivery with flagged variances and finance manager approval

Trigger fires for a period with three line items above threshold (T09 input); finance manager approval action sent via the approval mechanism

Data Fetch Agent populates sheet; Variance Analysis Agent flags and writes commentary; Distribution Agent waits for approval signal; on approval, exports PDF and posts Slack message

Distribution Agent does not fire until approval signal is received; Slack message confirms report approved and distributed; all agents log completed status

T17

Approval gate: distribution fires automatically after review window lapses with no rejection

Trigger fires with flagged variances; no approval or rejection action taken within the configured review window (set to 5 minutes for sandbox test)

After the review window lapses, the Distribution Agent fires automatically; Slack message notes that the report was auto-distributed after the review window; PDF saved to Drive

Distribution fires after exactly the configured window; Slack message contains the auto-distribution notice; log records the lapse event with timestamp

T18

Finance manager rejects the report during the review window

Trigger fires with flagged variances; finance manager sends a rejection signal within the review window

Distribution Agent does not fire; Slack alert sent to the finance manager confirming the report has been held; run marked as rejected in the log; no PDF exported

No PDF in Drive; no stakeholder Slack message; rejection alert delivered to finance manager; run log status shows rejected

T19

Google Drive API unavailable when Distribution Agent attempts PDF export

Distribution Agent fires but the Drive API returns HTTP 503

Agent retries three times with exponential backoff (30s, 60s, 120s); after three failures, logs a critical error and sends an alert to the error notification channel; does not post to Slack

Three retry attempts logged with timestamps; critical error entry present; no Slack message sent; alert delivered to error channel within 5 minutes of final retry failure

T20

Slack API unavailable when Distribution Agent attempts notification

PDF successfully saved to Drive but Slack API returns HTTP 503 on the webhook call

Agent retries the Slack call twice; after failure, logs the Slack delivery error but does not delete or re-export the PDF; sends alert to error notification channel with the Drive file link so the report can be distributed manually

PDF confirmed present in Drive; Slack delivery error logged; error alert contains the Drive link; report is not lost

Edge-case tests

Scenario

Input

Expected output

Pass criteria

E01

Duplicate trigger: schedule fires twice for the same reporting period

Two trigger events arrive within 2 minutes for the same period identifier

The second trigger is detected as a duplicate via the period lock check; second run is suppressed; a warning is logged; only one report is produced

Single PDF in Drive for the period; single Slack message; duplicate-suppression warning in run log; no data overwritten by the second run

E02

Missing budget data in Google Sheets (no budget row for a line item)

Data Fetch Agent populates actuals; Variance Analysis Agent reads the sheet and finds one line item with no corresponding budget figure

Agent skips the variance calculation for the missing budget line; notes the missing budget line in the commentary block with a prompt for the finance manager to add the budget figure; does not error out

Commentary includes a note identifying the missing budget line by name; all other variance calculations complete normally; run does not halt

E03

Xero API rate limit hit mid-fetch (HTTP 429)

Data Fetch Agent makes multiple API calls and receives HTTP 429 after the second call

Agent pauses for the retry-after period specified in the response header (default 60 seconds), then retries; if three consecutive 429 responses are received, the run is aborted and an alert is raised

Retry behaviour confirmed in log with correct wait intervals; if aborted, alert delivered and no partial data written to sheet

E04

HubSpot deal currency mismatch (non-USD deal values present)

HubSpot returns deals with amounts in AUD and GBP alongside USD deals

Agent flags the currency mismatch in the run log; writes only USD-denominated deals to the pipeline cell; inserts a currency-mismatch warning into the commentary block

Non-USD values excluded from pipeline total; warning present in commentary and log; finance manager can see which deals were excluded

E05

Finance manager approval notification not delivered (Slack DM fails)

Variance Analysis Agent flags variances and the system attempts to send the approval-request DM to the finance manager via Slack; Slack DM API returns an error

System falls back to sending the approval request to the finance channel (tagged mention); logs the DM failure; review window timer starts from the fallback delivery time

Approval request visible in finance channel with tagged mention; DM failure logged; review window timer reflects fallback delivery timestamp

E06

Google Sheets formula error introduced by a layout change in the template

Data Fetch Agent writes to cell references that have shifted because a row was inserted in the template manually

Agent detects a #REF or #VALUE error in at least one formula cell after writing; logs a cell-mapping error identifying the affected references; halts the run before Variance Analysis Agent fires; alert sent to the error channel

Run halted before variance analysis; specific affected cell references named in the log; alert delivered; no commentary or PDF produced until the cell mapping is corrected

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: use an internal test address and an internal Slack channel for the initial production run so no draft report reaches external stakeholders. Confirm with the FullSpec team that the Slack webhook is pointed at an internal finance-team-only channel before the trigger fires. After the run completes and T24 is signed off, void the test record: delete the test PDF from the production Drive folder, archive the test report tab in Google Sheets, and remove the Slack message from the channel. Do not leave a test report in the production Drive folder where it could be retrieved by a board member or lender.

All cases below use live production credentials and the live Google Sheets report template. The FullSpec team confirms that Phase 1 and Phase 2 have passed in full before this phase begins. The process owner, in the role of Finance Manager, joins the session from case T23 onward.

Scenario

Input

Expected output

Pass criteria

T21

Production trigger fires and Data Fetch Agent completes successfully

Scheduled trigger fires in the production automation platform for the current or most recently closed reporting period; live Xero OAuth token and live HubSpot API key in use

P&L, balance sheet, and cash-flow figures retrieved from the live Xero organisation; HubSpot pipeline totals retrieved from the live portal; all figures written to the correct cells in the production Google Sheets report; formula totals recalculate without errors

All mapped cells populated with live figures; formula audit shows zero errors; figures cross-checked by the FullSpec team against a manual Xero export of the same period; discrepancy tolerance is zero for any figure above $100

T22

Variance Analysis Agent runs on production data and produces commentary

Production sheet populated in T21; live budget figures present in the budget column

Variance Analysis Agent reads the production sheet; identifies all line items above the configured 10% threshold; writes a commentary block to the production summary section; flagged rows highlighted in the sheet

Commentary block present and readable; each flagged line item named with its actual figure, budget figure, and percentage variance; commentary is factually consistent with the figures in the sheet; no lorem ipsum or placeholder text

T23

Finance manager reviews draft report and approves distribution

Commentary and flagged items from T22 visible in the production sheet; finance manager (process owner) joins the session and reviews the draft

Finance manager reviews the variance commentary and flagged line items; if satisfied, sends the approval signal through the configured review mechanism; Distribution Agent holds until this signal is received

Distribution Agent remains in a waiting state until the finance manager approval signal is received; FullSpec team confirms the approval event is logged in the run log with the approver identity and timestamp; finance manager confirms the commentary is accurate and the figures match their expectations

T24

Distribution Agent exports PDF to Drive and posts Slack notification; process owner confirms acceptance

Approval signal received in T23; production Google Sheets report complete and approved

Distribution Agent exports the production report as a PDF to the production Google Drive reporting folder with the correct filename; Slack message posted to the internal finance channel with the Drive link and variance highlights; process owner reviews the delivered Slack message and the Drive PDF and confirms the automation output meets the required standard for live operation

PDF present in the correct production Drive folder with the correct filename; Slack message delivered to the correct channel within 60 seconds of export completion; message contains the Drive link and at least one variance highlight; process owner verbally or in writing confirms acceptance; FullSpec team records the process owner confirmation in the FullSpec project confirmation log; no separate signature block is required. Post-test: FullSpec team and process owner void the test record per the instructions in the note above before closing the session.

On completion of T24, the FullSpec team logs the process owner confirmation internally and marks the QA plan as passed. The automation is cleared for live scheduled operation from the next reporting period. Any issue identified during Phase 3 that cannot be resolved within the same session returns the process to Phase 2 remediation before a second Phase 3 attempt is scheduled. Contact the FullSpec team at support@gofullspec.com for any questions about test results or remediation scope.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process