Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Cash Flow Forecasting Automation

[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance suite for the Cash Flow Forecasting automation. It covers all three testing phases: unit tests against individual agents in isolation, integration and edge-case tests across agent handoffs, and a final end-to-end production run with process owner sign-off. The FullSpec team runs Phases 1 and 2 entirely in sandbox environments. The process owner joins Phase 3 to verify production output and record approval. No phase begins until the previous phase has fully passed all test cases.

01Testing approach

Testing is structured across three sequential phases: Unit, Integration, and End-to-end. Each phase must reach a full pass before the next phase is initiated. A single failing test case blocks progression. Any defect found is logged, fixed, and re-tested from the start of the phase in which it was found. This sequential gate approach ensures that integration problems are never masked by unresolved unit failures, and that the production environment is only touched once isolated and integrated behaviour has been verified.

Phase 1 — Unit Testing

Each agent tested in isolation against sandbox credentials

Scope

Data Collection Agent, Categorisation Agent, and Distribution Agent tested individually. No cross-agent data flows exercised.

Environment

Sandbox only. Xero demo company, Stripe test mode, Google Sheets staging tab, Gmail sandbox address, Slack test workspace.

Credential rule

All sandbox credentials only. No production API keys, OAuth tokens, or live account identifiers may be used in this phase.

Pass condition

All test cases T01 through T14 return the expected output with no unhandled exceptions. All failure-mode cases produce the correct alert or fallback response.

Who runs it

FullSpec team only.

Phase 2 — Integration and Edge-Case Testing

Agent handoffs and boundary conditions tested end-to-end in staging

Scope

Data Collection to Categorisation handoff, Categorisation to Sheets write, and approval trigger to Distribution Agent. Plus a dedicated edge-case suite covering duplicates, missing data, tool outages, and approval timeout.

Environment

Staging environment using sandbox credentials and a cloned version of the production Google Sheet with test data only.

Credential rule

Sandbox credentials only. The staging Google Sheet must not be linked to any live stakeholder email addresses or the live Slack finance channel.

Pass condition

All integration cases T15 through T22 pass. All edge cases E01 through E07 produce the correct exception behaviour with no data corruption or unhandled crash.

Who runs it

FullSpec team only.

Phase 3 — End-to-End Production Test

Full live run verified by process owner before go-live

Scope

One complete production run from Monday schedule trigger through to Gmail delivery and Slack post. Error log verification and process owner review included.

Environment

Production. Live Xero account, live Stripe account, live Google Sheet, Gmail with real stakeholder addresses (temporarily replaced by internal test address for T23 and T24), and the live Slack finance channel.

Credential rule

Production OAuth tokens and API keys active. All credentials must be stored in the secrets manager before this phase begins. No hard-coded credentials permitted.

Pass condition

All cases T23 through T27 pass, including the process owner approval case T27 recorded in the FullSpec confirmation log.

Who runs it

FullSpec team runs T23 through T26. Process owner joins for T27 (review and approval).

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only. Do not connect to live Xero, Stripe, or Gmail accounts during Phase 1. Use the Xero demo company, Stripe test mode keys, a dedicated staging Google Sheet, and a test Gmail address. No real financial data should be read, written, or transmitted at this stage.

Data Collection Agent — Estimated build time: 8 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T01

Happy path: scheduled trigger fires on time

Monday 8 am cron trigger fires in sandbox

Agent initialises, authentication to Xero sandbox and Stripe test mode both succeed, run log entry created

Run log shows status 'started'; both auth calls return HTTP 200 within 10 seconds

T02

Happy path: Xero transactions fetched and paginated

Xero demo company with 120 transactions across 7 days, 2 pages of results

All 120 transactions returned in structured payload; page 2 fetched automatically after page 1

Payload transaction count equals 120; no pagination errors in log

T03

Happy path: AR ageing buckets fetched

Xero demo company with invoices in 0-30, 31-60, and 61-90 day buckets

Three AR bucket totals returned with correct invoice counts and amounts

Payload AR section contains three bucket objects; amounts match Xero demo values

T04

Happy path: upcoming bills fetched for 8-week window

Xero demo company with 9 bills due within the next 56 days

9 bill records returned with amounts and due dates; bills outside 56-day window excluded

Payload bills array length equals 9; no bill with due date beyond 56 days present

T05

Happy path: Stripe payout schedule fetched

Stripe test mode account with 3 scheduled payouts

3 payout records returned with amounts and expected arrival dates

Payload payout array length equals 3; arrival dates are future-dated relative to test run date

T06

Failure mode: Xero OAuth token expired

Expired OAuth access token injected into sandbox config

Agent attempts token refresh using stored refresh token; on success, retries the Xero call; on refresh failure, posts alert to Slack staging channel and halts

Xero call retried once after refresh; if refresh fails, Slack alert sent within 60 seconds and run marked 'failed'

T07

Failure mode: Xero API rate limit hit mid-run

Xero sandbox configured to return HTTP 429 after the first API call

Agent pauses for the retry-after period specified in the response header, then retries; up to 3 retries before halting with alert

Retry attempts logged with timestamps; alert fired if all 3 retries exhausted; no duplicate records in payload

T08

Failure mode: Stripe API returns 503 unavailable

Stripe test mode endpoint mocked to return 503

Agent logs the Stripe failure, skips the payout block, flags the payload with 'stripe_unavailable: true', and continues the run with a partial dataset

Payload flag 'stripe_unavailable' set to true; Xero data present; Slack staging alert sent noting partial data

Categorisation Agent — Estimated build time: 8 hours. Complexity: Complex.

Scenario

Input

Expected output

Pass criteria

T09

Happy path: high-confidence transaction matched to chart-of-accounts rule

Transaction description 'AWS Monthly Invoice' against known rule mapping to 'Software and Hosting'

Transaction assigned category 'Software and Hosting'; confidence score above threshold; no flag raised

Category field populated correctly; flagged field is false; confidence score logged

T10

Happy path: full batch of 120 transactions categorised

120-transaction payload from T02 passed to agent

All 120 transactions assigned a category; low-confidence items flagged for review

Output array length equals 120; every item has a non-null category; flagged items contain a reason string

T11

Happy path: low-confidence transaction flagged rather than guessed

Transaction description 'Ref 88291A' with no matching chart-of-accounts rule

Transaction added to flagged list with reason 'no matching rule'; category field set to null

Category field is null; flagged is true; reason string present; item appears in flagged summary

T12

Failure mode: empty transaction payload received

Payload passed to agent with transactions array of length 0

Agent detects empty payload, logs a warning, writes an empty dataset to the staging Sheet tab with a 'no transactions' note, and does not proceed to distribution

Google Sheets staging tab updated with 'no transactions' marker; no PDF export triggered; warning in run log

T13

Failure mode: malformed transaction record in payload

One transaction record missing the 'amount' field

Agent skips the malformed record, logs it with field name and record ID, flags it for human review, and continues categorising the remainder

Skipped record appears in error log with field name 'amount'; remaining records categorised correctly

Distribution Agent — Estimated build time: 6 hours. Complexity: Moderate.

Scenario

Input

Expected output

Pass criteria

T14

Happy path: approval cell set to 'Approved' triggers distribution

Google Sheets staging tab approval cell updated to 'Approved'

Agent detects cell change within polling interval, exports PDF from summary tab, sends email to test address, posts Slack message to staging channel

PDF attachment present in test inbox; Slack message posted within 2 minutes of approval; run log shows 'distribution complete'

T15

Happy path: Gmail send succeeds with correct subject line and attachment

PDF generated from staging Sheet; test recipient address used

Email delivered with subject line matching template, PDF attached, body text populated from standard commentary template

Email received at test address; subject line correct; attachment opens without error; body text non-empty

T16

Failure mode: Gmail API returns 429 rate limit during send

Gmail API mocked to return 429 on first attempt

Agent waits the back-off period and retries up to 3 times; if all retries fail, logs error and alerts via Slack staging channel

Retry count logged; email eventually delivered if retry succeeds; Slack alert fired if all retries exhausted

T17

Failure mode: Slack post fails with channel not found error

Slack staging channel ID set to an invalid value

Agent logs the Slack error, does not halt the run, and sends an internal alert to the FullSpec monitoring channel with the error detail

Gmail send completed successfully regardless; Slack error present in run log; monitoring alert received

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests verify that data passes correctly between agents and that the full automated chain from trigger to Sheets write behaves as designed. All tests use the staging environment and sandbox credentials. Test IDs continue from Phase 1.

Scenario

Input

Expected output

Pass criteria

T18

Data Collection to Categorisation handoff: payload passes with correct schema

Full Monday trigger run in staging environment; Data Collection Agent completes successfully

Categorisation Agent receives payload within 30 seconds; all required fields present: transactions, ar_buckets, bills, payouts

Categorisation Agent starts processing without schema validation errors; all four payload sections non-null

T19

Categorisation to Google Sheets write: categorised data written to correct weekly columns

Categorised dataset of 120 transactions with AR, bills, and payout figures

All figures written into the correct weekly columns of the staging Sheet; existing formulas not overwritten; chart data ranges intact

Staging Sheet column values match agent output; formula cells recalculate correctly; no #REF or #VALUE errors present

T20

Flagged items appear in review section of staging Sheet

5 transactions flagged by Categorisation Agent

Flagged items written to the 'Review Required' tab with description, amount, and reason for each

Review tab contains exactly 5 rows; each row has a non-empty reason string; no flagged item appears in the main forecast columns

T21

Approval trigger detected by Distribution Agent after manual cell update

Finance lead role simulated: staging Sheet approval cell set to 'Approved' after categorisation step completes

Distribution Agent detects change within one polling cycle (maximum 60 seconds), initiates PDF export

PDF export starts within 60 seconds of cell change; run log event 'approval_detected' timestamped correctly

T22

Full three-agent chain completes without manual intervention in staging

Monday schedule trigger fired in staging; no manual steps taken

Data Collection, Categorisation, and Sheets write complete sequentially; run pauses correctly at the approval gate awaiting finance lead action

Run log shows three sequential agent completion events; status shows 'awaiting_approval' before Distribution Agent starts; no agent fails

Edge-case tests below cover boundary conditions, unexpected inputs, and tool unavailability scenarios. These cases use IDs in the E series.

Scenario

Input

Expected output

Pass criteria

E01

Duplicate transaction records in Xero payload

Xero sandbox returns two identical transaction records with the same ID and amount

Categorisation Agent deduplicates on transaction ID; only one instance written to Sheets; duplicate logged in run log

Sheets write contains one instance of the duplicated transaction; run log entry 'duplicate_skipped' with transaction ID present

E02

Missing AR ageing data: Xero returns empty AR response

Xero AR endpoint returns an empty array

Agent writes zero values to AR bucket columns in Sheets; flags 'AR data unavailable' in the review section; does not halt run

AR columns in staging Sheet show zero; Review tab contains 'AR data unavailable' flag; run continues to completion

E03

Stripe payout data missing because Stripe is in sandbox mode with no test payouts configured

Stripe test account returns empty payouts list

Agent writes zero to payout rows; payload flag 'no_payouts_found' set; Finance Lead review tab notes absence

Payout rows in Sheet show zero; flag logged; no error thrown; run completes normally

E04

Google Sheets API unavailable during write step

Sheets API mocked to return HTTP 503 for all write calls

Categorisation Agent retries write up to 3 times with exponential back-off; after all retries fail, sends alert to monitoring channel and halts run with status 'sheets_write_failed'

Three retry attempts logged with timestamps; run status set to 'sheets_write_failed'; monitoring alert received; no partial write to Sheet

E05

Finance lead does not approve the forecast within 48 hours

Approval cell left unchanged for 48 hours after Categorisation Agent completes

Distribution Agent sends a reminder notification to the finance lead via Slack after 24 hours; after 48 hours, posts an escalation alert to the business owner Slack handle and logs 'approval_timeout'

Slack reminder received by finance lead handle at 24-hour mark; escalation alert received at 48-hour mark; run log shows 'approval_timeout' event

E06

Transaction amount field contains a non-numeric value

One transaction record with amount field set to the string 'N/A'

Categorisation Agent skips the record, logs field name and value, flags it for human review with reason 'invalid_amount_format'

Record absent from Sheets write; Review tab entry present with reason 'invalid_amount_format'; remaining transactions unaffected

E07

Categorisation Agent confidence scoring returns null for all transactions due to empty rules set

Chart-of-accounts rules set loaded as empty array

All transactions flagged for human review; none auto-categorised; finance lead notified via Slack that manual categorisation is required for the full batch; run does not halt

All transactions appear in Review tab; Slack notification sent referencing empty rules set; main forecast columns contain only zero or formula values; no crash

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test email address (not the live stakeholder distribution list) for T23 and T24 so that no draft or partial forecast reaches external recipients. Post the Slack summary to a private test channel rather than the live finance channel. After T23 and T24 complete, void or delete the test forecast entry in the production Google Sheet before T25 begins the full verified run. Confirm with the FullSpec team that the test record has been removed before proceeding. Do not run Phase 3 during a live Monday window when the real forecast is also expected.

Phase 3 uses live production credentials, the real Xero account, the real Stripe account, and the production Google Sheet. The process owner joins this phase to verify the output and record approval. All test cases in this phase continue the T sequence from Phase 2. Phase 3 may only begin once all Phase 1 and Phase 2 cases have passed without exception.

Scenario

Input

Expected output

Pass criteria

T23

Production trigger fires and data collection completes against live Xero and Stripe

Monday schedule triggered manually by FullSpec team at agreed test time; live Xero and Stripe credentials active

All live transactions, AR ageing, bills, and Stripe payouts fetched and assembled into payload; run log records exact fetch timestamps and record counts

Payload record counts are non-zero and plausible against known account state; no authentication errors; run log timestamps present for each API call

T24

Categorisation writes correctly to the production Google Sheet without breaking existing formulas

Live payload from T23 passed to Categorisation Agent; production Sheet used

Categorised data written to correct weekly columns; all existing formulas recalculate; flagged items appear in Review Required tab; no formula errors introduced

Finance lead confirms column values look correct for the known week; no #REF, #VALUE, or #NAME errors visible; Review tab populated if any low-confidence transactions present

T25

Error log verification: all agent run events recorded correctly in the run log

Run log examined after T23 and T24 complete

Log contains entries for: trigger fired, Xero fetch complete, Stripe fetch complete, categorisation complete, Sheets write complete; each entry includes timestamp, record count, and status

All five log entry types present; no entries show status 'error' or 'failed'; timestamps are sequential and consistent with elapsed run time

T26

Full distribution: PDF exported and delivered to internal test address; Slack summary posted to private test channel

Finance lead approves staging approval cell; Gmail recipient set to internal test address; Slack channel set to private test channel

PDF exported from production summary tab; email delivered to internal test address with correct subject and attachment; Slack message posted with correct opening balance, 4-week projection, and any cash risk flags

Email received within 5 minutes of approval; PDF opens correctly and displays the current week's figures; Slack message format matches the approved template; no external stakeholders receive email at this stage

T27

Process owner review and approval: owner reviews production output, confirms accuracy, and records approval in the FullSpec confirmation log

Process owner (or nominated finance lead) reviews the production Google Sheet, the PDF delivered in T26, and the Slack summary; compares figures against known Xero and Stripe balances

Process owner confirms figures are accurate, distribution format is acceptable, and the automation is ready for live operation; approval recorded in the FullSpec confirmation log with name, date, and any noted conditions

FullSpec confirmation log entry created showing process owner name, approval date, and status 'approved for go-live'; any conditions or follow-up items noted in the log; no separate signoff block required

Once T27 is recorded in the FullSpec confirmation log, the automation is cleared for live Monday operation. The FullSpec team will update the process status to 'live' and notify the finance lead that the first fully automated forecast run will execute on the following Monday at 8 am. Any issues found after go-live should be reported to support@gofullspec.com with the run log reference.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process