Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Budget vs Actuals Reporting Automation

[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]

This document defines the complete testing and quality assurance sequence for the Budget vs Actuals Reporting automation. It covers all three test phases: unit testing of individual agents in isolation, integration and edge-case testing of agent handoffs and failure scenarios, and a final end-to-end production run that includes process owner sign-off. The FullSpec team runs Phases 1 and 2 in full; the process owner joins Phase 3. No phase begins until the previous phase has fully passed.

01Testing approach

Testing is structured as three sequential phases. Each phase must reach a full-pass state before the next phase begins. Partial passes or deferred failures are not acceptable gate conditions. The FullSpec team owns all execution in Phases 1 and 2. The process owner joins the FullSpec team in Phase 3 to validate the production run and record formal approval.

Phase 1: Unit Testing

Estimated time: 2 to 3 business days

Scope

Each agent tested in isolation: Actuals Sync Agent, Variance Analysis Agent, Report Distribution Agent

Environment

Sandbox only. No production QuickBooks data, no live Google Sheets budget file, no real Slack channels

Credential rule

Sandbox credentials exclusively. Production tokens must not be loaded in Phase 1 under any circumstances

Pass condition

All test cases T01 through T14 return the expected output with zero critical failures

Who runs it

FullSpec team only

Phase 2: Integration and Edge-Case Testing

Estimated time: 2 to 3 business days

Scope

Agent-to-agent handoffs across the full automated flow, plus edge cases covering duplicates, missing data, tool unavailability, and human timeout

Environment

Sandbox for all tool calls; a staging copy of the Google Sheets budget template populated with prior-period test data

Credential rule

Sandbox credentials only. The staging Google Sheet must not be linked to the live Drive folder

Pass condition

All integration cases T15 through T22 and edge cases E01 through E06 return expected outputs or graceful error states

Who runs it

FullSpec team only

Phase 3: End-to-End Production Test

Estimated time: 1 business day (coordinated with month-end close)

Scope

Single full production run traced end-to-end: period-close trigger through to stakeholder Slack notification, using live credentials and the live budget template

Environment

Production. Live QuickBooks OAuth token, live Google Sheets budget file, live Slack workspace

Credential rule

Production credentials active. Internal test addresses must be used for all Slack notifications; the test report record must be voided after the run

Pass condition

All cases T23 through T26 pass, including the final process owner review and approval case

Who runs it

FullSpec team, with the process owner present and signing off on case T26

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only for all Phase 1 tests. Do not load production QuickBooks OAuth tokens, live Google Sheets files, or real Slack workspace tokens at any point during this phase. All test data must be drawn from prior closed periods or synthetic datasets.

Actuals Sync Agent — Complexity: Moderate. Estimated build and unit-test time: 14 hours.

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: scheduled trigger fires on period-close date

Cron trigger fires at configured period-close datetime; QuickBooks sandbox returns a full P&L with 30 account lines

Actuals written to correct cells in sandbox Google Sheet; no formula rows overwritten; all 30 lines mapped

Row count in actuals column equals 30; formula cells unchanged; no error log entry

T02

QuickBooks API returns HTTP 401 (expired token)

OAuth access token expired before the sync call

Token refresh attempted automatically; sync retried once; if refresh fails, error alert triggered

Refresh attempt logged; sync completes on retry or error event fires within 60 seconds of failure

T03

QuickBooks API returns HTTP 429 (rate limit)

API rate limit hit mid-fetch

Workflow pauses for the retry-after interval and re-attempts the fetch; no partial write to Google Sheets

No partial data written; retry succeeds within configured back-off window; completion logged

T04

Unmapped account code in P&L response

QuickBooks returns a new account code not present in the mapping table

Unmapped line flagged in the error log; all other lines written correctly; sync completes without halting

Error log entry created for the unmapped code; mapped lines all written; agent status returns partial-success

T05

Google Sheets write fails mid-batch (API timeout)

Sheets API times out after writing 15 of 30 rows

Workflow detects incomplete write, rolls back any partial changes, and retries the full batch

Sheet returns to pre-run state after rollback; full retry completes successfully; no duplicate rows

T06

Empty P&L response from QuickBooks (zero-transaction period)

QuickBooks returns an empty dataset for the period

Sync agent halts gracefully, logs a warning, and sends an alert rather than writing blank values

No cells overwritten; warning log entry created; alert event fires to the configured notification channel

Variance Analysis Agent — Complexity: Moderate. Estimated build and unit-test time: 14 hours.

Scenario

Input

Expected Output

Pass Criteria

T07

Happy path: variance calculation across 30 lines, 4 lines exceed threshold

Populated sandbox sheet with budget and actuals columns; threshold set at 10%

Dollar and percentage variance calculated for all 30 lines; 4 flagged lines highlighted; Slack prompts queued for relevant department heads

All 30 variance cells populated; exactly 4 lines flagged; 4 Slack messages queued with correct cost-centre attribution

T08

No lines exceed variance threshold

All actuals within 10% of budget

No lines flagged; no Slack prompts sent; commentary column left blank; agent completes with clean status

Zero flagged lines; zero Slack messages queued; completion log shows clean run

T09

Budget cell is zero (division-by-zero scenario)

One budget line has a value of zero; actuals have a non-zero figure for the same line

Agent detects zero-budget denominator, skips percentage calculation for that line, flags it as a data-quality issue, and continues

Line flagged as data-quality error in log; dollar variance still calculated; percentage field returns N/A rather than crashing

T10

Prior-period trend data unavailable for commentary draft

Flagged line has no data in the prior-period reference columns

Commentary draft for that line omits trend language; variance size and direction still included

Commentary cell populated with variance-only text; no null or error value in cell; agent completes without halting

T11

Slack API unavailable when sending department head prompts

Slack returns HTTP 503 during prompt dispatch

Prompts queued for retry; agent logs the failure; variance calculation and commentary draft continue independently

Retry queue populated; agent does not halt; calculation and draft steps complete; Slack retry fires after back-off interval

Report Distribution Agent — Complexity: Simple. Estimated build and unit-test time: 10 hours.

Scenario

Input

Expected Output

Pass Criteria

T12

Happy path: finance manager approves commentary; report published and stakeholders notified

Approval signal received from finance manager in sandbox Google Sheet; all commentary cells populated

Report saved to sandbox Google Drive folder with correct date-stamped filename; Slack notification sent to test stakeholder channel with working link

File present in Drive with correct naming convention; Slack message delivered; link resolves to the correct file

T13

Google Drive API returns permission error on file creation

Service account lacks write permission on the target Drive folder

Agent logs the permission error, does not overwrite any existing file, and sends an alert to the configured admin address

No file created or overwritten; error log entry created; admin alert fired within 60 seconds

T14

Slack stakeholder notification fails after successful Drive publish

Drive publish succeeds; Slack API returns HTTP 500 on notification send

File remains published in Drive; Slack failure logged; retry attempted; if retry fails, manual notification flagged

Drive file intact; retry log entry present; fallback flag raised after two failed retry attempts

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests verify that data and control signals pass correctly between agents across the full automated flow. All tests run in the sandbox environment using the staging budget template. IDs continue from the Phase 1 sequence.

Scenario

Agents Under Test

Input

Expected Output

Pass Criteria

T15

Actuals Sync Agent completion signal correctly triggers Variance Analysis Agent

Actuals Sync Agent → Variance Analysis Agent

Actuals Sync Agent completes a full successful write to the sandbox sheet

Variance Analysis Agent starts automatically within 30 seconds of sync completion signal

Variance agent run log shows trigger timestamp within 30 seconds of sync completion; no manual intervention required

T16

Variance Analysis Agent passes flagged line data to Slack prompt dispatch correctly

Variance Analysis Agent → Slack

4 lines flagged in variance calculation; 3 distinct cost-centre owners mapped in configuration

Each cost-centre owner receives only the Slack prompt for lines attributed to their centre; no cross-contamination

3 Slack messages sent to 3 distinct test users; each message contains only the lines for that user's cost centre

T17

Slack replies from department heads are correctly ingested into commentary draft

Slack → Variance Analysis Agent → Google Sheets

Two of three department heads reply within the 24-hour window; one does not reply

Replied content pre-populated into commentary cells for those two lines; unanswered line marked as pending; all three lines visible in the sheet

Two commentary cells contain reply text; one cell marked pending; no blank or null cells; report not blocked by the missing reply

T18

Finance manager approval in Google Sheets correctly triggers Report Distribution Agent

Google Sheets → Report Distribution Agent

Finance manager sets the approval cell to Approved in the sandbox sheet

Report Distribution Agent fires within 60 seconds; Drive publish and Slack notification complete

Agent trigger log timestamp within 60 seconds of approval cell change; Drive file created; Slack notification sent

T19

Variance Analysis Agent does not fire if Actuals Sync Agent exits with a partial-success status

Actuals Sync Agent → Variance Analysis Agent

Actuals Sync Agent exits with partial-success due to unmapped account (T04 scenario)

Variance Analysis Agent is blocked; operator alert fired; workflow paused pending manual resolution of unmapped lines

Variance agent run log shows no start event; alert message delivered to configured admin contact

T20

Report Distribution Agent does not fire if commentary contains any blank non-pending cells

Google Sheets → Report Distribution Agent

Finance manager sets approval to Approved but one commentary cell is completely blank (not marked pending)

Distribution agent blocked; validation error raised; finance manager notified to resolve the blank cell before re-approving

No Drive file created; no Slack notification sent; validation error log entry present; finance manager alert fired

T21

End-to-end agent chain completes within acceptable runtime for a 50-line report

All three agents in sequence

Sandbox P&L with 50 account lines; 6 flagged variances; all department heads reply within window

Full chain from trigger to Slack notification completes within 15 minutes (excluding the human review window)

Total automated runtime log shows completion under 15 minutes; all intermediate agent logs show no errors

T22

Token refresh mid-chain does not break agent handoff

Actuals Sync Agent → Variance Analysis Agent

QuickBooks OAuth token expires between the sync call and the variance trigger

Token refresh completes silently; variance agent receives the correct trigger and runs normally

Token refresh event logged; variance agent start confirmed; no data loss or duplicate write

Edge-case tests below cover scenarios outside the normal flow that the automation must handle gracefully without halting the pipeline or silently corrupting data.

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate period-close trigger fires (e.g. scheduler misconfiguration sends two signals on the same date)

Two trigger events fire within 5 minutes for the same accounting period

Second trigger detected as a duplicate; workflow deduplication logic suppresses the second run; single sync completes

Exactly one sync run in the log for the period; second trigger flagged as duplicate; no double-write to Google Sheets

E02

QuickBooks P&L contains a duplicate account code (same code appears on two lines)

QuickBooks sandbox returns a P&L with account code 5100 appearing on two separate lines

Agent detects the duplicate, sums the values, writes the combined total to the mapped budget line, and logs a data-quality warning

Single row written for account 5100 with summed value; data-quality warning in log; no two rows written for the same code

E03

Google Sheets budget template has been manually restructured since the last run (column order changed)

Actuals column index has shifted by two columns due to a manual edit

Agent detects column header mismatch, halts the write, logs the structural change, and fires an alert rather than writing to wrong columns

No data written; structural mismatch logged; alert sent to configured admin; sheet left unchanged

E04

Slack is unavailable for the entire 24-hour department head response window

Slack API returns errors for the full 24-hour window; no department head prompts delivered

After the window expires, all flagged lines are marked as pending-no-response; commentary draft uses variance-only language; report is not blocked

All flagged lines show pending-no-response status; commentary cells populated with fallback text; approval gate opens on schedule

E05

Department head reply arrives after the configured 24-hour response window has closed

Slack reply received 26 hours after the prompt was sent

Late reply logged but not automatically merged into the commentary; finance manager notified of the late reply for manual consideration

Commentary cell retains the pre-close draft; late reply logged with timestamp; finance manager alert sent

E06

Finance manager does not approve or reject commentary within 48 hours of the variance agent completing

No action taken on the approval cell in Google Sheets for 48 hours

Escalation alert sent to the configured secondary contact (e.g. Operations Lead); workflow remains paused; no report published until approval received

Escalation alert fired at the 48-hour mark; secondary contact notified; workflow log shows paused state; no Drive file created

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test Slack channel and a test email address for all stakeholder notifications during this run. Do not send live Slack messages to real stakeholders or distribute the test report to the CEO or board. After the run completes and T26 is signed off, void the test report record in Google Drive by moving it to the designated QA archive folder and removing it from the live reporting folder. The live QuickBooks period must not be in an open or partially-reconciled state when this test fires.

Phase 3 is run against production credentials and the live budget template. The FullSpec team coordinates timing with the process owner to align this run with or immediately after a real period-close date. The process owner must be available for the commentary review and approval step (T26).

Scenario

Environment

Steps Verified

Expected Output

Pass Criteria

T23

Full production run: period-close trigger through to Google Sheets write

Production: live QuickBooks OAuth, live Google Sheets budget file

Scheduled trigger fires; QuickBooks P&L fetched for the closed period; actuals written to the budget sheet

All cost-centre lines written to correct cells; formula rows untouched; actuals column populated within 5 minutes of trigger

Row count matches the QuickBooks P&L line count; formula integrity check passes; run log shows no errors; completion within 5 minutes

T24

Variance calculation, Slack prompts, and commentary draft in production

Production: live Google Sheets, live Slack workspace (test channel only)

Variance agent reads the populated sheet; calculates all variances; flags lines exceeding threshold; sends Slack prompts to the test channel; pre-populates commentary draft

Variance figures match manual spot-check calculations for at least 3 spot-checked lines; flagged lines match expected threshold breaches; Slack prompt visible in the test channel; commentary draft cells populated

Manual spot-check of 3 randomly selected lines confirms dollar and percentage variance values; test Slack channel receives the prompt message; commentary cells contain non-blank draft text

T25

Error log verification across the full production run

Production: automation platform run log and Google Sheets audit tab

FullSpec team reviews the full run log after T24 completes; checks for any warning or error entries; confirms audit trail entries are present in the Google Sheets audit tab

Run log shows no critical errors; any warnings (e.g. a late Slack reply or a near-threshold line) are documented and explainable; audit tab in Google Sheets records the sync timestamp, agent version, and actuals-write confirmation

Zero critical errors in the run log; all warnings reviewed and accepted by the FullSpec team; audit tab contains at minimum: run date, period reference, row count written, and agent completion status

T26

Process owner review, commentary approval, report publish, and stakeholder notification — final sign-off

Production: live Google Sheets, live Google Drive, live Slack workspace (test channel for notification)

Process owner reviews the drafted commentary in the live budget sheet; edits any lines as required; sets the approval cell to Approved; Report Distribution Agent publishes the file to the QA archive folder (not the live folder); Slack notification fires to the test channel

Process owner confirms the commentary draft quality is acceptable; the published Drive file is correctly named and formatted; the Slack notification contains the correct period reference and a working link to the file; process owner verbally confirms approval to the FullSpec team

Approval cell change logged with timestamp; Drive file present in QA archive folder with correct date-stamp naming; Slack test channel notification received with working link; FullSpec team records the process owner's confirmation in the FullSpec project confirmation log, which serves as the formal go-live sign-off for this automation

After T26 passes, the FullSpec team moves the test report to the QA archive, restores the live Google Drive folder to its pre-test state, and confirms go-live readiness in writing to support@gofullspec.com. The automation is then cleared for the first live production cycle.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process