Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Bank Reconciliation Automation

[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]

This document defines the full quality assurance process for the Bank Reconciliation automation, covering all three test phases from isolated unit tests through to a live production run. It is written for the FullSpec engineering team who will execute Phases 1 and 2, and for the process owner who joins Phase 3 to confirm the automation meets operational requirements before the manual process is retired. Every test case must achieve a pass status before the following phase begins. No phase is skipped and no phase starts until the previous one fully passes.

01Testing approach

Testing is structured as three sequential phases: Unit, Integration, and End-to-end. No phase begins until every test case in the preceding phase carries a pass status. Phase 1 tests each agent in complete isolation using sandboxed credentials and synthetic data. Phase 2 tests the handoffs between agents and exercises edge cases against controlled inputs. Phase 3 runs a single supervised end-to-end cycle in the production environment with real credentials and real transaction data, concluding with formal process owner sign-off recorded in the FullSpec confirmation system.

Phase 1: Unit Testing

Each agent tested in isolation, synthetic data only

Scope

Individual agent logic: Bank Feed Sync Agent, Matching and Exceptions Agent, Notifications and Close Agent

Environment

Sandbox only. Plaid sandbox, Xero demo company, Google Sheets test workbook, Slack test workspace, Gmail test account

Credential rule

No production API keys permitted. All credentials must be sandbox or developer-tier tokens scoped to test environments

Pass condition

All cases T01 through T15 return expected outputs with no unhandled exceptions or data loss

Who runs it

FullSpec team only

Phase 2: Integration and Edge Case Testing

Agent handoffs and failure-mode coverage

Scope

Cross-agent handoffs, full workflow data flow, edge cases including duplicates, missing fields, tool unavailability, and human timeout

Environment

Sandbox environment with injected edge-case payloads. No production data used

Credential rule

Sandbox credentials only. Plaid sandbox tokens, Xero demo tenant, test Slack channel, test Gmail alias

Pass condition

Integration cases T16 through T22 and edge cases E01 through E06 all pass; no data written to production systems

Who runs it

FullSpec team only

Phase 3: End-to-End Production Test

Single supervised live run with process owner present

Scope

Full production run traced from scheduled trigger through Xero period close, including error log review and exception handling

Environment

Production. Live Plaid connection, live Xero organisation, live Google Sheets workbook, live Slack channel, live Gmail

Credential rule

Production credentials active. Plaid access token must be rotated and confirmed before Phase 3 begins. Xero OAuth token must carry reconciliation write scope

Pass condition

Cases T23 through T26 all pass; process owner confirms outputs match expectations and signs off in the FullSpec confirmation system

Who runs it

FullSpec team executes; process owner reviews outputs and records approval (T26)

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only for all Phase 1 tests. Do not use production API keys, live Plaid access tokens, or the live Xero organisation at any point during this phase. All test data must be synthetic. Any accidental write to a production system must be reported to the FullSpec team lead immediately and the affected record reversed before testing continues.

Bank Feed Sync Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: scheduled pull returns transactions

Plaid sandbox returns 25 normalised transactions for the test date range; Xero demo returns 22 ledger entries

Combined dataset of 47 records written to the staging payload with no nulls in amount, date, or reference fields

All 47 records present; schema validates against the normalisation spec; no error events logged

T02

Manual out-of-cycle trigger

Manual trigger fired via the orchestration layer with a custom date range override

Agent fires immediately, pulls only transactions within the specified range, and writes the same normalised output

Payload date range matches the override; no duplicate records from the previous scheduled run

T03

Plaid returns zero new transactions

Plaid sandbox configured to return an empty transaction array for the period

Agent logs a 'no new transactions' status event and halts gracefully without triggering downstream agents

No empty payload passed downstream; status event recorded; no error thrown

T04

Plaid API authentication failure

Invalid Plaid access token injected

Agent catches the 401 response, logs the credential error, and sends an alert to the FullSpec error channel

No partial payload written; error log entry created with timestamp and error code; downstream agents not triggered

T05

Xero API timeout on ledger fetch

Xero sandbox configured to return a 504 timeout on the transactions endpoint

Agent retries up to three times with exponential backoff, then logs the failure and halts

Three retry attempts visible in logs; final error event created; no data written downstream

Matching and Exceptions Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T06

Happy path: all transactions match exactly

20 bank transactions each with an exact amount and reference match in the Xero ledger dataset

20 matched pairs tagged as 'confirmed'; all 20 posted to the Xero demo company; exceptions sheet row count remains zero

Xero demo shows 20 newly reconciled entries; Google Sheets exceptions tab contains no new rows

T07

Probable match: amount matches, reference differs slightly

5 transactions where bank reference contains a truncated version of the ledger description (e.g. 'INV-0042' vs 'Invoice 0042')

All 5 tagged as 'probable match' and written to the exceptions sheet with confidence score and suggested pairing

Exceptions sheet contains 5 rows; each row includes bank reference, ledger reference, amount, and confidence score above the fuzzy threshold

T08

Genuinely unmatched bank transaction

3 bank transactions with no corresponding ledger entry within the date and amount tolerance

All 3 written to the exceptions sheet as 'unmatched'; no entry posted to Xero

Xero demo unchanged for these 3 records; exceptions sheet shows 3 rows with status 'unmatched'

T09

Google Sheets write failure

Exceptions sheet API call returns a 403 permission error

Agent logs the write failure, retries once, then halts the run and sends an error alert

Error event logged with HTTP status and sheet ID; no data silently dropped; downstream Notifications agent not triggered

T10

Multi-currency transaction normalisation

3 GBP-denominated bank transactions present in a USD-base Xero demo company

Transactions converted to USD at the rate embedded in the Plaid payload; matched against the Xero base-currency ledger entry

Converted amounts within 0.01 of the Xero ledger value; currency conversion event logged for each record

Notifications and Close Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T11

Happy path: Slack summary sent after full match

Matching agent completes with 20 matched pairs and zero exceptions; trigger fires for Notifications agent

Slack message posted to the test finance channel with matched count (20), exception count (0), and closing balance

Slack message arrives within 30 seconds; all three data points present and correct

T12

Slack summary includes exception link

Matching agent completes with 15 matched pairs and 3 exceptions

Slack message includes matched count (15), exception count (3), and a direct URL to the Google Sheets exceptions tab

URL in Slack message resolves to the correct sheet; exception count matches the number of rows in the sheet

T13

Gmail confirmation sent to finance lead

Exceptions marked as resolved in Google Sheets; agent fires Gmail step

Email sent to the test Gmail alias with subject line 'Reconciliation complete: [period]' and a summary body

Email received at test alias within 60 seconds; subject line and body contain the correct period label and matched totals

T14

Xero period close record posted

Exceptions resolved flag set to true in the orchestration context; Xero close step fires

A reconciliation record written to the Xero demo company with status 'closed', matched total, and timestamp

Xero demo shows the period close record; record timestamp is within 5 seconds of the exceptions-resolved event

T15

Slack API rate limit hit

Slack sandbox configured to return a 429 Too Many Requests response on the first call

Agent waits for the retry-after value in the response header, then resends the message successfully

Slack message delivered on the retry; rate limit event logged; no duplicate messages sent

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Phase 2 integration tests confirm that data flows correctly across agent boundaries and that the orchestration layer passes context without loss or corruption. All tests run in the sandbox environment with injected payloads. IDs continue from the Phase 1 sequence.

Scenario

Input

Expected Output

Pass Criteria

T16

Bank Feed Sync to Matching Agent handoff

Bank Feed Sync Agent completes successfully and passes a 30-record normalised payload to the Matching Agent

Matching Agent receives all 30 records with no field truncation; processing begins within 5 seconds of the sync agent completing

Matching Agent input record count equals 30; no null amounts, dates, or references in the received payload

T17

Matching Agent to Notifications Agent handoff when no exceptions exist

Matching Agent completes with 30 confirmed matches and zero exceptions; triggers Notifications Agent

Notifications Agent receives a context object with matched_count: 30, exception_count: 0, and period label

Slack message and Gmail confirmation reflect the correct counts; Xero close step fires without waiting for an exceptions-resolved flag

T18

Matching Agent to Notifications Agent handoff when exceptions exist

Matching Agent completes with 25 confirmed matches and 5 exceptions; Notifications Agent should wait for human resolution

Notifications Agent fires the Slack alert immediately, then pauses the Xero close step pending the exceptions-resolved signal from Google Sheets

Xero close record is NOT written until the resolved flag is set; Slack message correctly shows exception_count: 5

T19

Full three-agent pipeline in sequence

Scheduled trigger fires; all three agents run in order with a 30-record dataset containing 2 exceptions

Plaid pull, normalisation, matching, exception logging, Slack alert, Gmail email, and Xero close all complete in a single run

Each agent's completion event is logged in sequence; total pipeline duration under 3 minutes; all expected outputs present

T20

Parallel run output matches manual reconciliation

Automation run against the same 3-month historical dataset used for manual reconciliation during the QA parallel run

Matched pairs, exception items, and closing balances produced by the automation match the manually prepared reconciliation outputs

Discrepancy rate between automation and manual output is zero for exact matches; any differences are investigated and documented before Phase 3

T21

Orchestration retry on transient Xero write failure

Xero sandbox returns a 503 on the first reconciliation write attempt mid-pipeline

Orchestration layer retries the write step twice, succeeds on the second retry, and the pipeline continues

Two retry events logged; reconciliation record written on retry; no duplicate records created in Xero

T22

Scheduling: no duplicate run if previous run is still active

Scheduled trigger fires while a previous run is still in progress (simulated by setting a long-running lock on the orchestration context)

New trigger is queued or skipped according to the configured concurrency policy; no duplicate data written

Only one active pipeline instance at any time; queue or skip event logged; no duplicate Xero postings

Edge Case Tests

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate transaction in bank feed

Plaid sandbox returns the same transaction ID twice in a single pull (simulating a known Plaid deduplication edge case)

Bank Feed Sync Agent deduplicates by transaction ID before passing the payload downstream; only one record written

Downstream payload contains one record for the duplicated transaction ID; deduplication event logged

E02

Missing amount field in bank transaction

One bank transaction in the Plaid payload has a null amount field

Agent rejects the malformed record, writes it to a dead-letter log with the transaction ID and error reason, and continues processing the remaining records

All other records processed normally; dead-letter log contains the rejected record with a 'null amount' error tag

E03

Missing reference field in bank transaction

One bank transaction has an empty reference/description string

Matching Agent falls back to amount-plus-date matching only for that record; if a match is found it is tagged as 'probable match' rather than 'confirmed'

Record appears in the exceptions sheet with status 'probable match' and a note indicating no reference was available for matching

E04

Xero API unavailable for the full run

Xero sandbox returns 503 on all calls throughout the test run

Bank Feed Sync Agent logs the failure after three retries, halts the pipeline, and sends an error alert to the FullSpec error channel; no data written anywhere

Error event logged with all retry attempts; Slack error alert sent; Google Sheets and Xero untouched; next scheduled run starts fresh

E05

Bookkeeper does not resolve exceptions within 48 hours

Exceptions sheet has unresolved rows 48 hours after the Slack alert was sent

A reminder Slack message is sent to the finance channel and a follow-up email is sent to the finance lead Gmail address

Reminder Slack message and Gmail email both sent; reminder event logged; Xero close step remains paused

E06

Bank feed covers a different date range to the Xero ledger pull

Plaid sandbox returns transactions for 1 May through 31 May; Xero demo is queried for 1 May through 28 May due to a date parameter bug

Agent detects the date range mismatch, logs a data integrity warning, and halts the matching step until the date parameters are reconciled

No matching attempted on mismatched ranges; data integrity warning logged with both date ranges; pipeline halted with descriptive error message

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test transaction to exercise the full pipeline. Arrange with the finance lead to post a small, clearly labelled internal transfer (for example, a $1.00 inter-account transfer labelled 'QA TEST RUN') so the automation processes at least one known transaction end to end. The Xero period close record created during Phase 3 must be voided or reversed immediately after the test is confirmed as passed. Do not rely on the live Slack channel for the Phase 3 run without first notifying the finance team that a test message will appear. Confirm with the process owner that all production credentials are active and correctly scoped before the run begins.

Scenario

Input

Expected Output

Pass Criteria

T23

Full production pipeline triggered end to end

Scheduled trigger fires against the live Plaid connection; live Xero organisation queried; real transactions for the current period used

Bank feed pull completes, normalised dataset passed to Matching Agent, matched pairs posted to live Xero, exceptions (if any) written to the live Google Sheets workbook, Slack alert sent to live finance channel, Gmail sent to finance lead

All six pipeline steps produce outputs that match the expected schema; each agent completion event is logged with a timestamp; total run time under 5 minutes for up to 400 transactions

T24

Error log review

Retrieve all log events generated during the T23 production run from the orchestration layer's log store

Log entries present for every agent start, agent complete, Xero write, Sheets write, Slack post, and Gmail send event; no unhandled exception events; any expected retries documented

Zero unhandled exceptions in the log; all expected event types present; log is exportable as a structured JSON or CSV file for audit purposes

T25

Exception handling in production: at least one unmatched item

At least one transaction in the live period that cannot be matched (the QA TEST RUN transfer if no natural exception exists)

The unmatched transaction appears in the live exceptions sheet with bank reference, amount, date, and suggested category pre-filled; Slack alert reflects exception_count greater than or equal to 1

Exception row is present and fully populated; Xero close step is paused pending resolution; bookkeeper can resolve the exception by updating the sheet status column

T26

Process owner review and approval

Process owner reviews the T23 pipeline outputs: Xero reconciled entries, Google Sheets exceptions log, Slack summary message, Gmail confirmation, and Xero period close record

Process owner confirms all outputs are correct, the exception handling matches operational expectations, and the automation is approved for ongoing production use; approval is recorded in the FullSpec confirmation system

FullSpec confirmation record shows process owner name, approval timestamp, and the test run ID for T23. No separate signoff block is required. The manual reconciliation process is cleared for retirement once this case passes.

Once T26 is recorded as passed in the FullSpec confirmation system, the Bank Reconciliation automation is considered fully tested and production-ready. The FullSpec team will notify the process owner and provide the handover documentation. For any questions during or after testing, contact support@gofullspec.com.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process