Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Expense Management Automation

[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]

This document defines the structured test and quality assurance programme for the Expense Management automation, covering all three agents: Receipt Extraction and Policy Agent, GL Coding and Xero Sync Agent, and Approval Routing and Notification Agent. Testing runs in three sequential phases: unit tests in sandbox, integration and edge-case tests in sandbox, and a final end-to-end run in production. The FullSpec team executes Phases 1 and 2 independently; the process owner joins Phase 3 to review outputs and record formal sign-off. No phase begins until every test case in the preceding phase achieves a passing result.

01Testing approach

Testing is structured as three phases in sequence. No phase begins until the previous phase fully passes. Each phase has a defined scope, environment rule, credential requirement, and a named pass condition tied to the test-case ID ranges below. This sequencing ensures that integration and end-to-end tests are only run against logic that has already been individually verified, reducing the risk of compounding failures and ambiguous root causes.

Phase 1: Unit Testing

Individual agent logic verified in isolation

Scope

Each agent tested independently using mocked or static inputs. No live API calls to Xero, Gusto, Slack, or Gmail are made.

Environment

Sandbox only. All tool connections point to sandbox or test instances (Expensify test company, Xero demo organisation, Gusto sandbox, Slack test workspace).

Credential rule

Sandbox credentials only. No production API keys, OAuth tokens, or live account connections are permitted during Phase 1.

Pass condition

All test cases T01 through T14 return the expected output and meet the stated pass criteria. Zero unhandled exceptions permitted.

Who runs it

FullSpec team only.

Phase 2: Integration and Edge-Case Testing

Agent handoffs and boundary conditions verified in sandbox

Scope

Cross-agent data handoffs, tool-to-tool API calls, and edge-case scenarios including duplicates, missing data, unavailable tools, and human timeout paths.

Environment

Sandbox only. Live API connections are permitted to sandbox instances only. Slack test workspace used for approval routing tests.

Credential rule

Sandbox credentials only. Any credential used in Phase 2 must be labelled and stored separately from production secrets in the credential vault.

Pass condition

All integration cases T15 through T22 and all edge-case cases E01 through E08 pass. Every handoff payload is verified against the expected schema. No data is written to production systems.

Who runs it

FullSpec team only.

Phase 3: End-to-End Production Test

Full live run with process owner review

Scope

Complete end-to-end run using real production credentials, a designated internal test submitter, and a real (voided) expense claim. All downstream records in Xero and Gusto are verified then voided after testing.

Environment

Production. All tool connections point to live accounts. Only the designated internal test address is used as the submitting employee.

Credential rule

Production credentials activated for this phase only. Access is confirmed with the process owner before Phase 3 begins. Credentials are rotated or scoped to least-privilege before go-live if not already done.

Pass condition

All production cases T23 through T27 pass, including the process owner review case (T27). FullSpec records the owner's confirmation in the FullSpec platform before the automation is marked live-ready.

Who runs it

FullSpec team executes; process owner reviews outputs and approves at T27.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only for all Phase 1 tests. Do not connect to live Expensify, Xero, Gusto, Slack, or Gmail accounts during this phase. All receipt files used must be synthetic or redacted test documents. No real employee or financial data should appear in any Phase 1 test input.

Agent 1: Receipt Extraction and Policy Agent

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: compliant claim with valid receipt

Expensify submission: JPEG receipt, amount $85.00, date current, merchant 'City Bistro', category Meals

Extracted fields: amount=$85.00, date=valid, merchant='City Bistro', currency=USD. Policy verdict: PASS.

All four fields extracted correctly. Verdict field equals PASS. No flag reason populated.

T02

Missing receipt: no file attached

Expensify submission: no receipt attachment, amount $40.00, merchant 'Uber'

Policy verdict: FAIL. Flag reason: 'Receipt not attached.' Gmail chase email queued.

Verdict equals FAIL. Flag reason string is non-empty. Chase email payload present in output.

T03

Unreadable receipt: blurred image

Expensify submission: low-resolution JPEG (20x20 px), amount $55.00

Policy verdict: FAIL. Flag reason: 'Receipt image unreadable.' Gmail chase email queued.

OCR extraction returns null or empty for amount and merchant. Verdict equals FAIL.

T04

Amount mismatch: receipt differs from claimed amount

Expensify submission: receipt shows $60.00, claimed amount $90.00

Policy verdict: FAIL. Flag reason: 'Claimed amount does not match receipt amount.'

Extracted amount ($60.00) does not equal claimed amount ($90.00). Verdict equals FAIL with correct flag reason.

T05

Over-limit claim: amount exceeds policy cap

Expensify submission: valid receipt, amount $450.00, category Meals (policy cap $75.00 per meal)

Policy verdict: FAIL. Flag reason: 'Meal expense exceeds $75.00 policy limit.'

Amount extracted correctly. Policy rule evaluated. Verdict equals FAIL with limit-breach flag reason.

T06

Disallowed category: personal expense

Expensify submission: valid receipt, merchant 'Netflix', amount $15.99, category Entertainment

Policy verdict: FAIL. Flag reason: 'Expense category not permitted under company policy.'

Merchant and category matched to disallowed list. Verdict equals FAIL with category flag reason.

T07

Foreign currency receipt

Expensify submission: valid receipt in GBP (£70.00), Expensify-converted amount $88.50 USD

Extracted fields include original currency (GBP), original amount (70.00), and converted USD amount (88.50). Verdict: PASS if within limits.

Currency field populated as GBP. Converted USD amount used for policy check. Verdict reflects USD value.

Agent 2: GL Coding and Xero Sync Agent

Scenario

Input

Expected Output

Pass Criteria

T08

Happy path: known merchant mapped to GL code

Passing claim record: merchant='City Bistro', category=Meals, amount=$85.00, receipt URL populated

Draft spend money record created in Xero sandbox: GL code 420 (Staff Entertainment), amount=$85.00, receipt attached.

Xero sandbox returns HTTP 200. Draft record ID present in output. GL code matches mapping table. Receipt attachment confirmed.

T09

Unknown merchant: not in mapping table

Passing claim record: merchant='Acme Supplies Ltd', no mapping table entry

Draft Xero record created with GL code flagged as 'Unclassified (499)'. Flag emitted for finance admin review.

Record created with GL code 499. Flag field populated with 'Merchant not in mapping table.' No blocking error thrown.

T10

Xero sandbox connection failure

Valid passing claim; Xero sandbox API returns HTTP 503

Retry logic executes up to three times at 30-second intervals. After third failure, error event logged and claim held in queue.

Three retry attempts logged. Claim status set to 'Xero sync failed'. No data lost. Error event visible in orchestration layer logs.

T11

Receipt file attachment to Xero record

Passing claim with Google Drive receipt URL (PDF, 2.1 MB)

Receipt file fetched from Google Drive and attached to the Xero draft record as a document.

Xero draft record contains one attachment. Attachment filename matches original receipt filename.

Agent 3: Approval Routing and Notification Agent

Scenario

Input

Expected Output

Pass Criteria

T12

Happy path: Slack approval message sent to correct manager

Xero draft record created; employee department=Marketing; manager Slack ID mapped to @treyes

Slack message sent to @treyes with claim details, receipt preview link, and Approve/Reject action buttons.

Slack API returns HTTP 200. Message appears in correct channel or DM. Action buttons rendered. No other manager notified.

T13

24-hour reminder: manager has not responded

Slack approval message sent; no action taken within 24 hours (simulated via timestamp override in sandbox)

Reminder Slack message sent to same manager. Original message timestamp referenced.

Second Slack message sent after simulated 24-hour window. Reminder text distinct from original. No duplicate reminders after second message within the same window.

T14

Manager rejects claim via Slack

Manager clicks Reject on Slack approval message; rejection reason entered: 'Not a business expense'

Xero draft record remains in draft (not finalised). Finance admin notified via Slack. Employee receives Gmail rejection email with stated reason.

Xero record status unchanged (draft). Finance admin notification sent. Gmail rejection email delivered to test inbox with correct rejection reason in body.

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Phase 2 integration tests verify that data passes correctly between agents and that the orchestration layer handles agent handoffs without data loss or schema mismatch. All tests run in sandbox. IDs continue from the Phase 1 sequence.

Scenario

Input

Expected Output

Pass Criteria

T15

Handoff: Agent 1 passes PASS verdict to Agent 2

Agent 1 completes T01 scenario and emits output payload to the orchestration layer

Agent 2 receives payload with all required fields: claim_id, amount, merchant, currency, receipt_url, policy_verdict=PASS.

Agent 2 input schema validated. All required fields present and correctly typed. No null values in mandatory fields.

T16

Handoff: Agent 1 FAIL verdict does not trigger Agent 2

Agent 1 emits FAIL verdict with flag reason

Orchestration layer routes to Gmail chase flow. Agent 2 is not invoked. Agent 2 logs show no execution event.

Agent 2 execution count remains zero for this claim. Gmail chase payload dispatched. Claim held in 'awaiting receipt' status.

T17

Handoff: Agent 2 passes Xero draft record ID to Agent 3

Agent 2 completes T08 scenario and emits output payload

Agent 3 receives payload with xero_draft_id, employee_id, department, amount, and manager_slack_id populated.

Agent 3 input schema validated. xero_draft_id is a valid non-null string. manager_slack_id resolves to a known Slack user in the test workspace.

T18

Approval: Agent 3 finalises Xero record on manager approval

Manager clicks Approve in Slack (sandbox); Agent 3 receives webhook callback

Xero record status updated from draft to authorised. Gusto sandbox reimbursement line created. Confirmation Gmail sent to test employee address.

Xero sandbox record status equals 'AUTHORISED'. Gusto sandbox shows new reimbursement line with correct amount and employee ID. Gmail delivery confirmed via sandbox SMTP log.

T19

Gusto reimbursement line creation

Approved Xero record with amount=$85.00, employee_id=EMP-009 (sandbox)

Gusto sandbox API call POST /v1/employees/EMP-009/reimbursements returns HTTP 201. Reimbursement amount=$85.00 in next payroll cycle.

HTTP 201 received. Reimbursement record ID returned and logged. Amount matches approved claim amount. Payroll cycle date field populated.

T20

Gmail confirmation email: correct content and recipient

Approved claim; employee email=test-employee@[YourCompany.com]; payroll date=next Friday

Gmail sends confirmation email to test-employee@[YourCompany.com] with subject 'Your expense claim has been approved' and payroll date in body.

Email delivered to sandbox inbox. Subject line matches template. Payroll date string appears in body. No other recipients in To or CC fields.

T21

Full sequential run: compliant claim from submission to Gusto

Employee submits compliant Expensify claim in sandbox; all three agents execute in sequence

Claim passes Agent 1 policy check, GL coded by Agent 2, approved via Slack by Agent 3, Xero record authorised, Gusto line created, confirmation email sent.

All six downstream outputs present and correct. Total elapsed time from trigger to final Gmail under 3 minutes in sandbox. Zero manual interventions required.

T22

Full sequential run: non-compliant claim from submission to finance admin flag

Employee submits over-limit Expensify claim; amount $450.00, category Meals

Agent 1 returns FAIL. Gmail chase or rejection email sent. Agents 2 and 3 not invoked. Finance admin notified with flag reason.

Agents 2 and 3 show zero execution events for this claim. Gmail notification sent. Finance admin Slack alert or email received in sandbox.

Edge-case tests target boundary conditions that are most likely to cause silent failures or data corruption in a production environment. Each scenario is assigned an E-series ID.

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate submission: same claim submitted twice by employee

Employee submits identical Expensify claim (same amount, merchant, date) twice within 10 minutes

Deduplication check triggers on second submission. Second claim flagged as probable duplicate. Finance admin alerted. First claim continues processing normally.

Second claim receives status 'duplicate-flagged'. Only one Xero draft record created. Finance admin receives duplicate alert message.

E02

Missing employee mapping: submitter not in manager assignment table

Expensify claim submitted by employee ID EMP-099 (not in department-manager mapping table)

Agent 3 cannot resolve manager_slack_id. Claim held with status 'manager-unresolved'. Finance admin notified to assign manager manually.

No Slack message sent to an incorrect manager. Finance admin notification sent with employee ID and claim details. Claim status set to 'manager-unresolved'.

E03

Expensify webhook delivery failure: payload not received by orchestration layer

Expensify fires webhook; orchestration layer returns HTTP 500

Expensify retries webhook up to three times per its default retry policy. If all retries fail, claim appears in Expensify with no processing status. FullSpec alerting fires within 15 minutes.

FullSpec monitoring alert triggered within 15 minutes of first failed delivery. Claim not silently dropped. Retry attempts logged.

E04

Xero API rate limit hit during bulk submission

10 expense claims submitted simultaneously in Expensify (simulating month-end batch)

Orchestration layer queues Xero API calls with exponential back-off. All 10 claims eventually processed. No claims dropped. No duplicate Xero records.

All 10 Xero sandbox records created. No HTTP 429 errors propagate as failures. Queue depth and retry counts visible in orchestration logs.

E05

Slack unavailable: API returns HTTP 503 during approval routing

Agent 3 attempts to send Slack approval message; Slack API returns HTTP 503

Retry logic executes up to three times at 60-second intervals. After third failure, fallback Gmail approval email sent to manager. Claim status updated to 'slack-fallback'.

Fallback Gmail sent to manager's email address. Claim status equals 'slack-fallback'. Three Slack retry attempts logged before fallback triggered.

E06

Manager approval timeout: no response after 48 hours

Slack approval message sent; 24-hour reminder sent; still no response after simulated 48-hour window

Claim escalated to finance manager (secondary approver). Escalation Slack message and Gmail sent. Claim status updated to 'escalated'.

Escalation message sent to finance manager contact. Original manager's 24-hour reminder confirmed as already sent. Claim status equals 'escalated'.

E07

GL mapping table returns multiple possible codes: ambiguous merchant

Passing claim; merchant='Amazon', which maps to both Office Supplies (GL 430) and IT Equipment (GL 610)

Agent 2 selects the highest-frequency match from historical mapping. Flags the record with 'Multiple GL codes possible' for finance admin review. Draft Xero record created with selected code.

Xero draft record created. Flag field contains 'Multiple GL codes possible' with both candidate codes listed. Finance admin notified.

E08

Gusto API rejects reimbursement: employee not on active payroll

Approved claim for employee EMP-045; Gusto returns HTTP 422 'Employee not found or inactive'

Reimbursement line not created. Finance admin notified with error detail. Claim status updated to 'reimbursement-failed'. Xero record remains authorised.

No Gusto reimbursement record created. Finance admin notification contains HTTP 422 detail and employee ID. Xero record status unchanged (authorised).

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: use only the designated internal test address (a real employee account agreed with the process owner before this phase begins) as the submitting employee. Submit a low-value claim of $1.00 or less using a real receipt. After the test run is complete, void the Xero spend money record, remove the Gusto reimbursement line from the payroll batch, and delete or archive the Expensify test claim. Do not allow the test transaction to proceed to a live payroll run. Confirm voiding is complete before marking Phase 3 as passed.

Production credentials must be active for Phase 3. Confirm with the process owner that all API connections (Expensify, Xero, Gusto, Slack, Gmail) are pointing to live accounts and that the Gusto sandbox is no longer in use. FullSpec will verify each connection before executing the first test case.

Scenario

Input

Expected Output

Pass Criteria

T23

Full production run: compliant claim submitted and processed end to end

Internal test user submits $1.00 claim in live Expensify with valid JPEG receipt, category Travel, merchant 'Parking Meter'

Agent 1 extracts fields and returns PASS verdict. Agent 2 assigns GL code and creates draft Xero record with receipt attached. Agent 3 sends Slack approval to designated test manager. Manager approves via Slack. Xero record finalised (AUTHORISED). Gusto reimbursement line created. Confirmation Gmail sent to internal test address.

All six downstream outputs present in live systems. Elapsed time from Expensify submission to Gmail confirmation under 5 minutes. No manual intervention required. All outputs verified by FullSpec before voiding.

T24

Production error log verification

Review orchestration layer logs for the T23 run immediately after completion

Log entries present for: trigger receipt, Agent 1 execution and verdict, Agent 2 Xero API call and response, Agent 3 Slack message, approval webhook receipt, Xero finalisation, Gusto API call, Gmail send.

All eight log entry types present. No unhandled exceptions recorded. No WARNING or ERROR level entries that were not anticipated and handled. Log timestamps sequential and consistent with observed elapsed time.

T25

Production run: rejected claim handling

Internal test manager rejects the T23 claim via Slack (if not already approved) OR a separate $1.00 submission is made and rejected by test manager

Xero draft record remains in draft (not authorised). Finance admin receives Slack notification with rejection reason. Employee test address receives Gmail rejection email with reason.

Xero record status equals DRAFT. Finance admin notification confirmed received. Gmail rejection email in test inbox. Rejection reason text matches what test manager entered.

T26

Production void and cleanup verification

FullSpec voids Xero record, removes Gusto reimbursement line, and archives Expensify test claim after T23 and T25

Xero record status equals VOIDED. Gusto payroll batch no longer includes the $1.00 reimbursement line. Expensify claim marked as archived or deleted.

All three cleanup actions confirmed in respective live systems. No $1.00 transaction proceeds to payroll. Process owner notified that cleanup is complete.

T27

Process owner review and approval: final sign-off

Process owner reviews the outputs from T23 through T26, including Xero records, Gusto entries, Slack messages, Gmail confirmations, orchestration logs, and void confirmations

Process owner confirms that all outputs match expectations, that the automation behaves as specified in the SOP and Integration Spec, and that the system is approved for go-live. FullSpec records the approval in the FullSpec platform and marks the automation as live-ready.

Process owner approval recorded in the FullSpec platform. Automation status updated to 'Live-ready'. No separate signoff block required. FullSpec confirmation email sent to [Your name] and [Rep email].

Once T27 is recorded, the FullSpec team will activate monitoring, enable production alerting, and hand over the SOP and Runbook to your finance team. Any issues surfaced after go-live should be reported to support@gofullspec.com with the claim ID and a description of the unexpected behaviour.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process