Test and QA Plan

How the build is validated before go-live: three test phases, edge cases, pass criteria, and formal sign-off.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Receivables and Invoice Chasing

01Testing approach

Testing is structured across three sequential phases. Phase 1 validates each agent in isolation with mocked inputs. Phase 2 verifies cross-agent integration and edge-case handling. Phase 3 runs a full end-to-end rehearsal against production-equivalent data before go-live sign-off. No phase may begin until every blocking failure from the previous phase is resolved.

Phase 1: Unit Testing

Estimated time: 2 to 3 days

Attribute

Detail

Scope

Individual agent logic: Chase Scheduler Agent, Reminder Sender Agent, Payment Reconciliation Agent tested in isolation with stubbed API responses.

Environment

Staging workspace with Xero sandbox, Stripe test mode, and Gmail test account. No live invoices are touched.

Pass condition

Every test case in Section 02 returns the expected output and no unhandled exceptions are logged. Zero blocking failures permitted before Phase 2 begins.

Phase 2: Integration and Edge Case Testing

Estimated time: 2 to 3 days

Attribute

Detail

Scope

Agent-to-agent hand-offs, Xero webhook delivery, Stripe payment events, Gmail send reliability, partial payment logic, and all documented edge cases.

Environment

Staging environment with realistic synthetic invoice data covering 30 to 40 invoice records. API credentials use sandbox tokens only.

Pass condition

All integration test cases in Section 03 pass. Edge cases produce defined fallback behaviour with no silent data loss. Amber findings are documented and accepted by the process owner before Phase 3.

Phase 3: End-to-End Production Test

Estimated time: 1 to 2 days

Attribute

Detail

Scope

Full workflow from invoice overdue trigger through reminder scheduling, sending, payment matching, reconciliation, and team notification using historical invoice data.

Environment

Production environment with a controlled batch of 10 to 15 real historical invoices that are already settled. Live credentials active. No new reminders sent to real clients during this phase.

Pass condition

All end-to-end cases in Section 04 pass. Payment match rate reaches 96% or above across the test batch. Average days-to-reconcile is within the 31-day target. Process owner and developer both sign off.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 unit test cases

Chase Scheduler Agent: tests cover overdue detection, reminder staging, and prioritisation logic.

Scenario

Input

Expected Output

Pass Criteria

U-CSA-01

Invoice 1 day overdue

Xero invoice: status AUTHORISED, due_date yesterday, amount_due $1,200

Reminder scheduled at tier 1 cadence (Day 1 template), chase list entry created

Scheduled record exists in FullSpec Automation queue within 60 seconds

U-CSA-02

Invoice 7 days overdue

Xero invoice: status AUTHORISED, due_date 7 days ago, amount_due $3,500

Reminder scheduled at tier 2 cadence (Day 7 template), higher priority flag set

Priority flag present; correct template ID assigned

U-CSA-03

Invoice 30+ days overdue

Xero invoice: status AUTHORISED, due_date 31 days ago, amount_due $800

Reminder scheduled at tier 3 cadence, escalation flag set for practice manager review

Escalation flag present; tier 3 template assigned

U-CSA-04

Invoice already paid

Xero invoice: status PAID, amount_due $0

No chase entry created; invoice excluded from queue

No queue record generated; agent exits cleanly

U-CSA-05

Invoice marked voided

Xero invoice: status VOIDED

Invoice skipped; warning log entry written

Warning log present; no reminder scheduled

U-CSA-06

Duplicate trigger for same invoice

Same invoice ID fires overdue trigger twice within 5 minutes

Idempotency check deduplicates; only one queue entry created

Queue contains exactly one record for the invoice ID

Reminder Sender Agent: tests cover email dispatch, pay link attachment, and activity logging.

Scenario

Input

Expected Output

Pass Criteria

U-RSA-01

Scheduled reminder reaches send time

Queue record: invoice_id INV-001, tier 1 template, recipient client@example.com

Gmail sends email with correct body; Stripe pay link embedded; activity log entry written

Email delivered to test inbox; log record timestamp within 30 seconds of send

U-RSA-02

Pay link generation

Stripe test mode: invoice amount $1,200, currency USD

Stripe payment link URL returned and embedded in email body

URL present in sent email; Stripe dashboard shows test link created

U-RSA-03

Gmail send failure (SMTP error)

Simulated 503 from Gmail API on first attempt

Retry after 5 minutes; alert logged after 3 consecutive failures

Retry log entries visible; failure alert raised on third attempt

U-RSA-04

Reminder for disputed account flag

Invoice flagged as disputed in FullSpec Automation

Reminder suppressed; notification sent to bookkeeper for manual handling

No email sent; human notification created in activity log

U-RSA-05

Activity log completeness

Any successful reminder send

Log entry contains: invoice_id, client_email, template_tier, send_timestamp, pay_link_url

All five fields present in log record

Payment Reconciliation Agent: tests cover payment matching, partial payments, and team notification.

Scenario

Input

Expected Output

Pass Criteria

U-PRA-01

Full payment received

Xero payment event: amount $1,200 matched to INV-001 amount_due $1,200

Invoice marked PAID in Xero; team notification sent via FullSpec Automation

Invoice status PAID in Xero sandbox; notification record exists

U-PRA-02

Partial payment received

Xero payment event: amount $600 against INV-001 amount_due $1,200

Invoice remains open; partial payment logged; updated balance $600 visible in chase queue

Invoice status still AUTHORISED; partial amount recorded; chase queue updated

U-PRA-03

Payment with no matching invoice

Xero payment event: reference does not match any open invoice

Unmatched payment flagged; bookkeeper notification sent for manual review

Unmatched alert raised; no invoice updated

U-PRA-04

Overpayment received

Xero payment event: amount $1,500 against INV-001 amount_due $1,200

Invoice marked PAID; overpayment of $300 flagged for bookkeeper review

Invoice PAID; overpayment flag present in log

U-PRA-05

Team notification content

Any successful full reconciliation

Notification contains: invoice_id, client_name, amount_paid, reconciliation_timestamp

All four fields present in notification payload

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 integration and edge case tests

Integration tests verify that agents hand off correctly to one another and that external API events are processed end-to-end without manual intervention.

Scenario

Systems Involved

Expected Behaviour

Pass Criteria

I-01

Chase Scheduler to Reminder Sender hand-off

Xero, FullSpec Automation, Gmail

Overdue invoice detected by Chase Scheduler Agent; queue entry triggers Reminder Sender Agent automatically at scheduled time

Reminder Sender fires without manual trigger; email received in test inbox

I-02

Reminder Sender to Payment Reconciliation hand-off

Gmail, Stripe, Xero, FullSpec Automation

After reminder sent, simulated Stripe payment event triggers Payment Reconciliation Agent; invoice marked reconciled

Payment Reconciliation Agent fires within 2 minutes of Stripe test event; Xero status updated

I-03

Xero webhook delivery reliability

Xero, FullSpec Automation

Xero fires invoice.updated webhook; FullSpec Automation receives and processes within SLA

Webhook received and acknowledged within 10 seconds; no dropped events across 10 consecutive tests

I-04

Stripe pay link click-through to payment

Stripe, Xero

Client clicks pay link in test email; completes Stripe test payment; Xero receives payment event

Stripe test payment appears in Xero sandbox within 3 minutes

I-05

Gmail reply detection routing

Gmail, FullSpec Automation

Test reply sent to reminder email; automation detects inbound reply and routes notification to bookkeeper queue

Reply detection triggers within 5 minutes; bookkeeper notification created; no further automated reminders sent to that invoice

I-06

Multi-invoice batch processing

Xero, FullSpec Automation

15 invoices set overdue simultaneously; all 15 scheduled and reminded without throttling failures

15 queue entries created; 15 emails delivered within 10-minute window; no API rate limit errors

I-07

Activity log end-to-end completeness

FullSpec Automation

Single invoice passes through all three agents; activity log contains trigger, schedule, send, and reconciliation records

Four distinct log entries present for the invoice ID; timestamps are sequential

Edge case tests validate that the system handles abnormal or boundary conditions safely and routes exceptions to a person without silent failure.

Edge Case

Input Condition

Expected Fallback

Pass Criteria

E-01

Xero API timeout

Simulated 504 timeout on aged receivables poll

Retry up to 3 times with exponential back-off; alert raised if all retries fail

Retry log shows 3 attempts; failure alert raised; no data written to queue

E-02

Stripe pay link creation failure

Stripe API returns 400 invalid_request

Reminder sent without pay link; fallback payment instructions included in email body; error logged

Email delivered with fallback text; Stripe error logged; no crash

E-03

Invoice amount_due is zero but status AUTHORISED

Xero invoice with amount_due 0 and status AUTHORISED

Invoice excluded from chase queue; warning log entry written

No reminder scheduled; warning present in log

E-04

Client email address missing from Xero contact

Invoice contact has no email field populated

Reminder suppressed; bookkeeper notification raised with invoice ID and client name

No email attempted; notification contains invoice_id and client_name

E-05

Duplicate payment event from Xero

Same payment_id fires twice within 60 seconds

Idempotency check prevents double reconciliation; second event discarded with log entry

Invoice reconciled exactly once; duplicate discard log entry present

E-06

Reminder cadence exhausted with no payment

Invoice reaches tier 3 and 30+ days overdue with no response

Escalation flag set; practice manager notified; no further automated reminders sent

Escalation notification delivered; chase queue marks invoice as escalated; automated reminders halted

E-07

FullSpec Automation orchestration outage

Simulated FullSpec Automation service unavailability for 15 minutes

Queued events held; processing resumes automatically on recovery; no events lost

All queued events processed after recovery; no duplicate sends; event count matches pre-outage queue depth

All amber edge cases must be reviewed with the process owner before Phase 3 begins. Accepted risks must be documented in the sign-off record.

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 end-to-end production test

Phase 3 runs against a controlled batch of 10 to 15 historical invoices that are already settled. Live credentials are active but no reminders are delivered to real clients during this phase. The batch is selected to include a mix of full payments, partial payments, and previously escalated accounts to exercise all code paths.

Scenario

Test Data

Steps Exercised

Expected Outcome

Pass Criteria

E2E-01

Standard overdue invoice full lifecycle

Historical invoice: $2,400, 5 days overdue at test time, full payment on record

Chase Scheduler Agent, Reminder Sender Agent, Payment Reconciliation Agent

Reminder staged and logged; payment matched; invoice marked reconciled; team notified

All four activity log records present; Xero status PAID; notification delivered

E2E-02

Partial payment lifecycle

Historical invoice: $3,600, partial payment of $1,800 on record

Chase Scheduler Agent, Reminder Sender Agent, Payment Reconciliation Agent

Tier 1 reminder sent; partial payment logged; updated balance visible; follow-up reminder staged

Partial balance correct in queue; second reminder scheduled; no premature reconciliation

E2E-03

Escalated overdue account

Historical invoice: $900, 35 days overdue, no payment received

Chase Scheduler Agent (tier 3), escalation routing

Tier 3 reminder suppressed; escalation flag set; practice manager notification delivered

No email sent to client; escalation notification present; invoice flagged in queue

E2E-04

Client reply detection

Historical invoice with a logged reply on record; reply injected into test Gmail inbox

Reminder Sender Agent reply detection, human routing

Reply detected; bookkeeper notification created; no further automated reminders for that invoice

Notification delivered within 5 minutes; reminder queue entry marked held

E2E-05

Batch of 10 invoices processed simultaneously

10 historical invoices across tiers 1, 2, and 3

All three agents across all 10 invoices

All 10 invoices correctly tiered, reminded or escalated, and reconciled where payment exists

10 of 10 invoices have correct status; no errors in run log; processing completes within 15 minutes

E2E-06

Payment match rate validation

Batch containing 10 invoices with known payment records

Payment Reconciliation Agent across full batch

At least 9 of 10 invoices (96% or above) correctly matched without manual intervention

Match rate 96% or above; any unmatched invoice has a corresponding alert and log entry

E2E-07

Audit trail completeness

Any invoice processed end-to-end in Phase 3

All three agents plus logging

Full audit trail recoverable: trigger timestamp, schedule record, send record, reconciliation record

Four sequential log entries retrievable by invoice_id for every processed invoice

Phase 3 must be completed within the final build week (Week 4). Any blocking failure resets Phase 3. Amber findings from Phases 1 and 2 that were accepted must be re-confirmed as non-blocking before the go-live pass criteria review.

05Go-live pass criteria

All requirements below must be satisfied before the automation is switched to live production traffic. The developer and process owner independently confirm each row. Any requirement marked amber at sign-off must have a documented mitigation and acceptance note.

Requirement

Test IDs

Status

All Phase 1 unit test cases pass with zero blocking failures

U-CSA-01 to U-CSA-06, U-RSA-01 to U-RSA-05, U-PRA-01 to U-PRA-05

To confirm

All Phase 2 integration tests pass with no dropped events across 10 consecutive Xero webhook tests

I-01 to I-07

To confirm

All Phase 2 edge cases return defined fallback behaviour with no silent data loss

E-01 to E-07

To confirm

Phase 3 end-to-end batch of 10 to 15 invoices processes correctly with all expected outcomes met

E2E-01 to E2E-07

To confirm

Payment auto-match rate is 96% or above across the Phase 3 test batch

E2E-06

To confirm

Average days-to-reconcile target of 31 days confirmed achievable based on Phase 3 data

E2E-01, E2E-05

To confirm

Reminders sent on time in 100% of non-disputed, non-escalated test cases

U-RSA-01, I-01, I-06, E2E-01, E2E-05

To confirm

Disputed and sensitive account suppression confirmed: no automated reminder sent to flagged invoices

U-RSA-04, E-06

To confirm

Full audit trail recoverable by invoice_id for every invoice processed in Phase 3

U-RSA-05, U-PRA-05, I-07, E2E-07

To confirm

Idempotency confirmed: no duplicate reminders or reconciliations triggered by repeated events

U-CSA-06, E-05

To confirm

All amber Phase 2 edge cases reviewed with process owner and accepted or mitigated

E-01, E-02, E-04, E-06, E-07

To confirm

Reminder tone, timing, and escalation thresholds confirmed by process owner prior to go-live

U-CSA-01 to U-CSA-03, E-06

To confirm

Go-live is blocked until every requirement above is confirmed. Amber status rows must not remain unresolved at the time of sign-off. Each accepted risk must be countersigned by the process owner.

06Sign-off

Both signatories confirm that all go-live pass criteria in Section 05 have been reviewed, all test phases have been completed, and any accepted risks have been documented. Signing authorises the automation to process live invoice data.

Role

Name

Date

Signature

Developer

Process owner

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 unit test cases

03Phase 2 integration and edge case tests

04Phase 3 end-to-end production test

05Go-live pass criteria

06Sign-off

More documents for this process