FS-DOC-06Quality
Test and QA Plan
Subscription and Recurring Billing
[YourCompany.com] · Finance Department · Prepared by FullSpec · [Today's Date]
This document defines the complete testing programme for the Subscription and Recurring Billing automation, covering all three agents: the Billing Cycle Agent, the Failed Payment Recovery Agent, and the Reconciliation and Notification Agent. Testing is structured in three sequential phases. The FullSpec team executes Phases 1 and 2 entirely in sandbox environments; the process owner joins Phase 3 for a live production run. No phase begins until the previous phase has fully passed. All test case IDs are sequential across phases so every result can be tracked in a single log.
01Testing approach
Testing follows three phases in strict sequence. No phase begins until every test case in the previous phase has achieved a pass status and any defects have been resolved and re-tested. This ensures that integration and end-to-end tests run on a foundation that is already proven at the unit level, and that the production environment is only touched once the full automation is stable.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox credentials
Scope
Individual agent logic: subscriber fetch, invoice creation, Stripe charge, failure detection, retry scheduling, reconciliation matching, email send, and Slack post
Environment
Sandbox only. Stripe test mode, Xero demo company, HubSpot sandbox portal, Gmail test alias, Slack test workspace
Credential rule
No live API keys permitted. All credentials must be sandbox or test-mode equivalents. Live keys must not be present in the environment at this phase
Pass condition
All cases T01 through T15 return the expected output with no unhandled exceptions
Who runs it
FullSpec team only
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure scenarios tested across connected tools
Scope
Cross-agent data handoffs, webhook reliability, duplicate record handling, missing data scenarios, tool unavailability, and human-review timeout behaviour
Environment
Sandbox only. Same sandbox credentials as Phase 1; no production data introduced
Credential rule
Sandbox credentials only. Webhook endpoints must point to the test environment receiver, not the live automation platform listener
Pass condition
All integration cases T16 through T22 and all edge cases E01 through E06 pass without data corruption, missed handoffs, or silent failures
Who runs it
FullSpec team only
3Phase 3: End-to-End Production Test
Full billing cycle run against live credentials with process owner present
Scope
Complete automated billing cycle from trigger through to Slack summary, run against live tools using a controlled internal test subscriber record
Environment
Production. Live Stripe account (test card), live Xero company, live HubSpot portal, live Gmail sender, live Slack finance channel
Credential rule
Live production credentials used. The FullSpec team confirms all sandbox keys have been replaced before Phase 3 begins. No test-mode keys may remain active in the production configuration
Pass condition
All cases T23 through T27 pass, including process owner review and sign-off recorded in the FullSpec confirmation log
Who runs it
FullSpec team executes; process owner reviews outputs and approves at T27
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. Every test in this phase must be run against test-mode Stripe keys, the Xero demo company, the HubSpot sandbox portal, and a Gmail test alias. Do not use live credentials at any point during Phase 1. If a live key is accidentally configured, halt testing, rotate the key, and restart the affected agent's unit tests from the beginning.
Billing Cycle Agent — tests T01 through T06
ID
Scenario
Input
Expected output
Pass criteria
T01
Happy path: billing date reached, single active subscriber
HubSpot sandbox record: 1 active subscriber, monthly plan, valid Stripe customer ID
1 Xero invoice created, 1 Stripe charge attempted, charge status captured
Invoice amount matches plan price; Stripe charge object returned with status 'succeeded'; no errors logged
T02
Happy path: billing date reached, 10 active subscribers
HubSpot sandbox: 10 active subscriber records across two plan tiers
10 Xero invoices created with correct line items per tier; 10 Stripe charges attempted
Each invoice maps to the correct tier price; all 10 charge attempts logged with outcomes; no duplicate invoices
T03
HubSpot returns zero active subscribers
HubSpot sandbox: active subscriber filter returns empty list
Agent exits cleanly with a 'no subscribers found' log entry; no invoices created; no Stripe calls made
Zero Xero invoices; zero Stripe API calls; clean exit log with timestamp
T04
Xero invoice creation fails for one subscriber
HubSpot sandbox: 3 subscribers; Xero sandbox configured to reject invoice creation for subscriber 2
Subscribers 1 and 3 invoiced and charged; subscriber 2 flagged in error log; no Stripe charge attempted for subscriber 2
Error log entry for subscriber 2 includes subscriber ID and Xero error code; billing continues for remaining subscribers
T05
Stripe charge returns 'card_declined' for one subscriber
Xero sandbox: invoice created for 1 subscriber; Stripe test card configured to decline
Charge attempt logged as failed; failed event passed to Failed Payment Recovery Agent trigger
Stripe charge object status is 'failed'; decline code captured; handoff event fired within 30 seconds
T06
HubSpot API unavailable during subscriber fetch
HubSpot sandbox endpoint returns 503 for duration of test
Agent retries fetch up to 3 times with exponential backoff; after third failure, posts alert to error log and halts run
3 retry attempts logged with timestamps; final log entry shows 'HubSpot unavailable, billing run halted'; no Xero or Stripe calls made
Failed Payment Recovery Agent — tests T07 through T11
ID
Scenario
Input
Expected output
Pass criteria
T07
Happy path: failed Stripe charge event received, recovery email sent
Stripe sandbox webhook: 'charge.failed' event for 1 subscriber; Gmail sandbox configured
Recovery email sent to subscriber email address; HubSpot record updated to 'payment_failed'; retry scheduled at T+72 hours
Gmail send confirmed; HubSpot contact property 'billing_status' set to 'payment_failed'; retry job queued with correct timestamp
T08
Retry succeeds after 72-hour interval
Stripe sandbox: retry charge attempt returns 'succeeded'
HubSpot record updated to 'active'; Xero invoice marked paid; no further retry jobs queued
HubSpot 'billing_status' reset to 'active'; Xero invoice status is 'PAID'; retry queue cleared for this subscriber
T09
Retry fails again after 72 hours; second retry scheduled
Stripe sandbox: first retry returns 'failed'
Second recovery email sent; second retry scheduled at T+144 hours from original failure
Second Gmail send confirmed; second retry job timestamp is exactly 144 hours after original failure event
T10
Failed charge event received but subscriber email address missing in HubSpot
Stripe sandbox: 'charge.failed' for subscriber whose HubSpot record has no email field
Agent logs error for missing email; retry still scheduled; error flagged for human review
Error log entry references subscriber ID and 'missing email'; retry job still queued; no unhandled exception thrown
T11
Gmail send fails during recovery email dispatch
Gmail sandbox: SMTP connection refused for one send attempt
Agent logs send failure; retry email queued for re-attempt after 15 minutes; HubSpot status still updated
Send failure logged with timestamp and error code; re-attempt queued; HubSpot 'billing_status' updated regardless of email outcome
Reconciliation and Notification Agent — tests T12 through T15
ID
Scenario
Input
Expected output
Pass criteria
T12
Happy path: Stripe payment confirmed, Xero invoice reconciled
Stripe sandbox: 'charge.succeeded' event; matching open Xero invoice exists
Xero invoice marked as PAID; confirmation email sent via Gmail; renewal reminder queued; Slack summary posted
Xero invoice status is 'PAID'; Gmail send confirmed; Slack message appears in test workspace finance channel with correct totals
T13
Slack billing summary contains correct aggregated figures
Sandbox: 8 invoices raised, 7 paid, 1 failed
Slack message shows: Invoices raised: 8, Collected: 7, Failed: 1, Retrying: 1
Slack message values match sandbox run totals exactly; message posted to correct channel within 60 seconds of last reconciliation
T14
Stripe payment amount does not match Xero invoice amount
Stripe: charge of $99.00 confirmed; Xero invoice total is $109.00
Invoice flagged as unmatched; item added to exception list for human review; invoice not marked as paid automatically
Xero invoice status remains 'AUTHORISED'; exception log entry includes subscriber ID, Stripe amount, and Xero amount; no auto-reconciliation applied
T15
Slack API unavailable when billing summary is due to post
Slack sandbox: API returns 503 at summary post time
Agent retries post up to 3 times; after failure, logs error and stores summary payload for manual review
3 retry attempts logged; final error log includes the full summary payload as a string so it can be posted manually; no unhandled exception
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Integration tests confirm that data passes correctly between agents at each handoff point. These tests run with all three agents active simultaneously in the sandbox environment. IDs continue from Phase 1.
ID
Scenario
Input
Expected output
Pass criteria
T16
Billing Cycle Agent to Failed Payment Recovery Agent handoff on charge failure
Sandbox: Billing Cycle Agent completes; 2 of 5 charges fail
Failed Payment Recovery Agent receives exactly 2 failure events; 2 recovery emails sent; 2 retry jobs scheduled
Failed Payment Recovery Agent trigger log shows 2 received events matching the 2 failed Stripe charge IDs; no events dropped
T17
Failed Payment Recovery Agent to Reconciliation Agent handoff on retry success
Sandbox: retry charge succeeds for 1 subscriber
Reconciliation and Notification Agent receives payment confirmation; Xero invoice reconciled; confirmation email sent
Xero invoice status updated to 'PAID' within 2 minutes of retry success event; confirmation email delivered to test alias
T18
Billing Cycle Agent to Reconciliation Agent handoff on immediate payment success
Sandbox: 5 subscribers all charge successfully on first attempt
Reconciliation and Notification Agent receives 5 success events; 5 Xero invoices reconciled; 5 confirmation emails sent; 1 Slack summary
All 5 invoices marked PAID; all 5 emails confirmed sent; Slack summary totals: Raised 5, Collected 5, Failed 0
T19
Full cycle: invoice raised, charge failed, retry succeeded, reconciled, Slack posted
Sandbox: 1 subscriber; first charge fails; retry at simulated T+72 succeeds
Recovery email sent after failure; retry succeeds; Xero invoice reconciled; Slack summary reflects 1 retry success
All six steps logged in sequence with no gaps; Slack summary shows: Retried and recovered: 1; final Xero status is PAID
T20
HubSpot subscriber status updated correctly after full cycle
Sandbox: subscriber starts as 'active', payment fails, retry succeeds
HubSpot 'billing_status' transitions: active to payment_failed on failure; payment_failed to active on retry success
HubSpot contact property change history shows both transitions with correct timestamps
T21
Webhook delivery delay from Stripe to automation platform
Stripe sandbox: 'charge.failed' webhook delayed by 90 seconds due to simulated network lag
Agent receives event after delay and processes it correctly; retry timing calculated from event timestamp, not receipt timestamp
72-hour retry window calculated from Stripe event 'created' timestamp, not from the moment the automation platform received the webhook
T22
Concurrent billing run triggered twice within the same billing period
Sandbox: billing trigger fires twice for the same period start date within 5 minutes
Second run detected as a duplicate; agent exits cleanly on second trigger without creating duplicate invoices or charges
Idempotency check log shows second run suppressed; Xero invoice count equals subscriber count, not double; no duplicate Stripe charges
Edge case tests cover scenarios outside the normal flow that the automation must handle without data loss or silent failure.
ID
Scenario
Input
Expected output
Pass criteria
E01
Duplicate subscriber record in HubSpot (same email, two contact IDs)
HubSpot sandbox: two active contacts with identical email address and same Stripe customer ID
Agent detects duplicate on email or Stripe ID match; raises one invoice only; flags duplicate contact for human review
Single Xero invoice created; single Stripe charge attempted; duplicate flag logged with both HubSpot contact IDs
E02
Subscriber record missing Stripe customer ID
HubSpot sandbox: active contact with no 'stripe_customer_id' property value
Agent skips charge attempt for this subscriber; logs error with HubSpot contact ID; Xero invoice created but marked as pending manual payment
Xero invoice exists with status 'AUTHORISED' (not charged); error log entry references contact ID and missing field; no Stripe call made
E03
Xero API rate limit hit during batch invoice creation
Xero sandbox: rate limit header returns 429 after 50 requests in 60 seconds
Agent pauses batch, waits for rate limit window to reset (60 seconds), then resumes from the next unprocessed subscriber
No invoices skipped; total invoice count equals subscriber count; rate limit backoff logged with resume index
E04
Stripe webhook signing secret mismatch (tampered payload)
Automation platform receives a webhook with an invalid Stripe signature header
Agent rejects event; logs security warning with timestamp and raw header; does not process the payload
No downstream actions triggered; security log entry present; no Xero, HubSpot, or Gmail calls made from rejected event
E05
Human review of unmatched Xero item times out after 48 hours with no action taken
Reconciliation Agent flags 1 unmatched item; bookkeeper takes no action for simulated 48-hour window
Escalation notification sent to Finance Manager via Slack or email after 48-hour timeout; item remains flagged, not auto-resolved
Escalation message sent at exactly 48 hours after initial flag; item status in exception log updated to 'escalated'; Xero invoice not auto-closed
E06
All three agents complete successfully but Slack finance channel has been archived
Slack sandbox: finance channel archived before billing summary post
Agent catches Slack API 'channel_not_found' error; logs error with channel ID; stores summary payload; does not silently discard
Error log entry includes channel ID and full summary payload as plain text; no unhandled exception; FullSpec team alerted via error monitoring
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: Use a designated internal test subscriber record only. Set the subscriber email address to an internal address you control so no real customer receives a test email. Use a Stripe test card number (e.g. 4242 4242 4242 4242) mapped to the test subscriber's Stripe customer ID in the live account. After Phase 3 is complete, void or credit the test Xero invoice immediately, remove the test subscriber from the active billing list in HubSpot, and confirm with the FullSpec team that no retry jobs remain queued for the test record. Do not proceed with go-live until all Phase 3 test cases pass and the test record has been fully cleaned up.
Phase 3 runs against live production credentials with a single controlled test subscriber. The FullSpec team executes each step and records the output. The process owner observes the run and confirms each outcome before the next case begins. All results are logged in the FullSpec confirmation record for this build.
ID
Scenario
Input
Expected output
Pass criteria
T23
Production billing trigger fires and Billing Cycle Agent processes test subscriber
Live HubSpot: 1 internal test subscriber, active, monthly plan, Stripe test card customer ID
Live Xero invoice created with correct subscriber name, plan description, and amount; Stripe charge attempted against test card
Xero invoice visible in live company with status 'AUTHORISED'; Stripe dashboard shows charge attempt for test customer; amount matches plan price exactly
T24
Failed payment path: test card declined, recovery email received at internal address
Stripe live account: test card configured to decline (card number 4000 0000 0000 0002)
Recovery email delivered to internal test address within 5 minutes; HubSpot contact 'billing_status' set to 'payment_failed'; retry job visible in automation platform queue
Email received at internal address with correct subscriber name and payment update link; HubSpot property confirmed via CRM UI; retry job timestamp is T+72 hours from charge failure
T25
Successful payment path: test card charged, Xero reconciled, confirmation email received
Stripe live account: test card 4242 4242 4242 4242 charged successfully (swap from decline card for this case)
Live Xero invoice marked as PAID; confirmation email with invoice link delivered to internal address; renewal reminder queued in automation platform
Xero invoice status is 'PAID' in live company; email received at internal address; renewal reminder job visible in queue with correct future date
T26
Slack billing summary posted to live finance channel with correct figures
Live Slack workspace: finance channel active; test run totals: 1 invoice raised, 1 collected (or 1 failed depending on card used in T25)
Slack message appears in live finance channel listing correct invoice count, collection count, and any failed or retry counts
Slack message visible in finance channel within 2 minutes of reconciliation completing; all figures match the test run exactly; message formatted consistently with agreed template
T27
Process owner end-to-end review and approval
Process owner reviews: live Xero invoice, Stripe charge record, HubSpot contact update, Gmail confirmation email, Slack summary message, and automation platform run log
Process owner confirms all outputs are correct, complete, and match expected behaviour documented in this plan; approval recorded in FullSpec confirmation log
Process owner confirms approval verbally and via the FullSpec confirmation record. FullSpec logs the approval with timestamp, reviewer name, and a reference to this test case ID. No separate signature block is required. Go-live may proceed once T27 is marked passed.
After T27 passes: the FullSpec team will void the test Xero invoice, archive the test HubSpot record, confirm no retry jobs remain queued, and issue the go-live confirmation to your team. Contact support@gofullspec.com for any questions arising during or after Phase 3 testing.
Test and QA PlanPage 4 of 4