FS-DOC-06Quality
Test and QA Plan
Insurance Renewal Management
[YourCompany.com] · Legal / Compliance Department · Prepared by FullSpec · [Today's Date]
This document defines the structured test and quality-assurance programme for the Insurance Renewal Management automation. It covers all three test phases, unit through integration through end-to-end production, with sequentially numbered test cases for every agent, handoff point, edge condition, and final production run. The FullSpec team executes Phases 1 and 2 entirely in sandbox environments. The process owner joins Phase 3, which runs against live credentials and real policy data. No phase begins until the previous phase fully passes.
01Testing approach
Testing runs in three sequential phases: Unit, Integration, and End-to-end. Each phase must reach a full pass on every case before the next phase opens. Failures trigger a fix cycle inside the same phase; the phase counter does not advance until the fix is verified. This approach prevents integration assumptions from masking unit-level defects and prevents production runs from surfacing issues that belong in a sandbox.
1Phase 1: Unit Testing
Each agent tested in isolation against sandbox data
Scope
Individual agent logic: Renewal Tracker Agent, Quote Comparison Agent, Approval and Completion Agent. Each tested independently with mocked inputs.
Environment
Sandbox only. No live Notion databases, HubSpot contacts, DocuSign envelopes, Xero records, or Slack channels are touched.
Credential rule
Sandbox or developer-tier credentials for every connected tool. No production API keys loaded at this stage.
Pass condition
All cases T01 through T14 return the expected output with no unhandled errors. All failure-mode cases produce the correct error state or fallback action.
Who runs it
FullSpec team. Process owner is not required for Phase 1.
2Phase 2: Integration and Edge Case Testing
Agent handoffs and exception scenarios tested end-to-end in sandbox
Scope
Cross-agent handoffs, tool-to-tool data integrity, and all defined edge cases including broker non-response, missing quote data, DocuSign timeout, and Xero supplier record absence.
Environment
Sandbox only. Staging instances of HubSpot, DocuSign sandbox envelopes, Xero demo company, and a cloned Notion database are used throughout.
Credential rule
Sandbox credentials only. Real broker email addresses must not appear in any test payload. Use placeholder domains (e.g. testbroker.invalid).
Pass condition
All cases T15 through T22 pass, and all edge cases E01 through E06 either resolve correctly or route to the expected human-review state.
Who runs it
FullSpec team. Process owner may observe but sign-off is not required at this phase.
3Phase 3: End-to-End Production Test
Live credentials, real policy data, full workflow traced from trigger to Xero
Scope
Complete production run from Notion 90-day trigger through broker outreach, quote ingestion, AI comparison, Slack summary, DocuSign approval, broker confirmation, Xero bill creation, and Notion record update.
Environment
Production. Live Notion policy register, live HubSpot, live DocuSign account, live Xero company, and live Slack workspace. Test records are voided after each run.
Credential rule
Production API keys active. Restrict test DocuSign envelope recipients to an internal test address. Void Xero draft bill and remove test Notion record after sign-off is confirmed.
Pass condition
All cases T23 through T27 pass, including error-log verification. Final case T27 is process owner review and approval, recorded in the FullSpec confirmation.
Who runs it
FullSpec team executes T23 through T26. Process owner joins for T27 to review outputs, confirm accuracy, and record approval.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only for all Phase 1 cases. Do not connect production Notion databases, live HubSpot contacts, real DocuSign accounts, active Xero companies, or live Slack workspaces during this phase. All test policy records must use fictional policy numbers and placeholder broker addresses.
Renewal Tracker Agent — estimated build time: 8 hours — Complexity: Moderate
ID
Scenario
Input
Expected output
Pass criteria
T01
Happy path: policy enters 90-day window
Notion record with expiry date exactly 90 days from today; broker email field populated
Gmail broker request sent; HubSpot contact record updated with outreach date; Slack notification posted to compliance channel
All three tool actions fire within 60 seconds of trigger; HubSpot timestamp matches run time; Slack message contains correct policy reference
T02
Policy at 60-day window (second staged outreach)
Notion record with expiry 60 days away; prior 90-day outreach already logged in HubSpot
Second broker email sent with 60-day subject line; HubSpot follow-up task created; Slack notification updated
Email subject line matches 60-day template; HubSpot task due date is today plus 7 days; no duplicate 90-day email fired
T03
Policy at 30-day window
Notion record with expiry 30 days away; 60-day outreach logged
Third broker email sent; HubSpot chase timer reset; Slack urgent-flag message posted
Email body includes urgency language from 30-day template; Slack message tone tag matches 'urgent'; no earlier stage email resent
T04
Policy at 7-day window
Notion record with expiry 7 days away; 30-day outreach logged
Final broker email sent; Slack critical alert posted; HubSpot contact tagged 'critical-renewal'
Email sent to correct broker address; HubSpot tag present; Slack alert appears in compliance channel within 90 seconds
T05
Broker email field missing from Notion record
Notion record with expiry in 90-day window; broker_email field blank
No email sent; Slack error alert posted to compliance channel; HubSpot record flagged 'broker-contact-missing'
No outbound email generated; Slack alert message includes policy name and 'broker contact missing' label; HubSpot flag confirmed
T06
Notion API unavailable (simulated timeout)
Simulated 503 response from Notion on the daily check query
Retry attempted after 5 minutes (up to 3 retries); Slack error notification fired after final retry fails
Three retry attempts logged in run history; Slack notification contains 'Notion unavailable' message; no partial data written
T07
Policy already marked renewed in Notion
Notion record with expiry in 90-day window; status field set to 'Renewed'
No outreach triggered; record skipped silently; run log entry created
No email, HubSpot update, or Slack notification generated for this record; skip entry present in run log
Quote Comparison Agent — estimated build time: 10 hours — Complexity: Complex
ID
Scenario
Input
Expected output
Pass criteria
T08
Happy path: standard PDF broker quote received
Well-formatted PDF quote uploaded to intake folder; prior policy record present in Notion with premium, excess, and coverage limit fields populated
AI extraction produces structured JSON of premium, excess, coverage limits, and exclusions; comparison summary generated; Slack message posted with delta values
All four fields extracted with no null values; premium delta calculated correctly; Slack summary includes 'No material changes detected' or correct flag; comparison document attached
T09
Premium increase above configured threshold (10%)
PDF quote with premium 15% higher than prior year figure in Notion
Comparison summary flagged with 'Premium increase exceeds threshold'; Slack message includes red-flag indicator; approval request held pending review
Flag text present in Slack message; approval DocuSign envelope not sent until human review step is cleared; threshold value matches configuration
T10
Coverage limit reduced in new quote
PDF quote with public liability limit reduced from $5,000,000 to $2,000,000
Comparison summary flags 'Coverage reduction detected'; Slack message includes field-level diff; human review required
Field-level diff visible in Slack message; coverage reduction line item highlighted; agent does not auto-approve
T11
Quote document is image-scanned PDF with no text layer
Scanned image PDF uploaded to intake folder
OCR confidence score below threshold; Slack alert posted to compliance channel requesting manual review; no comparison generated
No partial extraction written; Slack alert contains 'Low OCR confidence, manual review required'; policy record not modified
T12
Quote uploaded but no matching Notion policy record found
PDF quote with policy reference number not matching any Notion record
Slack error alert posted; document held in intake folder unprocessed; run log entry created with unmatched reference
No comparison generated; Slack alert includes unmatched reference string; Notion not modified; document accessible in intake folder for manual resolution
T13
Intake folder email forwarded with no attachment
Email forwarded to intake address with no file attached
Agent detects missing attachment; Slack alert posted; no extraction attempted
No extraction run initiated; Slack message reads 'Quote email received with no attachment'; sender address logged in run history
Approval and Completion Agent — estimated build time: 10 hours — Complexity: Moderate
ID
Scenario
Input
Expected output
Pass criteria
T14
Happy path: DocuSign envelope signed by approver
Comparison summary posted to Slack; DocuSign envelope sent to sandbox approver address; signing simulated
DocuSign signed event received; broker acceptance email sent via Gmail; Xero draft bill created; Notion policy record updated with new premium, coverage dates, and next renewal trigger
All four completion actions fire in sequence; Xero bill status is 'Draft'; Notion record shows updated expiry date and policy number; Gmail sent-items log entry confirmed
T15
DocuSign envelope expires unsigned after 48 hours
Sandbox envelope created with 48-hour expiry; no signing action taken
Envelope expiry event received; Slack alert posted to compliance channel; HubSpot task created for manual follow-up; no broker email sent
Slack alert text includes 'Approval envelope expired'; HubSpot task due date is same day; broker email absent from sent log
T16
Xero supplier record missing for insurer
Signed DocuSign envelope; insurer name in Notion record does not match any Xero supplier
Xero bill creation fails gracefully; Slack error alert posted; Notion record updated with all other fields except bill reference; run log records failure reason
Xero API returns 400 or 404; error caught and logged; Slack alert reads 'Xero supplier not found: [insurer name]'; Notion updated for all non-Xero fields
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Integration tests verify that data passes correctly between agents and tools across the full handoff chain. All cases run in sandbox with mocked external responses where indicated. IDs continue from the Phase 1 sequence.
ID
Scenario
Handoff tested
Expected output
Pass criteria
T17
Renewal Tracker to Quote Comparison: HubSpot outreach log triggers intake watch
Renewal Tracker Agent logs outreach in HubSpot; Quote Comparison Agent intake watch activates for that policy
HubSpot record update propagates to intake watch filter; Quote Comparison Agent begins monitoring intake folder for matching policy reference
Policy reference in HubSpot matches filter condition in Quote Comparison Agent; no duplicate monitoring instances created
T18
Quote Comparison to Approval: comparison summary triggers DocuSign envelope
Quote Comparison Agent posts Slack summary with no material flags; Approval and Completion Agent fires
DocuSign envelope created and sent to configured approver email within 2 minutes of Slack post
Envelope created with correct signer email; comparison document attached as PDF; envelope subject line includes policy name; Slack summary link included in envelope body
T19
Approval to broker confirmation: DocuSign sign-off triggers Gmail broker email
Sandbox DocuSign envelope signed; Approval and Completion Agent receives webhook event
Broker acceptance Gmail sent within 60 seconds of signing event; email addressed to broker contact stored in Notion
Gmail sent-items log confirms delivery to broker address; email body includes policy reference and acceptance confirmation; no second email sent on webhook retry
T20
Broker confirmation to Xero and Notion: sequential completion actions
Broker Gmail sent; Approval and Completion Agent proceeds to Xero bill creation and Notion update
Xero draft bill created with correct amount, supplier, and account code; Notion record updated with new policy number, premium, coverage start and end dates, and next renewal date
Xero bill amount matches premium field from comparison summary; Notion next renewal date is exactly 12 months from coverage start date; all Notion fields populated with no nulls
T21
Slack notification delivery to correct channel across all agents
Each agent fires a Slack notification at its respective step
All Slack messages appear in the designated compliance channel; no messages posted to unintended channels; each message includes the correct policy name and action label
Three distinct Slack messages confirmed in compliance channel (outreach, summary, completion); no messages in general or other channels; policy name string consistent across all three
T22
HubSpot chase timer fires second email when no quote received within 7 days
HubSpot chase timer set to 7-day follow-up; no quote uploaded to intake folder within that window
Second broker chase email sent automatically via Gmail; HubSpot task updated to show chase sent; Slack reminder posted
Chase email sent on day 7 (not day 6 or 8); email body matches chase template; HubSpot task status updated; Slack reminder references original outreach date
The following edge case tests cover scenarios outside the standard happy path. Each is designed to confirm that the automation fails gracefully and routes the exception to a human without data loss or silent errors.
ID
Edge scenario
Simulated condition
Expected behaviour
Pass criteria
E01
Duplicate policy record in Notion
Two Notion records with identical policy number and overlapping expiry dates both enter the 90-day window on the same daily check
Only the most recently modified record triggers outreach; Slack alert posted flagging the duplicate; no double-email to broker
Exactly one broker email generated; Slack alert lists both record IDs and prompts manual deduplication; run log confirms which record was actioned
E02
Broker quote received with no policy reference number
PDF quote uploaded to intake folder; document contains no readable policy or reference number
Quote Comparison Agent posts Slack alert requesting manual matching; document held unprocessed; no Notion record modified
No extraction or comparison run; Slack alert includes intake timestamp and filename; document remains in intake folder accessible for manual action
E03
Slack API unavailable when comparison summary is due to post
Simulated Slack API 503 response at the point the Quote Comparison Agent attempts to post
Retry logic fires up to 3 times at 2-minute intervals; if all retries fail, comparison summary is emailed directly to the approver Gmail address as a fallback
Three retry attempts logged; fallback Gmail sent to approver; subject line includes 'Slack unavailable, summary via email'; DocuSign envelope creation proceeds after fallback confirmation
E04
DocuSign approver email address returns bounce
DocuSign envelope sent to approver address that returns a delivery failure
Envelope delivery failure event captured; Slack alert posted to compliance channel; no broker acceptance email sent; HubSpot task created for manual approver contact
No broker email in sent log; Slack alert includes approver address and bounce reason; HubSpot task due date is same day; envelope voided in DocuSign sandbox
E05
Xero API rate limit hit during bill creation
Simulated Xero 429 response during the bill creation step
Exponential back-off applied (30s, 60s, 120s); bill creation retried up to 3 times; if all retries fail, Slack alert posted and Notion record updated with 'Xero bill pending manual creation' note
Back-off timing confirmed in run log; on final failure, Notion field 'xero_bill_status' set to 'manual-required'; Slack alert posted with policy name and insurer
E06
Human review timeout: material flag raised but no human action taken within 24 hours
Quote Comparison Agent flags material coverage change; Slack message posted; no human response within 24 hours
Escalation Slack message sent to compliance channel mentioning the process owner by name; HubSpot task due date reset to today; DocuSign envelope not created until flag is cleared
Escalation message appears at 24-hour mark; original Slack message remains unmodified; DocuSign envelope absent from sandbox account; HubSpot task timestamp updated
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use an internal test email address as the DocuSign approver recipient for all production test envelopes. Do not send test envelopes to the actual CFO or business owner inbox. After each production test run, void the DocuSign envelope, delete or archive the test Notion policy record, and delete the Xero draft bill. Confirm with the FullSpec team that these cleanup steps are complete before signing off on any case in this phase.
Phase 3 uses live production credentials across Notion, Google Workspace, HubSpot, DocuSign, Xero, and Slack. Confirm with FullSpec that all API keys are scoped correctly and that the Xero supplier record for the test insurer exists before starting T23.
ID
Scenario
Production action
Expected output
Pass criteria
T23
Full production run: policy enters 90-day window
A test policy record is added to the live Notion policy register with an expiry date exactly 90 days from today and a test broker email address (internal domain). The daily check runs.
Real Gmail broker request sent to test address; live HubSpot contact record created and tagged; live Slack compliance channel receives outreach notification with correct policy name
All three tool actions confirmed in live environments; HubSpot record visible and timestamped; Slack message appears within 90 seconds; no production broker address contacted
T24
Production quote intake and AI comparison
A sample broker quote PDF (using realistic but fictional figures) is uploaded to the live intake folder. Prior policy figures are pre-populated in the test Notion record.
AI extraction produces structured output matching the uploaded figures; comparison summary generated with correct premium delta; Slack summary posted to live compliance channel with comparison document attached
Extracted premium, excess, and coverage limit values match the sample PDF to within rounding; delta calculation correct; Slack message posted to correct channel; comparison PDF accessible via link
T25
Production DocuSign approval and broker confirmation
No material flags in comparison summary. Approval and Completion Agent sends DocuSign envelope to internal test address. FullSpec team signs the envelope.
DocuSign signed event received by automation; broker confirmation Gmail sent to test broker address; no production broker contacted
Signed event webhook received within 30 seconds of signing; Gmail sent to test address only; email body matches broker acceptance template; DocuSign envelope status shows 'Completed'
T26
Production Xero bill creation and Notion record update
Following DocuSign sign-off, Approval and Completion Agent proceeds to create Xero bill and update Notion record.
Xero draft bill created with correct supplier, amount (matching test PDF premium), and account code; Notion test record updated with new policy number, premium, coverage start, coverage end, and next renewal trigger date
Xero draft bill visible in live Xero company under the correct supplier; all five Notion fields populated; next renewal date is exactly 12 months from coverage start; FullSpec runs cleanup immediately after confirmation
T27
Process owner review and approval sign-off
The process owner reviews all Phase 3 outputs: Slack messages, HubSpot record, DocuSign envelope, Xero draft bill, and updated Notion record. FullSpec presents a summary of all T23 through T26 results and the error log.
Process owner confirms that all outputs are accurate, complete, and match business expectations. Any required adjustments are noted. Process owner records approval in the FullSpec confirmation, which closes the QA cycle.
Process owner approval recorded in the FullSpec confirmation document. No unresolved issues outstanding. Error log reviewed and confirmed clean. Go-live authorised.
Once T27 is recorded as passed and the FullSpec confirmation is signed, the QA cycle is complete. The FullSpec team will proceed to go-live configuration, activate production monitoring, and hand over the runbook. Contact support@gofullspec.com for any questions arising after the QA cycle closes.
Test and QA PlanPage 4 of 4