FS-DOC-06Quality
Test and QA Plan
Customer Segmentation and List Management
[YourCompany.com] · Marketing Department · Prepared by FullSpec · [Today's Date]
This document defines the full testing programme for the Customer Segmentation and List Management automation. It covers three sequential phases: unit testing of each agent in isolation, integration and edge-case testing across agent handoffs, and end-to-end production verification. The FullSpec team runs Phases 1 and 2 entirely in sandbox environments. Your process owner joins Phase 3 to confirm the live run meets the agreed acceptance criteria before go-live is approved.
01Testing approach
Testing follows three phases in strict sequence. No phase begins until every test case in the previous phase has received a passing result. Any failure blocks progression and triggers a fix-and-retest cycle within the same phase before moving forward.
1Phase 1: Unit Testing
Each agent tested in isolation before any handoff logic is exercised
Scope
Individual agent behaviour: trigger handling, data transformation, deduplication logic, segment rule evaluation, and notification output — each tested independently with mock or sandboxed inputs.
Environment
Sandbox only. No writes to production HubSpot properties, live Mailchimp audiences, or the active Google Sheets log.
Credential rule
Sandbox API keys for HubSpot, Mailchimp, Typeform, and Segment must be used. No production credentials may be loaded during Phase 1.
Pass condition
All cases T01 through T12 return the expected output with no errors. Zero cases may be skipped or marked conditional.
Who runs it
FullSpec team only. Process owner is not required for Phase 1.
2Phase 2: Integration and Edge Case Testing
Agent handoffs and exception scenarios validated end to end in sandbox
Scope
Agent-to-agent handoffs, data continuity across the full merge-segment-notify chain, and structured edge cases including duplicate contacts, missing fields, API unavailability, and human review timeout.
Environment
Sandbox only. Mailchimp test audience, HubSpot sandbox portal, and a dedicated test tab in Google Sheets are used throughout.
Credential rule
Sandbox credentials only. Integration credentials must be stored in the automation platform's secrets vault and not hardcoded in any workflow step.
Pass condition
All cases T13 through T19 pass and all edge cases E01 through E06 resolve to the documented expected outcome. Slack test-channel messages must be inspectable.
Who runs it
FullSpec team only. Process owner is not required for Phase 2.
3Phase 3: End-to-End Production Test
Live environment run with process owner review and formal sign-off
Scope
A single full production run traced from trigger through all three agents to the Slack notification and Google Sheets log. Error-log verification and process owner approval are included.
Environment
Production. Live HubSpot portal, live Mailchimp audiences, live Slack channel, and the master Google Sheets log. A test contact record is used and voided after the run.
Credential rule
Production API credentials are loaded for this phase only, with approval from the process owner. All credentials remain in the secrets vault; no plaintext keys are shared over email or Slack.
Pass condition
All cases T20 through T23 pass, including the process owner review case (T23). Sign-off is recorded in the FullSpec platform confirmation; no separate signature block is required.
Who runs it
FullSpec team runs T20 through T22. Process owner is required for T23 to review outputs and confirm acceptance.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only during Phase 1. Do not load production HubSpot, Mailchimp, Typeform, or Segment API keys into the automation platform until Phase 3 is formally authorised. All writes in this phase must target sandbox or test resources only.
Agent 1: Data Merge and Dedup Agent. Complexity: Moderate. Estimated build time: 14 hours. Trigger: new or updated contact in HubSpot sandbox, or Typeform sandbox webhook.
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: new HubSpot contact triggers merge
Single new contact created in HubSpot sandbox with email, first name, last name, and lifecycle stage populated.
Contact appears in merged dataset with all four fields mapped correctly. No duplicate flag raised.
Merged record visible in Segment sandbox event stream within 60 seconds of trigger. All four fields present and correctly mapped.
T02
Happy path: Typeform webhook delivers new lead
Typeform sandbox submission with email, first name, and form response fields posted via webhook.
Submission ingested and merged into working dataset. Email matched against HubSpot for existing record.
Webhook received and acknowledged with HTTP 200. Merged record present in dataset. No duplicate created in HubSpot sandbox.
T03
Deduplication: exact email match across sources
One HubSpot contact and one Typeform submission share the identical email address.
A single canonical record is produced. The HubSpot record is treated as primary; Typeform fields are merged in where HubSpot fields are blank.
Dataset contains exactly one record for the shared email. Duplicate flag logged. No second record passed to downstream agent.
T04
Deduplication: case-insensitive email match
HubSpot contact with email 'User@Example.com' and Typeform submission with 'user@example.com'.
Records identified as the same contact. Canonical record produced with normalised lowercase email.
Dedup logic treats both strings as identical. One record in output dataset. Email stored in lowercase.
T05
Failure: HubSpot API returns 401 Unauthorized
Sandbox API key revoked mid-run before the HubSpot fetch step executes.
Agent halts with an authentication error. Error event logged. No partial dataset is passed to the next agent.
Workflow execution status shows 'error'. Error detail includes HTTP 401 and the affected step name. Downstream Segmentation Rules Agent does not trigger.
T06
Failure: Typeform webhook payload missing email field
Typeform sandbox submission posted with first name and phone number but no email address.
Record flagged as incomplete. Routed to exception log. Not passed to merge step.
Exception entry created in the test Google Sheets tab with the record identifier, the missing field name ('email'), and a timestamp. No merge attempted.
T07
Failure: Segment API timeout during deduplication
Dedup call to Segment API simulated to exceed 10-second timeout threshold.
Agent retries up to three times with exponential backoff. After three failures, execution halts and an error is logged.
Execution log shows three retry attempts with increasing intervals. Final status is 'error'. No partial output passed to Segmentation Rules Agent.
Agent 2: Segmentation Rules Agent. Complexity: Moderate. Estimated build time: 16 hours. Trigger: successful completion of the Data Merge and Dedup Agent passing a clean dataset.
ID
Scenario
Input
Expected Output
Pass Criteria
T08
Happy path: contact matches a single segment rule
Clean merged record with purchase history indicating high-value buyer (lifetime value above threshold configured in segment rules).
Contact assigned 'High-Value Buyer' segment tag. HubSpot sandbox contact property updated. Mailchimp sandbox audience membership updated.
Segment tag present on HubSpot sandbox contact record. Contact appears in the correct Mailchimp sandbox audience within 90 seconds.
T09
Happy path: suppression list exclusion applied
Contact present in Mailchimp sandbox unsubscribe list. Same contact also present in merged dataset.
Contact excluded from all active segment assignments. Suppression flag written to HubSpot sandbox contact property.
Contact does not appear in any Mailchimp sandbox audience. HubSpot sandbox property 'hs_email_optout' or equivalent suppression field set to true.
T10
Ambiguous segment assignment: contact matches two competing rules
Contact with both a lapsed-customer flag and a recent high-value purchase in the same 30-day window.
Contact flagged as ambiguous. Not assigned to either segment. Routed to human review queue.
Ambiguous flag present in output record. Contact not synced to any Mailchimp sandbox audience. Appears in the human review list passed to the Reporting and Notification Agent.
T11
Failure: Mailchimp API rate limit encountered
Batch of 250 contacts submitted for sync in a single call, exceeding the Mailchimp batch endpoint limit.
Agent detects the 429 response, splits the batch into chunks of 100, and retries each chunk with a 1-second delay between requests.
All 250 contacts synced successfully after batching. No contacts lost. Execution log shows the retry strategy was applied.
T12
Failure: HubSpot property missing on contact record
Contact record in HubSpot sandbox where the 'lifecycle_stage' property is blank, which is required for segment rule evaluation.
Contact flagged with a 'missing required property' error. Routed to exception log. Segment rule not applied.
Exception entry written to the test Google Sheets tab with the contact ID, the missing property name, and a timestamp. Contact not assigned to any segment.
Agent 3: Reporting and Notification Agent. Complexity: Simple. Estimated build time: 8 hours. Trigger: successful completion of the Segmentation Rules Agent sync run.
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Happy path: run summary posted to Slack
Completed sync run with 3 segments updated, 42 records changed, and 2 contacts flagged for human review.
Slack message posted to the designated test channel containing segment counts, records changed, flagged contact count, and a run timestamp.
Slack message appears in the test channel within 30 seconds of agent trigger. All four data points present. Message format matches the agreed template.
T14
Happy path: run log row written to Google Sheets
Same completed sync run as T13.
A new row appended to the test Google Sheets tab with run timestamp, segment counts, records changed, flagged count, and run status ('Success').
Row present in the test sheet within 30 seconds. All six columns populated. Row is appended, not overwriting an existing row.
T15
Failure: Slack API unavailable
Slack API simulated to return a 503 error at the notification step.
Agent logs the Slack failure. Google Sheets log row is still written successfully. Error status recorded in the run log.
Google Sheets row present and marked 'Partial: Slack failed'. Slack channel does not receive a message. Error detail visible in the automation platform execution log.
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Integration tests confirm that data flows correctly between agents and that the outputs of one agent are accepted and processed correctly by the next. All tests in this section continue in the sandbox environment.
ID
Scenario
Input
Expected Output
Pass Criteria
T16
Handoff: Data Merge and Dedup Agent to Segmentation Rules Agent
10 deduplicated contacts passed from the Data Merge and Dedup Agent, including 1 suppressed contact, 1 ambiguous contact, and 8 cleanly matched contacts.
Segmentation Rules Agent receives the full dataset. Suppressed contact is excluded. Ambiguous contact is flagged. 8 contacts are evaluated against segment rules.
Segmentation Rules Agent execution log shows 10 records received, 1 suppressed, 1 flagged, 8 evaluated. No records lost in transit.
T17
Handoff: Segmentation Rules Agent to Reporting and Notification Agent
Segmentation run completing with 8 segments updated, 1 contact flagged for review, and no errors.
Reporting and Notification Agent receives the run summary payload and posts to Slack test channel and writes to Google Sheets test tab.
Slack message and Sheets row both present within 60 seconds. Segment count in Slack matches the 8 updated in the previous step.
T18
Full chain: trigger to notification with no exceptions
A single new HubSpot sandbox contact created with all required fields populated and a clear segment match.
Contact flows through all three agents without manual intervention. Slack notification confirms 1 record changed and 0 flagged.
End-to-end execution completes within 3 minutes. All three agents show 'success' status in execution log. Slack and Sheets both updated.
T19
Handoff failure: Segmentation Rules Agent receives malformed dataset
Data Merge and Dedup Agent intentionally passes a dataset with a missing 'email' key on one record.
Segmentation Rules Agent rejects the malformed record, logs an exception for it, and continues processing the remaining valid records.
Exception log entry present for the malformed record. Remaining records processed normally. Slack notification includes a count of exception records.
Edge case tests cover scenarios that fall outside the normal data path: duplicate contact patterns, missing or null fields, tool unavailability, and human review timeout. These are designed to confirm that the automation degrades gracefully rather than failing silently.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate contact submitted multiple times in rapid succession
The same email address submitted via Typeform webhook three times within a 5-second window (simulating a double-click form submission).
Deduplication logic collapses all three into one record. Only one contact appears in the merged dataset.
Dataset contains exactly one record for the email. Execution log shows 2 duplicate suppression events. No triplicate written to HubSpot sandbox.
E02
Contact record with all optional fields null
HubSpot sandbox contact with only email address populated. All other properties (name, lifecycle stage, purchase history) are null.
Record passes deduplication and is evaluated for segment rules. No segment rule matches due to insufficient data. Record flagged for human review.
Contact appears in the human review list with a note indicating 'insufficient data for segment assignment'. No segment tag applied. Exception logged.
E03
Segment API (Segment.io) unavailable during deduplication
Segment API returns 503 for all calls during a simulated outage window.
Data Merge and Dedup Agent retries up to three times, then halts and logs a critical error. Downstream agents do not trigger. An alert is posted to the Slack error channel.
Execution halted after three retries. No partial data passed downstream. Slack error-channel message present with the outage detail and a timestamp.
E04
Human review contact not actioned within 24 hours
A contact flagged as ambiguous is placed in the human review queue. No action is taken by the marketing manager within the 24-hour review window.
The automation platform sends a reminder Slack message to the designated reviewer at the 24-hour mark. The contact remains unassigned until explicitly actioned.
Reminder Slack message posted at exactly 24 hours after the flag was set. Contact still shows as unassigned in HubSpot sandbox. No automatic segment assignment made.
E05
Mailchimp suppression list returns empty response
Mailchimp sandbox suppression list API returns an empty array (no unsubscribes or bounces on record).
Segmentation Rules Agent treats the empty suppression list as valid. No contacts are suppressed. Processing continues normally.
All contacts proceed to segment evaluation. No false suppression flags applied. Execution log notes that the suppression list returned 0 records.
E06
Very large batch: 500 contact updates in a single trigger cycle
A bulk CRM import into HubSpot sandbox triggers the workflow with 500 contact update events simultaneously.
The automation platform queues updates and processes them in batches of 100. All 500 contacts are processed without data loss. Mailchimp sync uses the batch endpoint with appropriate delays.
All 500 records appear in the merged dataset. Segment assignments correct for all records. No API rate-limit errors in the execution log. Total processing time under 15 minutes.
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use an internal test email address (such as qa-test@[YourCompany.com]) as the contact record for this run. Do not use a real customer's email. After the run is complete, the FullSpec team will void the test contact record from HubSpot, remove it from all Mailchimp audiences, and delete the test row from the Google Sheets log. Confirm with the FullSpec team that these cleanup steps are complete before marking the phase as passed.
Phase 3 is run once, in the live production environment, after all Phase 1 and Phase 2 cases have passed without exception. The FullSpec team coordinates the run timing with your process owner to ensure the Slack channel and Google Sheets log are being monitored in real time. Production API credentials are loaded by the FullSpec team immediately before the run and confirmed active.
ID
Scenario
Input
Expected Output
Pass Criteria
T20
Full production run: trigger to notification traced end to end
Test contact created in live HubSpot portal using qa-test@[YourCompany.com]. Contact has all required properties populated and a clear single-segment match (for example, newsletter-only subscriber criteria).
Contact flows through all three agents in production. Segment tag applied in live HubSpot. Contact appears in the correct live Mailchimp audience. Slack production channel receives a run summary. Google Sheets master log receives a new row.
All three agents show 'success' in the production execution log. HubSpot contact record has the correct segment tag. Mailchimp audience membership updated. Slack message present. Sheets row appended. Total run time under 3 minutes.
T21
Error log verification: confirm structured logging is active in production
Immediately after T20 completes, review the automation platform's execution log and the Google Sheets master log.
Execution log contains a timestamped entry for each of the three agent steps. Google Sheets row includes run timestamp, segment counts, records changed, flagged count, and status ('Success').
All six expected columns present in the Sheets row. Execution log entries cover all three agents with no gaps. Log retention setting confirmed at minimum 90 days.
T22
Cleanup verification: test record voided from all systems
FullSpec team removes the test contact from live HubSpot, live Mailchimp audiences, and the Google Sheets log row.
No trace of qa-test@[YourCompany.com] remains in any live system after cleanup.
HubSpot contact search returns no result for the test email. Mailchimp audience shows no subscriber at that address. Google Sheets log row for the test run is deleted or clearly marked 'TEST VOID'.
T23
Process owner review and acceptance sign-off
Process owner reviews the T20 and T21 outputs: the Slack run summary, the Google Sheets log row, and the HubSpot contact record (before cleanup), with the FullSpec team present to answer questions.
Process owner confirms that the Slack summary format, the segment assignment, the HubSpot property update, and the Google Sheets log row all match the agreed acceptance criteria documented in the Launch Plan.
Process owner explicitly confirms acceptance via the FullSpec platform confirmation step. This confirmation is timestamped and recorded in the FullSpec project record. No separate signature block is required. Phase 3 is marked complete only after this confirmation is received.
Once T23 is confirmed by the process owner, the FullSpec team marks the QA phase as closed, activates the production trigger on the live automation, and schedules the handover walkthrough. The manual segmentation process should not be retired until the first unassisted automated run completes successfully the following day.
Test and QA PlanPage 4 of 4