Back to Win/Loss Analysis

Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality
FS-DOC-06Quality

Test and QA Plan

Win/Loss Analysis Automation

[YourCompany.com] · Sales Department · Prepared by FullSpec · [Today's Date]

This document defines every test case the FullSpec team runs to validate the Win/Loss Analysis automation before and after go-live. It is structured in three sequential phases: unit tests for each agent in isolation, integration and edge-case tests covering agent handoffs and failure scenarios, and a final end-to-end production run with process owner sign-off. No phase begins until the previous phase fully passes. The process owner joins the FullSpec team in Phase 3 only.

01Testing approach

Testing runs in three strictly sequential phases. Each phase must achieve a full pass on every test case before the next phase opens. This sequencing ensures that agent-level defects are caught before integration is attempted, and integration defects are caught before any production data is touched. The FullSpec team owns execution of all three phases; the process owner is required to participate in Phase 3 only.

1
Phase 1: Unit Testing
Each agent tested in isolation against sandbox data
Scope
Individual agent logic: Deal Capture Agent, Outcome Classification Agent, and Insight Report Agent each tested independently with no dependency on live downstream systems.
Environment
Sandbox only. HubSpot sandbox pipeline, Typeform preview mode, Slack test workspace, Google Sheets staging tab, and Notion test page.
Credential rule
Sandbox credentials exclusively. No production API keys, OAuth tokens, or live CRM pipeline data may be used at any point during Phase 1.
Pass condition
All cases T01 through T12 return the expected output with no errors. Any failure blocks progression to Phase 2.
Who runs it
FullSpec team only.
2
Phase 2: Integration and Edge Case Testing
Agent handoffs and failure-mode scenarios validated end-to-end in sandbox
Scope
Agent-to-agent handoffs, data passing between HubSpot, Typeform, Slack, Google Sheets, and Notion, plus all defined edge cases including duplicates, missing data, unavailable tools, and human timeout scenarios.
Environment
Sandbox only. Same sandbox environment as Phase 1 with simulated tool-failure conditions introduced via stubbed responses and deliberate payload omissions.
Credential rule
Sandbox credentials only. Production credentials remain locked until Phase 3 is formally opened by the FullSpec team lead.
Pass condition
All cases T13 through T19 pass and all edge cases E01 through E06 are handled within defined tolerances. Any unhandled exception or incorrect fallback blocks Phase 3.
Who runs it
FullSpec team only.
3
Phase 3: End-to-End Production Test
Full workflow run on live systems with process owner review
Scope
One complete production run traced from HubSpot deal close through buyer survey dispatch, rep Slack notification, classification, Google Sheets logging, and Notion report generation. Error log verification included.
Environment
Production systems only. Live HubSpot pipeline (test deal record), live Slack workspace (test channel), live Google Sheets tracker (test row to be voided after test), and live Notion workspace (test page to be archived after test).
Credential rule
Production credentials in use. All test records must be clearly labelled TEST and voided or deleted after the run. Use an internal test email address for any buyer-facing survey sends.
Pass condition
All cases T20 through T23 pass, including final process owner review and approval recorded in the FullSpec confirmation log (T23).
Who runs it
FullSpec team executes T20 through T22. Process owner participates in and approves T23.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only during Phase 1. Do not connect production HubSpot API keys, live Gmail accounts, real Slack workspaces, or any Typeform form receiving real buyer responses. All test payloads must use synthetic deal records with fictional contact data.

Deal Capture Agent — Complexity: Moderate. Estimated build time: 14 hours.

ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: Closed Won deal triggers full capture loop
HubSpot sandbox deal moves to Closed Won; deal value $8,500; contact email present; rep assigned.
Deal record extracted; Typeform survey link dispatched via Gmail to contact; Slack message with structured form sent to assigned rep within 5 minutes.
All three downstream actions confirmed delivered. No errors in automation run log. Timestamp within 5 minutes of trigger.
T02
Happy path: Closed Lost deal triggers full capture loop
HubSpot sandbox deal moves to Closed Lost; deal value $4,200; contact email present; rep assigned.
Deal record extracted; Typeform survey dispatched; Slack rep notification sent with Closed Lost context included in form prompt.
Outcome context in Slack message reflects Closed Lost status, not Closed Won. All three actions confirmed delivered.
T03
Failure: No contact email on deal record
HubSpot sandbox deal closes; contact record has no email address field populated.
Automation logs a missing-email warning; Gmail survey step is skipped; Slack rep notification still fires; error is written to the run log.
No unhandled exception. Gmail step skipped gracefully. Slack notification delivered. Error entry present in run log with deal ID and reason.
T04
Failure: No rep assigned to deal
HubSpot sandbox deal closes; deal owner field is blank.
Automation logs a missing-owner warning; Slack notification cannot be routed; fallback alert sent to the sales manager Slack handle; error written to run log.
Slack fallback to manager confirmed. Error log entry present. No hard crash. Run continues to next available step.
T05
Failure: HubSpot API returns 429 rate limit on deal extract
Sandbox HubSpot stub returns HTTP 429 on the deal record GET request.
Automation applies exponential back-off and retries up to three times over 90 seconds. If all retries fail, error is logged and run halts cleanly.
Retry logic fires. Three attempts confirmed in run log at correct intervals. Clean halt on third failure. No partial data written downstream.

Outcome Classification Agent — Complexity: Complex. Estimated build time: 18 hours.

ID
Scenario
Input
Expected Output
Pass Criteria
T06
Happy path: Rep Slack form submitted with clear single reason
Rep selects 'Price too high' from Slack form; no buyer Typeform response present yet.
Classification agent maps to taxonomy category 'Price'; writes outcome tag and summary to HubSpot deal; appends structured row to Google Sheets tracker.
HubSpot deal record contains correct taxonomy tag. Google Sheets row contains all required columns: deal ID, close date, outcome, reason category, rep name, value. No blank mandatory fields.
T07
Happy path: Both rep form and buyer Typeform response received
Rep cites 'Competitor feature gap'; buyer Typeform response cites 'Chose [Competitor X] for better integrations'.
Classification agent synthesises both inputs; maps to taxonomy category 'Competitive Loss: Feature'; writes merged summary to HubSpot and Google Sheets.
HubSpot summary field contains reference to both rep and buyer inputs. Google Sheets competitor column populated. Taxonomy tag matches expected category from agreed taxonomy.
T08
Classification: Ambiguous multi-reason rep response
Rep free-text: 'Lost it because pricing was a stretch and they liked the competitor UI more'.
Classification agent assigns primary category 'Price' and secondary category 'Competitive Loss: UX'; writes both to HubSpot custom fields; flags as multi-reason in Google Sheets.
Primary and secondary category fields both populated in HubSpot. Multi-reason flag set to TRUE in Google Sheets. No single-category override applied silently.
T09
Failure: Neither rep form nor buyer Typeform received within 24 hours
No Slack form submission and no Typeform webhook received 24 hours after trigger.
Agent logs unanswered survey flag; sets HubSpot deal field 'Survey Status' to 'Awaiting Response'; does not write a taxonomy tag; does not append a Google Sheets row.
HubSpot Survey Status field updated correctly. No premature Google Sheets row. Unanswered flag visible in run log. No classification written.
T10
Failure: Typeform webhook payload malformed or missing required fields
Typeform webhook fires but response body is missing the deal-ID hidden field.
Agent cannot map response to a deal; logs orphaned response warning with raw payload stored; no CRM write attempted; FullSpec alert fired.
Orphaned response record stored (not discarded). No HubSpot write. Alert entry in run log. No silent data loss.

Insight Report Agent — Complexity: Moderate. Estimated build time: 10 hours.

ID
Scenario
Input
Expected Output
Pass Criteria
T11
Happy path: Scheduled trigger fires on first business day of month with full prior-month data
Scheduled trigger fires; Google Sheets tracker contains 35 rows for the prior month, all fully populated.
Agent reads all 35 rows; identifies top three win drivers and top three loss reasons; writes structured Notion page; posts Slack digest to sales channel with manager tagged.
Notion page created in correct workspace section with all required sections present. Slack digest posted with correct channel tag. Findings reference actual top categories from the data, not placeholder text.
T12
Failure: Scheduled trigger fires but prior-month Google Sheets data is empty
Scheduled trigger fires; Google Sheets tracker has zero rows for the prior month.
Agent detects empty dataset; skips Notion page creation; posts a Slack message to sales channel noting no data was captured for the month and no report was generated.
No blank Notion page created. Slack message confirms no-data state. Error log entry present. Manager is notified rather than receiving a misleading empty report.
Test and QA PlanPage 2 of 4
FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Phase 2 validates the handoffs between agents and confirms that data passed across tool boundaries is complete, correctly typed, and written to the right destination. IDs continue from Phase 1. A separate edge-case table covers failure and exception scenarios that span multiple agents or affect the entire pipeline.

ID
Scenario
Input
Expected Output
Pass Criteria
T13
Handoff: Deal Capture Agent to Outcome Classification Agent via Slack form submission
Deal Capture Agent completes successfully (T01 payload); rep submits Slack form within the 24-hour window.
Outcome Classification Agent receives the structured form response payload including deal ID, rep ID, and reason text; classification proceeds without manual re-entry.
No data re-entered manually at the handoff boundary. Deal ID in Classification Agent run log matches Deal Capture Agent run log. Latency from Slack submission to HubSpot write is under 3 minutes.
T14
Handoff: Deal Capture Agent to Outcome Classification Agent via Typeform webhook
Deal Capture Agent completes (T02 payload); buyer submits Typeform within 7 days; webhook fires.
Classification Agent receives Typeform payload with pre-filled deal ID from hidden field; merges with any existing rep form data; writes consolidated outcome to HubSpot and Google Sheets.
Hidden field deal ID correctly routes Typeform response to the correct deal. Merged payload contains both rep and buyer data where both are present. No duplicate Google Sheets row created.
T15
Handoff: Outcome Classification Agent to Insight Report Agent via Google Sheets
Classification Agent writes 10 structured rows to the Google Sheets staging tab over the course of a simulated month.
On the scheduled trigger, Insight Report Agent reads all 10 rows correctly; column mapping matches expected schema; aggregation logic produces accurate top-category counts.
Row count in Notion report matches row count in Google Sheets. Category totals are arithmetically correct. No rows skipped due to schema mismatch.
T16
Handoff: Insight Report Agent to Slack via digest post
Insight Report Agent successfully writes Notion page (T11 scenario).
Slack digest is posted to the correct channel; message contains a direct link to the Notion page; sales manager is tagged by Slack user ID.
Slack message appears in the designated channel within 2 minutes of Notion page creation. Notion link resolves correctly. Manager tag confirmed by Slack user ID, not display name.
T17
Integration: High-value lost deal with no survey response triggers manager exception path
Deal closes as Closed Lost; deal value is $12,000 (above $10,000 threshold); no rep form or buyer Typeform received within 24 hours.
Automation flags deal for manager review; Slack message sent directly to sales manager with deal name, value, and a prompt to initiate a manual follow-up call; deal marked 'Manager Review Required' in HubSpot.
Manager Slack notification confirmed. HubSpot field updated to 'Manager Review Required'. No taxonomy tag applied prematurely. No Google Sheets row created until manager resolves the flag.
T18
Integration: Two deals close within 60 seconds of each other
Two separate HubSpot sandbox deals both move to Closed Lost within 60 seconds; different reps assigned to each.
Both deals processed as independent workflow runs; no data from deal A appears in deal B's payload; both rep notifications sent to the correct rep; both Google Sheets rows contain distinct deal IDs.
Run logs show two separate execution threads. Cross-contamination check: deal ID in each Google Sheets row matches the originating HubSpot deal. Both Slack notifications routed to the correct rep Slack handle.
T19
Integration: Rep re-submits Slack form after initial submission (duplicate submission)
Rep submits Slack form once (T13 complete); rep submits the same form again 10 minutes later with a different reason.
Automation detects duplicate submission for the same deal ID; updates HubSpot and Google Sheets with the latest submission rather than creating a second row; logs an override event.
Google Sheets contains exactly one row per deal ID after two submissions. HubSpot outcome tag reflects the second submission. Override event logged with both submission timestamps and reason values.

Edge case tests are identified separately with the E prefix. These scenarios are designed to break normal flow and confirm the automation handles failure gracefully without data loss or unhandled exceptions.

ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate HubSpot deal stage event fires twice for the same deal
HubSpot sandbox emits two identical deal-stage-changed webhooks for the same deal ID within 5 seconds (webhook duplication scenario).
Idempotency check identifies the duplicate run; second execution is halted before dispatching a second Gmail survey or second Slack notification; only one run proceeds.
Gmail survey sent exactly once. Slack notification sent exactly once. Run log shows second execution flagged as duplicate and halted. No duplicate Google Sheets row.
E02
Slack API unavailable when rep notification is due
Deal Capture Agent reaches the Slack notification step; Slack API returns HTTP 503.
Automation retries Slack call up to three times with 30-second intervals; if all retries fail, fallback email sent to rep's Gmail address; error logged with deal ID.
Retry log shows three attempts. Fallback Gmail sent to rep. Error entry in run log. Gmail survey step not blocked by Slack failure.
E03
Typeform webhook never fires (buyer never responds)
Deal closes; Gmail survey sent; 7 days pass with no Typeform webhook received.
After 7-day timeout, automation marks the Typeform response as 'No Response' in Google Sheets buyer response column; HubSpot Survey Status updated to 'No Buyer Response'; unanswered flag raised for manager review if deal value exceeds $10,000.
Google Sheets buyer response column contains 'No Response' after 7 days. HubSpot Survey Status updated. High-value flag raised correctly. Classification proceeds using rep-only data if rep form was submitted.
E04
Google Sheets write fails due to permissions error
Outcome Classification Agent attempts to append row; Google Sheets API returns HTTP 403 Forbidden.
Automation logs the failed write with full row payload stored in the run log; retries once after 60 seconds; if retry fails, FullSpec alert fired with the payload for manual recovery; HubSpot write is not blocked.
HubSpot outcome tag written regardless of Sheets failure. Payload stored in run log for recovery. FullSpec alert triggered. No silent data loss.
E05
Rep Slack form response submitted outside the 24-hour window
Rep submits Slack form 30 hours after the original notification was sent.
Automation accepts the late submission; classification proceeds; Google Sheets row created with a 'Late Submission' flag in the response timing column; HubSpot outcome tag written normally.
Late submission accepted and processed. 'Late Submission' flag set to TRUE in Google Sheets. No error thrown. Classification output identical to on-time submission.
E06
Notion API rate limit hit during monthly report write
Insight Report Agent attempts to create Notion page; Notion API returns HTTP 429 after partial page creation.
Automation pauses for the Notion-specified retry-after interval; resumes write from the last successful block; does not create a duplicate page; Slack digest is held until Notion page is confirmed complete.
Notion page contains all required sections with no duplicated blocks. Slack digest posted only after Notion page write confirmed. Run log shows retry event and successful completion timestamp.
Test and QA PlanPage 3 of 4
FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test email address (not a real buyer contact) for all Gmail survey sends. Create the test deal record in the live HubSpot pipeline with 'TEST' prefixed to the deal name. After the test run is complete, void and delete the test deal record in HubSpot, remove the test row from the Google Sheets tracker, archive the test Notion page, and confirm the test Slack messages are deleted from the sales channel. Do not proceed with Phase 3 until the FullSpec team lead has confirmed all Phase 2 cases fully pass.
ID
Scenario
Input
Expected Output
Pass Criteria
T20
Full production run: Closed Lost deal traced from HubSpot trigger to Google Sheets log
Test deal record marked Closed Lost in live HubSpot pipeline; internal test email used as contact; FullSpec team member acts as rep and submits Slack form within 15 minutes.
HubSpot trigger fires within 60 seconds of stage change; deal record extracted; Gmail survey sent to internal test address; Slack notification sent to FullSpec team member; rep form submitted; Classification Agent maps reason to taxonomy; HubSpot deal updated with outcome tag; Google Sheets row appended with all required fields.
Each step confirmed in production run log with timestamp. Gmail received at internal test address. Slack notification received. HubSpot outcome tag and survey status fields updated. Google Sheets row contains all 10 required columns with no blanks in mandatory fields. End-to-end latency from trigger to Google Sheets write is under 10 minutes.
T21
Production verification: Notion monthly report generation triggered manually
FullSpec team manually triggers the Insight Report Agent against the live Google Sheets tracker using the current month's data (minimum 5 rows required for a valid test).
Notion page created in the designated workspace section with correct title, date stamp, top win drivers, top loss reasons, and deal count. Slack digest posted to sales channel with correct Notion link and manager tagged.
Notion page structure matches the agreed template sections. Data in the report is traceable to specific rows in Google Sheets. Slack digest posted to the correct channel. Manager tagged by Slack user ID. No placeholder text remaining in the Notion page.
T22
Production verification: Error log and monitoring check
FullSpec team reviews the production run logs from T20 and T21 in the automation platform monitoring view.
All execution steps for both runs are present in the log with timestamps, status codes, and payload summaries. No unlogged exceptions. Retry events (if any) are recorded with outcomes. Alert routing confirmed active.
Run log is complete with no gaps between steps. No silent failures identified. Any retry events include before and after status. Alert destination (FullSpec monitoring) confirmed reachable. Log retention period confirmed at minimum 30 days.
T23
Process owner review and approval
Process owner (Sales Manager) reviews the outputs of T20 and T21 in live systems: HubSpot deal record, Google Sheets row, Notion page, and Slack messages.
Process owner confirms each output matches expected content and format. Process owner records approval in the FullSpec confirmation log by submitting the provided approval form at support@gofullspec.com referencing the project ID. FullSpec records the approval timestamp and confirms go-live clearance.
Process owner approval received and logged in FullSpec confirmation record. Go-live clearance issued by FullSpec team lead. Test deal record, test Sheets row, test Notion page, and test Slack messages all voided and removed post-approval. No separate signoff block required.
Once T23 is approved and logged, the FullSpec team will confirm go-live readiness in writing to the process owner. Any issues raised during the process owner review in T23 are treated as blocking defects and must be resolved before go-live is declared. Contact the FullSpec team at support@gofullspec.com with your project reference for any test phase queries.
Test and QA PlanPage 4 of 4

More documents for this process

Every document generated for Win/Loss Analysis.

Launch Plan
Operations · Owner
View
ROI and Business Case
Finance · Owner
View
Process Runbook / SOP
Operations · Owner
View
Developer Handover Pack
Technical · Developer
View
Integration and API Spec
Technical · Developer
View