Back to Competitor Intelligence Tracking

Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Competitor Intelligence Tracking

[YourCompany.com] · Sales Department · Prepared by FullSpec · [Today's Date]

This document defines the full quality assurance programme for the Competitor Intelligence Tracking automation. It is written for the FullSpec build team and covers every test case required before signals flow into production. Testing runs in three sequential phases: unit tests against individual agents and tool connections, integration and edge-case tests covering agent handoffs and failure modes, and a final end-to-end production run signed off by the process owner. No phase begins until the previous phase fully passes. The process owner joins the FullSpec team for Phase 3 only.

01Testing approach

All testing follows three phases in strict sequence. No phase begins until every test case in the previous phase carries a confirmed pass status. Phase 1 validates each agent and each tool connection in isolation using sandbox credentials only. Phase 2 validates the handoffs between agents and tools, plus all edge cases that could arise from real-world data quality issues. Phase 3 runs one complete cycle against live production credentials with the process owner present to review outputs and record final approval.

Phase 1: Unit Testing

Individual agents and tool connections tested in isolation

Scope

Intelligence Agent scoring logic; Digest Agent compilation logic; Google Sheets read; Notion write; HubSpot deal note append; Slack post; Gmail send — each tested independently

Environment

Sandbox only — no production credentials, no live Notion database, no live HubSpot deals

Credential rule

All connections must use sandbox or test-tenant credentials. Production API keys must not be present in the environment during Phase 1

Pass condition

All test cases T01 through T14 return the expected output with no unhandled errors

Who runs it

FullSpec team only

Phase 2: Integration and Edge Case Testing

Agent handoffs, tool chain, and failure mode validation

Scope

Full signal pipeline from Google Sheets trigger through Notion log and HubSpot update; Digest Agent reading from Notion and posting to Slack; all edge cases including duplicates, missing fields, unavailable tools, and human timeout scenarios

Environment

Sandbox for integration tests; isolated staging Notion workspace and HubSpot sandbox account

Credential rule

Sandbox credentials only. Staging Notion workspace must be separate from the live intelligence database

Pass condition

Integration test cases T15 through T22 and all edge cases E01 through E06 pass with no data loss, no duplicate writes, and correct error handling logged

Who runs it

FullSpec team only

Phase 3: End-to-End Production Test

Full production run with process owner sign-off

Scope

One complete daily scan cycle and one complete Friday digest cycle run against live production credentials, live Notion database, live HubSpot deals, and the live Slack sales channel

Environment

Production — live credentials, live database, live Slack channel

Credential rule

Production API keys active. All test records written during Phase 3 must be voided or deleted after sign-off. Test Slack messages to be pinned and then removed from the channel

Pass condition

Test cases T23 through T26 all pass; process owner reviews outputs and records approval in the FullSpec confirmation log

Who runs it

FullSpec team executes; process owner joins to review outputs and record approval for T26

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only throughout Phase 1. Do not connect production API keys, the live Notion intelligence database, live HubSpot deal records, or the live Slack sales channel during any test in this phase. Use dedicated test-tenant accounts for every tool.

Intelligence Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: high-relevance signal scored and categorised

Raw signal: competitor pricing page change detected; source URL and raw text supplied

Relevance score 4 or 5; category tag 'Pricing'; structured summary written to Notion sandbox record

Score in range 4-5; category field = 'Pricing'; Notion sandbox write confirmed with all required fields populated

T02

Happy path: product update signal scored and categorised

Raw signal: competitor changelog entry for new feature; source URL and raw text supplied

Relevance score 3 or above; category tag 'Product'; summary written to Notion sandbox record

Score >= 3; category = 'Product'; all Notion fields present: competitor name, category, date, source URL, summary

T03

Happy path: market expansion signal detected

Raw signal: job posting for sales roles in a new region

Relevance score 3 or above; category tag 'Market Expansion'; Notion sandbox write confirmed

Score >= 3; category = 'Market Expansion'; record written without error

T04

Low-relevance signal discarded below threshold

Raw signal: generic blog post with no pricing, product, or market content; relevance expected <= 2

Signal discarded; no Notion record written; discard event logged

No new Notion record created; discard event appears in run log with signal ID and score

T05

Empty raw signal body received

Source URL resolves but body text is empty string

Agent skips signal; logs warning with source URL and timestamp; no crash

No unhandled exception; warning entry present in run log; Notion record not written

T06

Notion write fails mid-run

Valid scored signal; Notion sandbox API returns 503 on write attempt

Agent retries up to 3 times; after final failure logs error with signal ID and moves to next signal; does not halt entire batch

Retry count = 3 in log; error record written to error log table; subsequent signals in batch processed normally

T07

Google Sheets competitor list read — happy path

Google Sheets sandbox tab contains 5 competitor rows with name, URL, and keyword columns

All 5 rows read; list passed correctly to scan loop

Competitor count in run context = 5; no parse errors logged

T08

Google Sheets tab empty

Competitor list tab exists but contains header row only, no data rows

Run halts gracefully; alert sent to error log; no signals fetched

Run status = 'halted_no_competitors'; error log entry present; no downstream steps triggered

Digest Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T09

Happy path: weekly digest compiled from Notion entries

Notion sandbox database contains 12 signal records across 3 competitors added in the past 7 days

Digest groups signals by competitor and category; plain-English summary produced; Slack sandbox message formatted correctly

Digest contains all 3 competitors; grouping correct; Slack message posts to sandbox channel without error; links to Notion entries present

T10

Digest compiled with single competitor, single signal

Notion sandbox contains 1 signal record for 1 competitor

Digest produced with single competitor section; no empty sections rendered; Slack post successful

Digest body non-empty; competitor name present; Slack post confirmed; no formatting errors

T11

No signals logged in the past 7 days

Notion sandbox database contains records but all dated more than 7 days ago

Digest Agent sends a 'no new signals this week' placeholder message to Slack sandbox; does not crash or skip silently

Slack sandbox message received; message body contains 'no new signals' or equivalent; run log status = 'digest_empty_week'

T12

Slack post fails

Valid digest compiled; Slack sandbox webhook returns 500 on post attempt

Agent retries up to 3 times; after failure logs error and triggers Gmail fallback send to configured manager address in sandbox

Retry count = 3 in log; Gmail sandbox send triggered; error log entry present with Slack error code

T13

Gmail digest email send — happy path

Valid digest text; Gmail sandbox credentials configured; recipient address set to test inbox

Email delivered to sandbox test inbox with correct subject line and body

Email received in test inbox; subject contains 'Weekly Competitive Digest'; body matches Slack digest content

T14

HubSpot deal note append — happy path

Scored signal with competitor name matching a sandbox HubSpot deal record; deal ID resolved via competitor field lookup

Timestamped note appended to the matching HubSpot sandbox deal record; note contains competitor name, category, and summary

Note visible on sandbox deal record; timestamp within 60 seconds of run time; no duplicate notes created

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration Tests — Agent Handoffs and Full Pipeline

Scenario

Input

Expected Output

Pass Criteria

T15

Full daily scan pipeline: Sheets to Intelligence Agent to Notion to HubSpot

5-competitor Google Sheets sandbox config; 8 raw signals of mixed relevance injected

Signals above threshold written to Notion sandbox; matching HubSpot sandbox deals updated; low-relevance signals discarded; run log complete

Notion record count = number of signals scoring >= 3; HubSpot notes appended to correct deal IDs; discard count logged correctly

T16

Intelligence Agent to Digest Agent handoff: Notion data flows correctly into digest

10 Notion sandbox records written by T15 run; Digest Agent triggered manually

Digest Agent reads all 10 records; groups by competitor and category; produces formatted digest

Digest competitor count matches Notion record competitors; no records skipped; grouping verified by manual inspection

T17

Digest Agent to Slack to Gmail full delivery chain

Valid digest from T16; Slack sandbox webhook active; Gmail sandbox credentials active

Digest posted to Slack sandbox channel; Gmail digest email delivered to sandbox inbox

Both delivery confirmations in run log; Slack message timestamp and Gmail message ID logged

T18

Competitor name mismatch between Google Sheets and HubSpot deal field

Google Sheets competitor name: 'Acme Corp'; HubSpot sandbox deal competitor field: 'ACME'; signal scored above threshold

Name normalisation step catches mismatch; note appended to correct deal after normalisation; mismatch logged as warning

HubSpot note present on correct deal; warning log entry records both name variants; no duplicate deal note

T19

Scheduled trigger fires at correct time

Daily scan schedule configured for 06:00 in the automation platform sandbox; Friday digest schedule set for 08:00

Daily scan run log shows trigger time within 2 minutes of 06:00; digest run log shows trigger within 2 minutes of 08:00

Both run timestamps confirmed in logs; no manual trigger required

T20

Large batch: 50 signals processed in single daily scan

50 raw signals injected across 5 competitors; mix of high and low relevance

All 50 signals processed; correct subset written to Notion; no timeout or partial-run failure

Run completes within 5 minutes; Notion write count matches expected pass-threshold count; no unhandled errors

T21

Notion API rate limit hit mid-batch

Batch of 30 signals; Notion sandbox API rate-limit header set to trigger at signal 15

Agent pauses and retries after rate-limit window; remaining signals written successfully; no data loss

All above-threshold signals from the batch present in Notion after run completes; rate-limit event logged; run status = 'completed_with_retry'

T22

HubSpot deal with no matching competitor in open deals

Signal scored above threshold for competitor 'BetaCo'; no open HubSpot sandbox deals have 'BetaCo' in competitor field

Notion record written normally; HubSpot step skipped for this signal; skip event logged

Notion record present; no HubSpot note created; skip log entry contains competitor name and signal ID

Edge Case Tests

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate signal received: same URL and body ingested twice in same batch

Two identical raw signals for the same source URL submitted in one daily scan run

Deduplication check identifies duplicate; only one Notion record written; second signal discarded with 'duplicate' flag in log

Notion contains exactly one record for that source URL and date; duplicate log entry present

E02

Missing competitor name field in Google Sheets row

Google Sheets row with URL and keyword populated but competitor name cell empty

Row skipped; warning logged with row number; remaining rows processed normally

No signal fetched for the incomplete row; warning in log references row index; other competitors processed

E03

Notion API completely unavailable for full run duration

Notion sandbox returns 503 for all write attempts during a daily scan run

All scored signals buffered or logged to fallback error queue; alert triggered to error notification channel; run status = 'failed_notion_unavailable'

No signals silently lost; error queue populated with all above-threshold signals; alert message received in error notification channel

E04

Slack webhook unavailable on Friday digest run; Gmail also fails

Slack webhook returns 500; Gmail API returns 401 due to expired token

Both delivery failures logged; run status = 'digest_delivery_failed'; alert sent to error notification channel with digest content attached

Error log contains both failure events with HTTP codes; digest content preserved in log for manual re-send; alert received

E05

Intelligence Agent returns no category tag for a valid high-scoring signal

Raw signal body produces relevance score of 4 but AI model returns empty category field

Signal written to Notion with category = 'Uncategorised'; run log flags missing category; signal not discarded

Notion record present with category field = 'Uncategorised'; flag entry in log; signal not lost

E06

Weekly digest run fires but Notion read returns zero records due to a filter misconfiguration

Digest Agent date filter set to wrong range; Notion database contains records but none match filter

Digest Agent detects zero-record result; sends 'no signals found — check date filter' warning to error log; does not post empty digest to Slack

No Slack post sent; warning log entry present with filter parameters; run status = 'digest_aborted_no_records'

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test competitor entry (a domain you control) added temporarily to the live Google Sheets competitor list. Direct all HubSpot deal note writes during this test to a designated internal test deal record, not to active customer opportunities. Void, delete, or archive all test records in Notion, HubSpot, Slack, and Gmail immediately after the process owner records approval. Confirm the Slack sales channel is aware a test message will appear. Do not run Phase 3 until all Phase 1 and Phase 2 cases carry confirmed pass status.

Scenario

Input

Expected Output

Pass Criteria

T23

Full production daily scan: end-to-end signal capture and logging

Live Google Sheets competitor list with test competitor entry added; daily scan triggered manually at agreed time during Phase 3 session

Test competitor signals fetched, scored by Intelligence Agent, and written to live Notion intelligence database with correct fields; matching test HubSpot deal note appended with timestamp

Notion record visible in live database with all fields: competitor name, category, date, source URL, summary; HubSpot test deal note confirmed by process owner; run log complete with no errors

T24

Production digest compile and Slack delivery: end-to-end Friday run

T23 Notion record present in live database; Digest Agent triggered manually to simulate Friday run

Digest Agent reads T23 record from live Notion database; produces formatted plain-English briefing; posts to live Slack sales channel; sends Gmail digest to sales manager inbox

Slack message visible in sales channel within 2 minutes of trigger; Gmail digest received in sales manager inbox; digest content matches Notion records; Notion entry links functional

T25

Error log and monitoring verification in production

Introduce a controlled bad input: inject one signal with an empty body for the test competitor during T23 run

Empty-body signal logged as warning in production error log; no crash; remainder of run completes; error log accessible to FullSpec team

Warning entry present in production run log with signal ID and source URL; error notification channel receives alert; Notion record not created for the empty signal

T26

Process owner review and final approval

Process owner reviews: live Notion record from T23, HubSpot test deal note from T23, Slack digest message from T24, Gmail digest email from T24, and production run log summary provided by FullSpec

Process owner confirms all outputs are accurate, correctly formatted, and match the agreed specification; approval recorded in the FullSpec confirmation log

Process owner verbally or in writing confirms each output matches expectation; FullSpec team records approval in the FullSpec project confirmation log with date, process owner name, and any noted conditions; no separate sign-off block required

After T26 approval is recorded: remove the test competitor entry from the Google Sheets live list, delete or archive the test Notion record, remove the test deal note from the HubSpot test deal, and delete the test Slack message. The automation is then considered QA-complete and cleared for live operation. Contact the FullSpec team at support@gofullspec.com if any test case requires re-running or if a defect is identified after go-live.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process