Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Review and Reputation Management

[YourCompany.com] · Marketing Department · Prepared by FullSpec · [Today's Date]

This document defines the complete quality assurance programme for the Review and Reputation Management automation. It covers every test case the FullSpec team runs before a single live review is touched by the system, organised into three sequential phases: unit testing of individual agents in sandbox, integration and edge-case testing across agent handoffs, and a final end-to-end production run that your team witnesses and approves. No phase begins until the previous phase has fully passed. All test IDs are sequential and traceable; edge-case IDs are prefixed E to distinguish them from functional cases.

01Testing approach

Testing runs in three phases in strict sequence. Phase 1 (Unit) isolates each agent and confirms it behaves correctly in isolation using synthetic sandbox data. Phase 2 (Integration) verifies that agents hand off data correctly to one another and that the system handles edge cases gracefully. Phase 3 (End-to-end) runs a controlled production trial with real credentials on a live but monitored environment. No phase begins until every test case in the preceding phase carries a passing status.

Phase 1: Unit Testing

Individual agent behaviour in isolation

Scope

Review Classifier Agent, Response Drafting Agent, and CRM and Reporting Agent tested independently with synthetic inputs

Environment

Sandbox only. No live platform credentials are used in Phase 1

Credential rule

All API calls use sandbox or mock credentials. Production tokens must not be loaded into the environment at this stage

Pass condition

All cases T01 through T12 return expected outputs with zero deviation from defined pass criteria

Who runs it

FullSpec team only. No process owner involvement required in Phase 1

Phase 2: Integration and Edge Case Testing

Agent handoffs and failure mode coverage

Scope

Full agent chain from polling trigger through to Notion log and Slack alert; edge cases covering duplicates, missing data, tool unavailability, and human timeout

Environment

Staging environment with sandbox credentials for all tools. Notion and HubSpot use dedicated test workspaces

Credential rule

Sandbox credentials only. HubSpot, Notion, and Slack staging tokens in use; no production workspace is targeted

Pass condition

All integration cases T13 through T20 pass, and all edge cases E01 through E06 produce the correct fallback or error-handling behaviour

Who runs it

FullSpec team only. Results are documented and shared with the process owner ahead of Phase 3

Phase 3: End-to-End Production Test

Live environment, process owner sign-off

Scope

Full workflow run on live credentials: Google Business Profile, Trustpilot, Slack, HubSpot, Notion, and Gmail. One positive and one negative test review per platform

Environment

Production. All live API tokens active. Test records voided after confirmation

Credential rule

Production credentials in use. Only internal test reviewer accounts or designated test listings must be used. No real customer-facing review pages are posted to unless the record can be deleted immediately

Pass condition

All cases T21 through T27 pass, including process owner review and approval recorded in the FullSpec confirmation log

Who runs it

FullSpec team runs the automation; process owner and Marketing Manager join to observe and approve the final case

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only during Phase 1. No production API tokens, live HubSpot contacts, live Notion databases, or real review platform accounts may be used in any test case below. If a live token is accidentally used, invalidate it immediately and rotate before continuing.

Review Classifier Agent — classifies incoming review text and writes a structured record to the Notion tracker.

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: 5-star positive review

Review text: 'Absolutely brilliant service, will be back.' Star rating: 5. Platform: Google Business Profile.

Sentiment: positive. Confidence score: >=0.90. Topic summary: 'Service quality praise.' Notion row created.

Notion row present with correct sentiment, rating, platform, and topic fields populated within 60 seconds of trigger

T02

Happy path: 3-star neutral review

Review text: 'Fine experience, nothing special.' Star rating: 3. Platform: Trustpilot.

Sentiment: neutral. Confidence score: >=0.75. Topic summary: 'Average experience.' Notion row created.

Notion row created with sentiment=neutral, confidence field >=0.75, and source platform=Trustpilot

T03

Happy path: 1-star negative review

Review text: 'Waited two hours and no one helped us.' Star rating: 1. Platform: Google Business Profile.

Sentiment: negative. Confidence score: >=0.90. Topic summary: 'Wait time and staff availability.' Notion row created. Downstream flag set to escalate.

Notion row has sentiment=negative, escalation flag=true, and confidence >=0.90

T04

Failure mode: empty review body

Review text: '' (empty string). Star rating: 4. Platform: Trustpilot.

Agent skips classification. Error event logged. Review held in a pending queue. No Notion row written.

No Notion row created. Error log entry present with review ID and reason: 'empty_body'. Alert raised to FullSpec monitoring.

T05

Failure mode: confidence score below threshold

Review text: 'It was fine I guess maybe.' Star rating: 3. Ambiguous content.

Sentiment: neutral (best guess). Confidence score: <0.70. Review flagged for manual review rather than proceeding automatically.

Confidence field <0.70 causes workflow to pause and route to human review queue. No downstream agent triggered.

T06

Failure mode: Notion API unavailable

Valid review input. Notion endpoint returns 503 during write attempt.

Agent retries up to three times with exponential backoff. On third failure, error event logged and review held in retry queue.

Three retry attempts visible in execution log. Review not lost. Error event written to fallback log with timestamp and Notion error code.

Response Drafting Agent — generates a brand-voice reply and routes it correctly based on star rating.

Scenario

Input

Expected Output

Pass Criteria

T07

Happy path: positive review draft and auto-approve

Classified record: sentiment=positive, star rating=5, topic='Service quality praise.'

Draft response generated using brand voice guidelines. Response does not contain placeholder text. Auto-approve flag set. Response queued for posting.

Draft present in execution payload. No placeholder brackets remain. Auto-approve=true. Character count between 80 and 300.

T08

Happy path: negative review draft routed to Slack

Classified record: sentiment=negative, star rating=1, escalation flag=true.

Draft response generated. Slack message sent to manager channel containing review text, draft, star rating, and approval link.

Slack message received in staging channel within 90 seconds. Message contains all four required fields. Approval link is valid and unique to this review ID.

T09

Happy path: neutral review draft auto-approved

Classified record: sentiment=neutral, star rating=3.

Draft response generated. Auto-approve flag set (neutral treated same as positive for routing). Response queued for posting.

Draft present. Auto-approve=true. Tone of draft is warm but not effusive. No escalation to Slack.

T10

Failure mode: brand voice prompt missing

Classified record: sentiment=positive, star rating=4. Brand voice config variable empty.

Agent raises a configuration error. Draft not generated. Error logged. Review held in pending queue.

No draft produced. Error log entry references 'brand_voice_missing'. No response queued for posting. Alert raised.

T11

Failure mode: Slack API unavailable for negative review escalation

Classified record: sentiment=negative, star rating=2. Slack endpoint returns 503.

Slack send retried up to three times. On failure, fallback email sent to manager via Gmail with the same content.

Gmail fallback email delivered within 120 seconds of Slack failure. Subject line references review ID. Draft response included.

CRM and Reporting Agent — matches reviewer to HubSpot contact and compiles the weekly Slack report.

Scenario

Input

Expected Output

Pass Criteria

T12

Happy path: reviewer matched to HubSpot contact

Reviewer name and email present. Matching HubSpot contact exists with same email.

Timeline activity logged on the HubSpot contact: review platform, star rating, sentiment, and response status.

HubSpot contact timeline shows new activity within 60 seconds. Activity type='review_logged'. All four fields populated.

T13

Happy path: weekly Slack report generated

Notion tracker contains 14 review rows from the past 7 days with mixed sentiment and platforms.

Slack message posted to marketing channel with review volume (14), average star rating, response rate percentage, and top complaint theme.

Slack message received in staging channel. All four summary fields present. Volume figure matches Notion row count exactly.

T14

No HubSpot match found (anonymous reviewer)

Reviewer name only, no email. No matching HubSpot contact found.

Partial record created in HubSpot with reviewer name and review data. Record flagged as unverified. No existing contact modified.

New HubSpot contact created with source='review_unmatched'. Existing contacts unchanged. Flag field set.

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests verify that data flows correctly between agents across the full chain, from polling trigger through to Notion log and Slack report. IDs continue the T sequence from Phase 1.

Scenario

Input

Expected Output

Pass Criteria

T15

Full chain: positive review, end-to-end sandbox pass

New review event injected at polling trigger: 5-star, Google Business Profile, full review text.

Classifier creates Notion row. Drafting agent generates response and sets auto-approve. Response payload passed to posting action. CRM agent logs activity in HubSpot. All steps complete without error.

Execution log shows all five nodes completing with status=success. Notion row present. HubSpot activity present. Total chain completes within 3 minutes.

T16

Full chain: negative review triggers Slack escalation and approval

New review event: 1-star, Trustpilot, negative text. Manager approves via simulated approval link.

Classifier flags negative. Drafting agent generates draft and sends Slack alert. Simulated approval received. Response queued for posting. Notion row updated with status=approved. HubSpot logged.

Slack alert delivered. Approval event received and parsed correctly. Notion row status field updated to 'approved'. HubSpot timeline activity created.

T17

Handoff: Classifier to Drafting agent data integrity

Classifier output payload with all fields: sentiment, confidence, star rating, platform, topic summary, review ID.

Drafting agent receives all six fields intact. Prompt populated with correct values. No fields null or truncated.

Drafting agent input log shows all six fields present and matching Classifier output exactly. No data loss between agents.

T18

Handoff: Drafting agent approval state to CRM agent

Approval confirmed (positive auto-approve path). Response status=posted. Review ID passed to CRM agent.

CRM agent receives review ID and status=posted. HubSpot timeline activity created. Notion row status updated to 'complete'.

CRM agent execution log shows review_id and status=posted as inputs. Both HubSpot and Notion write actions confirm success.

T19

Polling trigger: no new reviews detected

Polling trigger fires on schedule. Zero new reviews present on connected platforms.

Workflow exits cleanly with status=no_new_reviews. No downstream agents triggered. No Notion rows created.

Execution log shows trigger completed with exit code=no_new_reviews. Zero agent nodes executed. No errors raised.

T20

Weekly report: correct aggregation across mixed sentiment rows

Notion tracker contains 10 positive, 3 neutral, and 4 negative review rows from the past 7 days.

Slack report shows total=17, sentiment breakdown matching source data, response rate calculated from status fields, top theme extracted from topic summaries.

All four report fields match expected values calculated from source Notion data. No rounding errors greater than 1%.

Edge case tests cover scenarios outside the happy path that the system must handle without data loss, silent failure, or uncontrolled behaviour.

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate review detection

Same review ID submitted twice to the polling trigger within a single polling cycle (platform API sends duplicate event).

Second instance detected as duplicate. Workflow deduplication check fires. Second run aborted. Only one Notion row created.

Execution log shows deduplication check triggered on second event. Single Notion row exists for the review ID. No duplicate HubSpot activity.

E02

Missing star rating in review payload

Review event received with review_text present but star_rating field null or absent.

Classifier applies sentiment analysis from text alone. Star rating field in Notion left blank. Escalation logic defaults to neutral path. Operator alert raised.

Notion row created with star_rating=null. Sentiment populated from text. Escalation path=neutral (not negative). Alert log entry present.

E03

HubSpot API rate limit hit during CRM update

CRM agent attempts to write 12 contact timeline activities in rapid succession. HubSpot returns 429 Too Many Requests.

Agent pauses and queues remaining writes. Retries with exponential backoff. All 12 activities eventually written.

All 12 HubSpot timeline activities present after retry cycle. No activities lost. Rate limit retry logic visible in execution log.

E04

Manager does not respond to Slack approval within 24 hours

Negative review escalated to Slack. No approval or rejection received within 24-hour window.

Timeout event fires. Second Slack reminder sent to manager. Review flagged as 'awaiting approval' in Notion. Response not posted.

Second Slack reminder sent at 24-hour mark. Notion row status=awaiting_approval. No response posted to review platform.

E05

Google Business Profile API returns 401 during polling

Polling trigger fires. Google Business Profile API returns 401 Unauthorized (token expired or revoked).

Workflow halts. Error event logged with error code 401 and platform reference. Alert sent to FullSpec monitoring. No reviews processed until credential is refreshed.

Execution log shows halt at polling step with error_code=401. Monitoring alert fired. No downstream agents triggered. Human action required to refresh token.

E06

Review text in non-English language

Review received in Spanish: 'Servicio excelente, muy recomendable.' Star rating: 5. Platform: Google Business Profile.

Classifier identifies language as non-English. Sentiment classified as positive from text signals. Topic summary generated in English. Operator flag set: 'non_english_review'. Draft response generated in same language as review.

Notion row has language_flag=non_english. Sentiment=positive. Draft response language matches review language. No error raised.

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use only an internal test reviewer account or a designated test business listing that can have its content deleted. Do not post test responses to any live customer-facing review page that cannot be immediately removed. After each test case completes, void the test record: delete the test review from the platform, remove the Notion row, delete the HubSpot test activity, and confirm the Slack messages are archived. The FullSpec team confirms this cleanup in the execution log before proceeding to the next case.

Scenario

Input

Expected Output

Pass Criteria

T21

Production run: 5-star positive review, full chain

Test reviewer posts a 5-star review to the designated test listing on Google Business Profile. Review text: 'Great team, very responsive.'

Polling trigger detects review within 15 minutes. Classifier returns sentiment=positive. Notion row created. Draft generated and auto-approved. Response posted to Google Business Profile via API. HubSpot timeline activity logged.

All six pipeline stages complete with status=success in the production execution log. Response visible on test listing. Notion row and HubSpot activity confirmed. End-to-end time under 15 minutes.

T22

Production run: 2-star negative review, Slack escalation and approval

Test reviewer posts a 2-star review to the test listing on Trustpilot. Review text: 'Took too long and communication was poor.'

Classifier returns sentiment=negative, escalation=true. Slack alert delivered to manager channel with draft and approval link. Marketing Manager clicks approve. Response posted to Trustpilot. Notion and HubSpot updated.

Slack alert received in under 5 minutes. Approval link functional. Post confirmed on Trustpilot test listing. Notion status=complete. HubSpot activity present.

T23

Production run: error log verification

Deliberately inject a malformed review payload (missing review_text field) via the test trigger during the production session.

Classifier detects missing field. Error event written to production error log. Review held in pending queue. FullSpec monitoring alert fires.

Error log entry present with timestamp, review ID, and reason=missing_review_text. No downstream agents triggered. Monitoring alert confirmed received.

T24

Production run: CRM contact match on known reviewer

Test reviewer account email matches a known contact in the production HubSpot instance (pre-seeded test contact).

CRM agent matches reviewer email to HubSpot contact. Timeline activity logged: platform=Google Business Profile, sentiment=positive, star_rating=5, status=responded.

HubSpot test contact timeline shows new activity. All four activity fields correct. No other contacts modified.

T25

Production run: weekly Slack report accuracy

Notion production tracker contains the review rows created during T21 and T22 (plus any pre-existing test rows). Report trigger fired manually for this test.

Slack report posted to marketing channel. Review count, sentiment breakdown, response rate, and top theme all reflect actual Notion data.

All four report fields verified against Notion source rows. No discrepancy greater than zero. Report delivered to correct Slack channel.

T26

Production run: Notion audit trail completeness

Review rows created during T21 and T22 inspected in the production Notion tracker.

Each row contains: review ID, platform, star rating, sentiment, confidence score, topic summary, draft response, approval status, response posted timestamp, and HubSpot contact match status.

Both rows contain all ten required fields with no null values except HubSpot match status (permitted null for unmatched reviewers). Timestamps are accurate.

T27

Process owner review and approval (final sign-off)

Process owner and Marketing Manager review all Phase 3 execution logs, Notion rows, HubSpot activities, Slack messages, and the production test listing responses compiled by the FullSpec team.

Process owner confirms the automation has met all acceptance criteria across Phases 1, 2, and 3. Approval is recorded by the FullSpec team in the confirmation log at support@gofullspec.com. No separate sign-off block is required.

FullSpec confirmation log entry timestamped and referencing T27. Entry states process owner name, date of approval, and sign-off scope. System cleared for go-live.

Once T27 is recorded in the FullSpec confirmation log, all sandbox credentials are revoked, test records are permanently deleted, and the production automation is enabled for live review monitoring. Contact support@gofullspec.com with the confirmation log reference if any issue arises after go-live.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process