Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Marketing Attribution Reporting

[YourCompany.com] · Marketing Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance plan for the Marketing Attribution Reporting automation. It covers all three sequential test phases: unit tests for each agent in isolation, integration and edge-case tests for agent handoffs and failure modes, and a final end-to-end production run. The FullSpec team executes Phases 1 and 2 independently. Your team joins Phase 3 to review the live output and confirm sign-off. No phase begins until the previous phase achieves a full pass on every test case.

01Testing approach

Testing runs in three sequential phases. Phase 1 validates each agent in isolation using sandbox credentials and synthetic data. Phase 2 tests the handoffs between agents and systematically exercises edge cases including duplicates, missing fields, unavailable APIs, and human-review timeouts. Phase 3 executes a full end-to-end run against the production environment with real credentials and a live reporting period. A phase may not begin until every test case in the preceding phase carries a recorded pass result. Any failure reopens the affected phase; the FullSpec team resolves the defect and re-runs all cases in that phase from T01 before advancing.

Phase 1: Unit Testing

Each agent tested in isolation using sandbox credentials and synthetic data

Scope

Individual agent behaviour: Data Fetch Agent (all four API calls), Attribution and Merge Agent (normalisation and model logic), Summary and Delivery Agent (summary generation and Slack post)

Environment

Sandbox only. No production credentials, no live ad accounts, no real Slack channels.

Credential rule

Sandbox OAuth tokens and test API keys only. Production credentials must not be loaded or referenced during Phase 1.

Pass condition

All cases T01 through T14 return expected outputs with no unhandled errors. Failures in any case block progression.

Who runs it

FullSpec team only. Process owner not required.

Phase 2: Integration and Edge-Case Testing

Agent handoffs and failure modes tested end-to-end in sandbox

Scope

Agent-to-agent handoff triggers, chain completion logic, plus edge cases covering duplicate data, missing fields, API unavailability, and human-review timeout handling

Environment

Sandbox for integration chain tests; controlled simulation for edge cases (stubbed API errors, injected duplicate rows).

Credential rule

Sandbox credentials only. Edge-case API failures are simulated via stub responses, not by testing against live platform error states.

Pass condition

All cases T15 through T21 and edge cases E01 through E07 pass without data loss, unhandled exceptions, or incorrect downstream writes.

Who runs it

FullSpec team only. Process owner not required.

Phase 3: End-to-End Production Test

Full live run traced against a real reporting period; process owner participates

Scope

Complete flow from scheduled trigger through data fetch, merge, attribution model application, anomaly detection, Slack delivery, and error log verification

Environment

Production. Real OAuth credentials, live ad accounts, live Google Sheet, and the designated Slack channel using an internal test address.

Credential rule

Production credentials active. The test uses an internal Slack channel and a clearly labelled test record in Google Sheets. The test record is voided after sign-off.

Pass condition

All cases T22 through T26 pass. Process owner reviews the Slack output and confirms accuracy. Owner sign-off is recorded in the FullSpec confirmation system (case T26).

Who runs it

FullSpec team executes the run. Process owner (Marketing Manager) reviews the output and confirms sign-off as the final test case.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only during Phase 1. No production OAuth tokens, live ad accounts, or real Slack workspaces may be loaded at any point in this phase. All API calls must target sandbox endpoints or return synthetic fixture data. Any test that inadvertently touches a production system is immediately voided and the credential configuration reviewed before the case is re-run.

Data Fetch Agent — fetches raw performance data from Google Ads, Meta Ads Manager, Google Analytics 4, and HubSpot on schedule, then writes structured tables to the staging Google Sheet. Complexity: Moderate. Estimated build time: 14 hours.

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: Google Ads API fetch

Scheduled trigger fires; sandbox Google Ads OAuth token; date range = last 7 days; synthetic campaign data available

Campaign, ad group, and conversion rows written to the Google Ads tab of the staging sheet with correct date range and field alignment

Row count matches synthetic fixture; all required fields (campaign_id, spend, clicks, conversions, date) present; no null values in mandatory columns

T02

Happy path: Meta Ads Manager API fetch

Sandbox Meta system user token; date range = last 7 days; synthetic ad set and creative data available

Spend, impressions, clicks, and conversions written to the Meta tab of the staging sheet

All four metric fields populated; date range boundaries match input; currency field returns USD

T03

Happy path: Google Analytics 4 API fetch

Sandbox GA4 property; reporting API call for source/medium dimension; date range = last 7 days

Source/medium rows with sessions and goal completions written to the GA4 tab

Source/medium dimension populated; sessions and conversions non-null; date range aligns with ad platform fetch

T04

Happy path: HubSpot deal source fetch

Sandbox HubSpot private app token; deal filter: closedate within last 7 days and dealstage = closedwon

Closed deal records with original source, lifecycle stage, and deal amount written to the HubSpot tab

Deal records contain hs_original_source, amount, and closedate; records outside date range excluded; no duplicate deal IDs

T05

Google Ads API auth failure

Expired or revoked sandbox OAuth token injected

Agent logs authentication error, marks the Google Ads fetch as failed, and halts further steps pending retry

Error event written to the run log with error_code and timestamp; no partial data written to the sheet; downstream agents do not fire

T06

Meta Ads API rate limit hit

Stub response returns HTTP 429 after the first page of results

Agent pauses, applies exponential backoff (30 s, 60 s, 120 s), retries up to three times, then logs a rate-limit failure if all retries are exhausted

Backoff intervals logged; retry count recorded; if all retries fail, fetch marked incomplete and operator alert triggered; no silent failure

T07

GA4 API returns empty dataset

Sandbox property has no events in the requested date range

Agent writes an empty-result marker to the GA4 tab and logs a data-gap warning

Empty-result marker present; warning log entry created; Attribution and Merge Agent does not proceed using a blank GA4 input without acknowledgement

T08

HubSpot API pagination incomplete

Stub returns three pages but the fourth page returns a 500 error mid-pagination

Agent logs the pagination failure at page 4, writes the records retrieved up to page 3, and flags the fetch as partial

Partial flag set on HubSpot fetch status; records from pages 1 to 3 written correctly; page 4 gap logged with offset value for retry

Attribution and Merge Agent — normalises UTM and source labels, merges all platform data into the master Google Sheet, and applies the configured attribution model to produce per-channel CPA and ROAS. Complexity: Moderate. Estimated build time: 16 hours.

Scenario

Input

Expected Output

Pass Criteria

T09

Happy path: merge and attribution model applied

All four platform tabs populated in the staging sheet; UTM labels consistent across sources; attribution model = last-touch

Merged attribution table written with CPA and ROAS calculated per channel; source labels normalised to the canonical set

CPA and ROAS values present for every channel row; source labels match the normalisation dictionary; no duplicate channel rows; sheet marked complete

T10

UTM normalisation: mixed-case and variant labels

Synthetic data contains utm_source values: 'google', 'Google', 'google_ads', 'GoogleAds'

All four variants normalised to 'google_ads' in the canonical source column

Canonical source column contains only 'google_ads' for all four input variants; original raw label preserved in a separate raw_source column

T11

Attribution model: linear distribution

Same merged dataset; attribution model switched to linear in the agent configuration

Conversion credit distributed equally across all touchpoints in the path; CPA recalculated accordingly

Sum of attributed conversions per channel equals total conversions; CPA figures differ from last-touch output by expected proportion

T12

Merge with missing Meta data

Meta Ads tab is empty (fetch marked incomplete in T06 failure scenario)

Agent writes a data-gap flag for the Meta channel in the merged table; CPA and ROAS for Meta left as null; all other channels calculated normally

Meta rows present with null metric values and a data_gap flag; no formula errors propagated to other channel rows; sheet still marked complete with a warning

T13

Duplicate campaign rows in source data

Synthetic Google Ads input contains two identical rows for the same campaign_id and date

Agent deduplicates on (campaign_id, date) before merge; only one row retained per key

Merged sheet contains exactly one row per (campaign_id, date) combination; deduplication event logged with count of removed rows

T14

Attribution model misconfiguration

Attribution model field in agent config is blank or contains an unrecognised string

Agent halts before writing to the attribution table and raises a configuration error

Configuration error logged with the invalid model value; no partial attribution rows written; operator alert sent

Summary and Delivery Agent — reads the completed attribution table, identifies the top-performing channel and any anomalies, writes a plain-language summary, and posts the report link to Slack. Complexity: Simple. Estimated build time: 8 hours.

Scenario

Input

Expected Output

Pass Criteria

T15

Happy path: summary generated and posted to Slack

Attribution table complete with data for all four channels; no anomaly flags; sandbox Slack webhook configured

Plain-language summary paragraph posted to the sandbox Slack channel with the Google Sheet link; top channel identified by ROAS

Slack message received in the sandbox channel; message body contains the sheet URL, top channel name, and at least one metric value; delivery timestamp logged

T16

Anomaly detected: ROAS drop greater than 30%

Meta channel ROAS drops from 4.2 to 2.1 week-over-week in the synthetic dataset

Summary flags the Meta ROAS anomaly with the percentage drop; Slack message includes an anomaly callout alongside the standard summary

Anomaly section present in the Slack message; percentage drop value accurate to one decimal place; no other channels incorrectly flagged

T17

Slack webhook unreachable

Sandbox Slack endpoint returns HTTP 503

Agent retries the post twice with a 30-second interval; if all retries fail, logs a delivery failure and writes the summary text to the error log for manual retrieval

Retry count and timestamps logged; delivery failure entry present in the run log; summary text accessible in the log even though Slack post did not succeed

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Phase 2 tests the handoffs between agents and the full chain from trigger to Slack delivery. Test IDs continue from T18. A separate edge-case table follows, using the E01 sequence.

Scenario

Input

Expected Output

Pass Criteria

T18

Trigger fires and all four fetches complete: Data Fetch Agent signals Attribution and Merge Agent

Scheduled trigger activates; all four API fetches succeed and all four platform tabs written in the sandbox sheet

Data Fetch Agent writes a completion flag to the run state; Attribution and Merge Agent fires within 60 seconds of the flag being set

Completion flag present in run state; Attribution and Merge Agent start timestamp within 60 s of Data Fetch Agent end timestamp; no manual intervention required

T19

Attribution and Merge Agent signals Summary and Delivery Agent on sheet completion

Attribution table fully written and the sheet marked complete by the Attribution and Merge Agent

Summary and Delivery Agent fires, reads the sheet, generates the summary, and posts to the sandbox Slack channel

Summary and Delivery Agent start timestamp follows Attribution and Merge Agent completion flag; Slack post arrives within 90 seconds of the sheet being marked complete

T20

One fetch fails mid-chain: Attribution and Merge Agent does not fire prematurely

Google Ads fetch fails (simulated HTTP 401); remaining three fetches succeed

Data Fetch Agent marks the run as partially complete; Attribution and Merge Agent does not fire; operator alert is sent

Attribution and Merge Agent has no execution record for this run; partial-complete status recorded; alert payload contains the identity of the failing source

T21

Full chain: anomaly detected, human review step triggered, chain resumes

Merged sheet contains a spend spike on the Google Ads channel exceeding the configured anomaly threshold; sandbox Slack channel available

Summary and Delivery Agent flags the anomaly in the Slack message; the marketing manager review step is surfaced; after simulated acknowledgement, the report is marked reviewed

Anomaly flag present in the Slack message; review step logged in the run record; on simulated acknowledgement, run status updates to reviewed-complete

Edge-case tests below target boundary conditions and failure scenarios that fall outside the standard agent flow. These are run by the FullSpec team using injected data and stub responses.

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate run triggered: scheduler fires twice within the same reporting window

Two trigger events fire with identical report period parameters within 5 minutes of each other

The second run is detected as a duplicate by the run-state check; it is suppressed and a duplicate-suppression event is logged

Only one execution record exists for the reporting period; duplicate-suppression log entry contains both trigger timestamps

E02

Missing utm_medium on 40% of GA4 rows

Synthetic GA4 export contains 40% of rows with a blank utm_medium field

Normalisation logic assigns 'unknown' to blank utm_medium values; a data-quality warning is added to the run log with the row count and percentage

No rows dropped; all rows with blank utm_medium carry the 'unknown' label; warning log entry present with an accurate count

E03

HubSpot API unavailable for the full retry window

Stub returns HTTP 503 on all three retry attempts for the HubSpot fetch

HubSpot fetch marked failed after the final retry; the run continues for the other three sources; the merged table flags HubSpot as missing; operator alert sent

HubSpot tab contains a failed-fetch marker; other three platform tabs populated correctly; alert delivered; attribution table notes the missing bottom-of-funnel layer

E04

GA4 API returns a date range one day shorter than requested

GA4 sandbox property returns data for days 1 to 6 of a 7-day window due to processing delay on the most recent day

Agent writes the 6-day data and logs a date-boundary warning noting the missing day; the merged table carries a date-gap flag for the GA4 channel

Six rows of GA4 data present; day 7 absent; date-gap flag on the GA4 tab; warning log entry specifies the missing date

E05

Google Ads returns a campaign with zero spend and zero conversions

Synthetic data includes a paused campaign row with all metric values at zero

Agent writes the zero-value row to the staging tab; normalisation and merge proceed without errors; zero-value rows do not affect CPA or ROAS calculations for other campaigns

Zero-value row present in the merged sheet; CPA and ROAS columns for that row display null or zero correctly; no division-by-zero error propagated

E06

Google Sheets write permission revoked mid-run

The service account's editor permission on the staging sheet is revoked after the Google Ads fetch completes but before the Meta fetch writes

Agent catches the permission error on the Meta write attempt, logs a sheet-access failure, and halts the run without corrupting existing data

Google Ads tab retains its written data; Meta write did not partially complete; permission error logged with the sheet ID and the affected tab name; no unhandled exception

E07

Human anomaly review timeout: no acknowledgement within 24 hours

The anomaly review step is surfaced to the marketing manager via Slack; no acknowledgement is received within the 24-hour timeout window

The automation escalates the alert to the Head of Growth Slack handle, logs the timeout event, and marks the run as pending-escalation

Escalation message posted to Head of Growth handle after 24 hours; run status set to pending-escalation; original anomaly flag preserved in the run record

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test Slack channel (not the live stakeholder channel) for the initial production run. Label the Google Sheet output clearly as a test record. After the process owner confirms sign-off, void the test record in Google Sheets by moving it to a clearly marked archive tab and removing it from any live reporting views. Do not share the Phase 3 output with external stakeholders or include it in any business reporting. Production OAuth credentials are active during this phase; ensure all tokens have been scoped to the minimum required permissions before the run begins.

Phase 3 exercises the full production flow against a real reporting period. The FullSpec team initiates the run, monitors each stage, and captures the run log. The process owner (Marketing Manager) reviews the Slack output and the completed Google Sheet, then confirms accuracy before sign-off is recorded. Test IDs continue from T22.

Scenario

Input

Expected Output

Pass Criteria

T22

Full production run: trigger to Slack delivery

Scheduled trigger fires for the most recently completed 7-day reporting window; production OAuth tokens for Google Ads, Meta, GA4, and HubSpot active; production Google Sheet and internal test Slack channel configured

All four platform data sets fetched, merged, and attributed; CPA and ROAS calculated per channel; plain-language summary and sheet link posted to the internal test Slack channel within 20 minutes of the trigger firing

Slack message received in the test channel; sheet populated with data for all four channels; delivery time from trigger to Slack post under 20 minutes; no unhandled errors in the run log

T23

Production data accuracy: spot-check against manual benchmark

The Attribution and Merge Agent output for Google Ads spend and conversions is compared against a manual export from the Google Ads dashboard for the same 7-day period

Automated figures for total spend and total conversions match the manual dashboard export within a 1% tolerance (accounting for API vs UI reporting windows)

Variance for spend within 1%; variance for conversions within 1%; any variance outside tolerance is documented and the source of the discrepancy identified before the test is marked passed

T24

UTM normalisation accuracy in production

The production Google Sheet's canonical source column is reviewed for all rows in the test run

All source labels conform to the agreed normalisation dictionary; no raw ad-platform naming variants appear in the canonical column

Zero rows with non-canonical source labels in the canonical source column; any unlisted variants are flagged in the run log for normalisation dictionary update

T25

Error log verification: clean run

The run log for the Phase 3 test run is reviewed in full by the FullSpec team

Run log contains no error-level entries; all four fetch steps, the merge step, and the Slack delivery step show completed status with timestamps

All six pipeline stages present in the log with a completed status; no error or warning entries except expected data-quality notices (such as paused campaigns with zero values); log retained and accessible

T26

Process owner review and sign-off

The Marketing Manager reviews the Slack summary message and the production Google Sheet output from the Phase 3 run; the FullSpec team provides the run log for reference

The Marketing Manager confirms that the Slack summary is accurate, the per-channel CPA and ROAS figures are consistent with their expectations for the period, and the anomaly detection output (if any) is correctly described

Process owner provides written confirmation via the FullSpec confirmation system that the output is accurate and the automation is approved for live operation. This confirmation is the formal sign-off for the Marketing Attribution Reporting automation. No separate signature block is required. The FullSpec team records the confirmation timestamp and the name of the approving process owner before closing the QA phase.

After T26 is passed and the process owner's confirmation is recorded, the FullSpec team will archive the test record in Google Sheets, switch the Slack delivery target from the internal test channel to the live stakeholder channel, and confirm the automation is active for the next scheduled run. Contact support@gofullspec.com for any questions or issues arising after go-live.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process