FS-DOC-06Quality
Test and QA Plan
Client Reporting and Status Updates
Process
Client Reporting and Status Updates
Build option
Standard (recommended)
Total build effort
56 hours
Agents under test
Status Gathering Agent, Update Drafting Agent, Send and Log Agent
01Testing approach
Testing is structured across three sequential phases. Each phase must reach its pass condition before the next begins. Phase 1 validates individual agent logic in isolation. Phase 2 confirms inter-agent data flow and system edge cases. Phase 3 runs a full end-to-end rehearsal against production credentials with live accounts suppressed where noted.
1Phase 1: Unit Testing
Scope: each agent tested in isolation against mock data
Scope
Status Gathering Agent, Update Drafting Agent, Send and Log Agent tested independently
Environment
Staging workspace in FullSpec Automation; sandboxed Asana project; Google Docs test folder; HubSpot sandbox; Gmail test account
Data
Mock Asana tasks with known status values; pre-authored status JSON fixtures; dummy HubSpot contact records
Pass condition
Each agent produces the correct output schema for every test case with zero unhandled errors; all assertions in Section 02 return PASS
2Phase 2: Integration and Edge Case Testing
Scope: agent-to-agent handoffs and system boundary conditions
Scope
Full agent chain from trigger through to CRM log; edge cases including empty Asana boards, API rate limits, and draft rejection paths
Environment
Staging environment with live API credentials (read-only scopes where possible); real Asana workspace restricted to a test project; HubSpot sandbox
Data
Mixture of fixture data and live staging records; no real client email addresses used
Pass condition
All inter-agent payloads match the agreed field contracts; edge cases are handled gracefully with correct error routing; no data leaks to production CRM or real client inboxes
3Phase 3: End-to-End Production Test
Scope: full run in production environment with suppressed email delivery
Scope
Complete workflow from reporting window trigger through to CRM log entry, covering the approved and rejected review paths
Environment
Production FullSpec Automation instance; live Asana, Google Docs, HubSpot, and Gmail credentials; email delivery intercepted via a test-mode flag or internal-only recipient
Data
One nominated real account with consent; client email replaced with an internal QA address for the duration of testing
Pass condition
Update drafted matches house style; account manager review step fires correctly; approved update reaches QA inbox and is logged in HubSpot; elapsed time per update is under 15 minutes end-to-end
No real client email addresses may be used during Phase 1 or Phase 2. Phase 3 must use an internal QA recipient address until sign-off is complete.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 unit test cases
The following tables cover unit test cases for each agent. Run cases in ID order. Record actual output and PASS or FAIL against each case before progressing to Phase 2.
Status Gathering Agent
ID
Scenario
Input
Expected output
Pass criteria
U-SG-01
Standard active account
Asana project with 5 tasks: 3 complete, 1 in-progress, 1 blocked
Structured status JSON: completed_count=3, in_progress_count=1, blockers=[task_id]
Output matches schema; blocker array is non-empty; no null fields
U-SG-02
All tasks complete
Asana project with 4 tasks: all marked complete
Structured status JSON: completed_count=4, in_progress_count=0, blockers=[]
blockers array is empty; status field reads 'all_complete'
U-SG-03
Empty project board
Asana project with zero tasks
Error event dispatched: code=NO_TASKS_FOUND; workflow halted gracefully
No unhandled exception; error event payload contains account_id and timestamp
U-SG-04
Asana API timeout
Simulated 30-second timeout on Asana tasks endpoint
Retry attempted twice then error event: code=ASANA_TIMEOUT
Retry count=2 logged; downstream agents not triggered
U-SG-05
Multiple accounts in window
Three Asana projects triggering simultaneously
Three independent status JSON objects emitted in sequence
Each object contains distinct project_id; no field cross-contamination
Update Drafting Agent
ID
Scenario
Input
Expected output
Pass criteria
U-UD-01
Standard status with blockers
Status JSON from U-SG-01; house style template loaded
Google Doc created; draft contains summary, progress bullets, blocker section, and next steps
Doc URL returned in payload; all four sections present; tone matches house style guide
U-UD-02
All-complete status, no blockers
Status JSON from U-SG-02
Google Doc created; blocker section omitted; next steps section reads 'project on track'
Blocker section absent from doc; doc URL returned; character count greater than 150
U-UD-03
Missing template
Status JSON with valid data; house style template file deleted or inaccessible
Error event: code=TEMPLATE_NOT_FOUND; no doc created
No orphaned Google Doc created; error payload includes account_id
U-UD-04
Special characters in task names
Asana task names containing HTML entities and Unicode characters
Draft rendered correctly with special characters escaped or displayed as plain text
No encoding artifacts visible in Google Doc; doc URL returned
U-UD-05
Long task list exceeding token budget
Asana project with 80 tasks
Draft truncated gracefully to top 20 items by priority; note appended indicating truncation
Truncation note present; no API error thrown; doc URL returned
Send and Log Agent
ID
Scenario
Input
Expected output
Pass criteria
U-SL-01
Approved draft, valid recipient
Approved Google Doc URL; HubSpot contact with valid email; Gmail credentials active
Email sent to QA inbox; HubSpot activity note created against contact
Gmail message_id returned; HubSpot activity_id returned; both logged in run record
U-SL-02
Draft rejected by account manager
Review decision=rejected; Google Doc URL
Workflow halted; no email sent; HubSpot note created with status='draft_rejected'
Gmail send not called; HubSpot note contains 'draft_rejected'; account manager notified
U-SL-03
HubSpot contact not found
Approved draft; account_id with no matching HubSpot contact
Email sent to QA inbox; HubSpot error event: code=CONTACT_NOT_FOUND; send still completes
Email delivered; error event logged; run does not fail entirely
U-SL-04
Gmail send failure
Valid draft; Gmail API returns 429 rate limit error
Retry attempted twice; error event: code=GMAIL_SEND_FAILED after retries exhausted
Retry count=2 in logs; HubSpot note not created; alert raised to operator
U-SL-05
Duplicate send guard
Same approved draft submitted twice within 60 seconds
Second send blocked; idempotency key match logged
Only one Gmail message_id exists for the draft_id; one HubSpot activity note created
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 integration and edge case tests
Integration tests verify the full agent chain. Edge case tests confirm that boundary conditions, invalid states, and external API failures are handled without data loss or unintended sends.
Integration tests
ID
Scenario
Agents involved
Expected behaviour
Pass criteria
I-01
Full happy-path chain, no edit needed
Status Gathering, Update Drafting, Send and Log
Trigger fires; status JSON passed to drafting agent; doc created; auto-approved path selected; email sent; HubSpot logged
Gmail message_id and HubSpot activity_id present in final run record; elapsed time under 15 min
I-02
Account manager edit path
Status Gathering, Update Drafting, Send and Log
Draft pauses at review step; account manager edits doc; approval submitted; Send and Log fires
Revised doc URL used for send; HubSpot note reflects edited draft; original draft URL not sent
I-03
Status JSON schema mismatch between agents
Status Gathering to Update Drafting
Status Gathering emits a payload missing the blockers field
Update Drafting catches missing field; uses empty array default; does not crash; doc created
Doc created without blocker section; warning logged; no unhandled exception
I-04
CRM and email log both succeed atomically
Send and Log, HubSpot, Gmail
Email sends successfully; HubSpot call then fails on first attempt but succeeds on retry
Email sent; HubSpot activity created after retry; retry count visible in logs
I-05
Concurrent runs for two accounts
All three agents running in parallel for two accounts
Two independent workflow runs execute without field cross-contamination
Each run has a unique run_id; no fields from account A appear in account B output
Edge case tests
ID
Scenario
Input condition
Expected handling
Pass criteria
E-01
Reporting window fires outside business hours
Cron trigger at 02:00 on a Saturday
Workflow queued; execution deferred to next configured business window
No email sent at 02:00; run_id created with status='deferred'
E-02
Asana project archived mid-run
Project archived after trigger fires but before status fetch completes
Error event: code=PROJECT_ARCHIVED; run halted; operator alerted
No draft created; alert payload contains project_id and account_id
E-03
Google Docs API quota exceeded
Docs API returns 429 during doc creation
Retry with exponential backoff up to 3 attempts; error event if all retries fail
Retry count visible in logs; if all fail, downstream agents not triggered
E-04
HubSpot contact has no email address set
HubSpot contact record lacks email field
Send and Log uses fallback email from account config; logs warning
Email delivered to fallback address; warning flag set in run record
E-05
Account manager approves wrong draft version
Stale doc URL submitted after a newer draft exists
System detects version mismatch via doc modified timestamp; rejects approval; prompts re-review
Stale approval blocked; account manager receives re-review prompt; no email sent from stale doc
E-06
Partial Asana data: some tasks have no assignee
Mix of assigned and unassigned tasks in project
Status Gathering groups unassigned tasks under 'unassigned' key; structure intact
Output JSON contains unassigned array; downstream agents parse without error
E-07
Network interruption mid-workflow
TCP connection dropped between Update Drafting and Send and Log
Workflow engine detects incomplete step; resumes from last successful checkpoint
No duplicate email sent; run completes from checkpoint; final status logged correctly
Edge cases E-05 and E-07 require a manual simulation step during testing. Document the simulation method used in the test run log.
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 end-to-end production test
Phase 3 runs against live production credentials with email delivery directed to an internal QA address. One nominated account is used throughout. All cases must be executed sequentially in a single reporting window.
ID
Scenario
Steps
Expected result
Pass criteria
P3-01
Scheduled trigger fires correctly
Confirm cron schedule is active; wait for or manually invoke the reporting window trigger for the test account
Run initialised with correct account_id and timestamp; Status Gathering Agent called
Run log entry created; account_id matches test account; trigger timestamp within 60 seconds of scheduled time
P3-02
Status gathered from live Asana
Allow Status Gathering Agent to query the live test Asana project
Structured status JSON produced reflecting actual task state at time of run
JSON contains valid completed_count, in_progress_count, and blockers fields; no null values
P3-03
Draft produced in Google Docs
Allow Update Drafting Agent to create a doc in the designated QA folder
Google Doc created in QA folder with summary, progress, and next steps sections
Doc URL returned; doc visible in QA Google Drive folder; all required sections present
P3-04
Account manager review step fires
Confirm review notification reaches the account manager; account manager opens the doc and submits approval
Review task appears in account manager inbox or FullSpec Automation review queue
Review notification delivered; approval action available; account manager able to edit and approve
P3-05
Approved update sent to QA inbox
Account manager approves draft; Send and Log Agent fires
Email delivered to internal QA inbox; email body matches the approved Google Doc content
QA inbox receives email; subject line matches expected format; body content matches approved doc; no client address in To field
P3-06
Activity logged in HubSpot
Inspect HubSpot contact record for the test account after send
Activity note created against test contact with correct date, subject, and body excerpt
HubSpot activity_id present in run log; note visible on contact timeline; timestamp matches send time
P3-07
Rejected draft path in production
Repeat trigger; account manager selects reject on the draft
No email sent; HubSpot note created with status draft_rejected; account manager notified
QA inbox receives no email; HubSpot note status field reads 'draft_rejected'; rejection timestamp logged
P3-08
End-to-end elapsed time
Measure wall-clock time from trigger to CRM log for P3-01 through P3-06
Total elapsed time from trigger to CRM log entry is under 15 minutes excluding account manager review dwell time
Automated steps complete in under 15 minutes; review dwell time excluded and documented separately
If P3-05 fails due to Gmail send issues, halt the test run, escalate to the developer, and do not retry until the root cause is identified. Do not re-run against a real client email under any circumstances.
05Go-live pass criteria
All requirements below must be met before the automation is cleared for live operation on real client accounts. Each requirement maps to one or more test IDs. Status must be confirmed by both the developer and the process owner.
Requirement
Test IDs
Status
All Phase 1 unit tests return PASS with zero unhandled exceptions
U-SG-01 to U-SG-05, U-UD-01 to U-UD-05, U-SL-01 to U-SL-05
To confirm
All Phase 2 integration tests return PASS
I-01 to I-05
To confirm
All Phase 2 edge case tests return PASS or documented accepted risk
E-01 to E-07
To confirm
All Phase 3 production test cases return PASS
P3-01 to P3-08
To confirm
End-to-end elapsed time (automated steps only) is under 15 minutes
P3-08
To confirm
No real client email address received a test email at any point during testing
U-SL-01, I-01, I-02, P3-05
To confirm
HubSpot activity notes created correctly for both approved and rejected paths
U-SL-01, U-SL-02, P3-06, P3-07
To confirm
Duplicate send guard confirmed functional
U-SL-05
To confirm
Error events route correctly and do not expose sensitive data in payloads
U-SG-03, U-SG-04, U-UD-03, U-SL-03, U-SL-04, E-02
To confirm
All credentials stored as encrypted environment variables; none hardcoded in workflow nodes
Developer review; no specific test ID
To confirm
Process owner has reviewed Phase 3 output (draft quality and CRM log) and approved the result
P3-02, P3-03, P3-05, P3-06
To confirm
Any requirement marked 'To confirm' at the time of the go-live decision must be escalated to the process owner. A documented accepted risk is only valid if signed by both the developer and the process owner.
06Sign-off
Both parties must sign below to confirm that all go-live pass criteria in Section 05 have been reviewed, that any accepted risks are documented, and that the automation is approved for live operation on real client accounts.
Test and QA PlanPage 4 of 4