FS-DOC-06Quality
Test and QA Plan
Project Milestone Tracking
[YourCompany.com] · Operations Department · Prepared by FullSpec · [Today's Date]
This document is the authoritative quality gate for the Project Milestone Tracking automation. It defines every test case the FullSpec team runs before any agent reaches production, the conditions each case must satisfy to pass, and the sequence in which testing proceeds. Testing covers all three agents: the Milestone Monitor Agent, the Owner Reminder Agent, and the Report Builder Agent. The process owner joins Phase 3 to confirm real-world behaviour and record sign-off. No phase begins until every test case in the preceding phase has a confirmed pass status.
01Testing approach
Testing is structured across three sequential phases: Unit, Integration, and End-to-end. No phase begins until the previous phase fully passes. Phase 1 exercises each agent in isolation using sandbox credentials and synthetic data. Phase 2 verifies agent handoffs and edge-case resilience within the integration environment. Phase 3 runs a single traced production run with real credentials and real project data, ending with a formal process owner review.
1Phase 1: Unit Testing
Each agent tested in isolation before any handoff is wired
Scope
Individual agent logic: trigger conditions, data reads, writes, and outbound actions for the Milestone Monitor Agent, Owner Reminder Agent, and Report Builder Agent tested independently
Environment
Sandbox only: Asana sandbox workspace, Google Sheets test spreadsheet, Gmail test account, Slack test workspace, Notion test page
Credential rule
No production credentials permitted. All API tokens must be sandbox or test-scoped. OAuth connections must point to sandbox apps registered under the FullSpec test organisation
Pass condition
All test cases T01 through T15 return the expected output with zero unhandled errors
Who runs it
FullSpec team only. Process owner is not required for Phase 1
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure modes verified end to end in the integration environment
Scope
Agent-to-agent data handoffs, conditional routing logic, and all identified edge cases including missing data, duplicate records, tool unavailability, and human timeout scenarios
Environment
Integration environment: sandbox credentials remain in use; live API rate limits observed to confirm the automation behaves within service quotas
Credential rule
Sandbox credentials only. Integration tests must not write to any live Asana project, production Google Sheet, real Slack channel, or live Notion workspace
Pass condition
All test cases T16 through T22 pass and all edge cases E01 through E06 either resolve gracefully or trigger the correct fallback behaviour with an error log entry
Who runs it
FullSpec team only. Process owner is not required for Phase 2
3Phase 3: End-to-End Production Test
Full production run traced from Asana trigger through to Slack digest, with process owner review
Scope
One complete, traced production run using real project data, real credentials, and live Slack channels and Notion workspace. Covers trigger to final digest and error log verification
Environment
Production: all live credentials active. A designated internal test project is used in Asana; test records are voided after the run
Credential rule
Production credentials active for this phase only. OAuth tokens must be scoped to the minimum required permissions as documented in the Integration and API Spec
Pass condition
All test cases T23 through T26 pass, including the final process owner review case (T26), and FullSpec logs the approval confirmation
Who runs it
FullSpec team executes T23 through T25. The process owner (Operations Manager) participates in T26 to review output and confirm approval
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. No test in Phase 1 may reference a live Asana project, a production Google Sheet, a real Gmail account, a live Slack channel, or a live Notion workspace. If a sandbox token is not available for a tool, the FullSpec team must mock the API response before proceeding. Do not substitute a production token.
Milestone Monitor Agent — Complexity: Moderate — Estimated build time: 14 hours
ID
Scenario
Input
Expected Output
Pass Criteria
T01
Happy path: milestone overdue by 1 day
Asana sandbox task with due date set to yesterday, status not complete
Task written to Google Sheets test sheet with overdue flag; escalation flag set to false (1 day, below 3-day threshold)
Row appears in sheet within 60 seconds of trigger; escalation column value is FALSE; all required fields populated
T02
Happy path: milestone overdue by 4 days
Asana sandbox task with due date 4 days ago, status not complete, no owner response
Task written to sheet with escalation flag set to true; row marked for escalation routing
Escalation column value is TRUE; breach threshold logic fires correctly at day 3 boundary
T03
Milestone completed on due date — no action expected
Asana sandbox task with today's due date, status marked complete
Task not written to the tracking sheet; no email or Slack action triggered
Zero rows added to sheet; no downstream actions fired; execution log confirms skip
T04
Multiple overdue milestones across two projects
Three Asana sandbox tasks across two projects, all overdue by 2 days
All three tasks written as separate rows to the tracking sheet with correct project name, owner, due date, and status
Three distinct rows in sheet; no row duplication; project name field matches source Asana project for each
T05
Asana task has no assignee set
Asana sandbox task overdue, assignee field empty
Task written to sheet with owner field set to a configurable fallback value (e.g. Unassigned); escalation flag follows normal threshold logic
Row written without error; owner field contains fallback string, not null or blank; no unhandled exception in execution log
T06
Google Sheets write fails due to API error
Simulated 500 error response from Sheets API on write attempt
Automation retries up to 3 times with exponential backoff; if all retries fail, error is written to the execution log with the affected task ID
Retry count matches configured maximum; error log entry present with task ID and error code; no silent failure
Owner Reminder Agent — Complexity: Moderate — Estimated build time: 12 hours
ID
Scenario
Input
Expected Output
Pass Criteria
T07
Happy path: reminder email sent to single owner
One flagged row in test sheet with a valid sandbox Gmail address as owner
Personalised Gmail reminder sent to sandbox address; email body contains task name, due date, and project context pulled from the Asana record
Email received in sandbox inbox within 2 minutes; task name, due date, and project name all match the Asana source record
T08
Happy path: Slack escalation posted for breach item
One row in test sheet with escalation flag TRUE and a configured sandbox Slack channel
Structured Slack message posted to sandbox channel; message includes milestone name, owner tag, days overdue count, and direct Asana task link
Message appears in sandbox Slack channel; owner is tagged using correct Slack user ID; Asana link resolves to correct task
T09
Multiple owners, personalised emails for each
Three rows in test sheet with three different sandbox owner email addresses
Three separate Gmail reminder emails sent, each addressed to the correct individual with their specific task details
Three emails received in sandbox; each email references only the tasks belonging to that owner; no cross-contamination of task details
T10
Owner email address missing from Asana task
Flagged row in sheet where the owner email field is blank
Reminder email not sent; error entry written to execution log identifying the task ID and missing field; escalation Slack alert posted to fallback channel if escalation threshold is also breached
No email sent; log entry present with task ID; if escalation flag is TRUE, fallback Slack message is posted to the configured ops fallback channel
T11
Gmail API rate limit reached mid-batch
Batch of 10 reminder emails triggered simultaneously, exceeding the Gmail per-second send quota
Automation queues remaining emails and retries with a delay; all 10 emails eventually sent; no emails dropped
All 10 emails received in sandbox inboxes; execution log shows queuing and retry events; total delivery time under 5 minutes
T12
Slack workspace API token expired
Valid rows with escalation flag TRUE; Slack OAuth token is expired or revoked in sandbox
Automation logs the Slack delivery failure with error code; does not retry indefinitely; FullSpec receives an alert via the orchestration layer error channel
Slack message not sent to test channel; error log contains HTTP 401 or 403 status code; retry count capped at configured maximum
Report Builder Agent — Complexity: Simple — Estimated build time: 10 hours
ID
Scenario
Input
Expected Output
Pass Criteria
T13
Happy path: Monday report generated in Notion
Test tracking sheet with 8 rows across 3 projects (3 on track, 3 at risk, 2 overdue); trigger set to Monday 7:30 am in sandbox scheduler
Notion sandbox page created with milestones grouped by project and status; headline counts match the sheet data
Notion page created within 3 minutes of trigger; project grouping correct; counts for on track, at risk, and overdue match source sheet exactly
T14
Happy path: Slack digest posted after Notion page creation
Notion page successfully created in T13 scenario
Slack digest message posted to sandbox leadership channel; message contains total on track, at risk, and overdue counts; includes a direct link to the Notion page
Slack message appears in sandbox leadership channel within 60 seconds of Notion page creation; counts match T13 output; Notion link resolves correctly
T15
Notion API unavailable at scheduled run time
Report Builder Agent fires at Monday 7:30 am; simulated Notion API 503 error
Automation retries up to 3 times at 5-minute intervals; if all retries fail, a fallback Slack message is posted to the leadership channel noting the report is delayed, with an estimated retry time
Retry count matches configuration; fallback Slack message posted to leadership sandbox channel; message clearly states report generation has been delayed and not skipped
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 verifies that data passes correctly between agents at each handoff point and that the automation behaves predictably when inputs are malformed, tools are unavailable, or human responses are absent. Integration test IDs continue the T sequence from Phase 1. Edge case tests use a separate E sequence.
ID
Scenario
Input
Expected Output
Pass Criteria
T16
Milestone Monitor Agent to Owner Reminder Agent handoff: standard path
Milestone Monitor Agent completes a daily run and writes 4 flagged rows (2 standard, 2 escalation) to the test sheet
Owner Reminder Agent fires automatically after the monitor run completes; sends 2 reminder emails and posts 2 Slack escalation messages
Owner Reminder Agent trigger timestamp occurs after Milestone Monitor Agent completion timestamp; all 4 rows processed with correct routing; no rows skipped
T17
Owner Reminder Agent to Milestone Monitor Agent feedback: owner updates Asana after reminder
Owner receives sandbox Gmail reminder and marks the Asana task as complete in the sandbox workspace
On the next Milestone Monitor Agent run, the completed task is no longer written to the sheet; no further reminder is sent for that task
Completed task absent from sheet after subsequent monitor run; no duplicate email sent; row in sheet updated or removed within the next scheduled cycle
T18
Report Builder Agent reads from live monitor sheet after Monday trigger
Milestone Monitor Agent completes its first Monday run at 8:00 am; Report Builder Agent fires at 7:30 am and at 8:05 am (re-trigger after monitor data is available)
Report Builder Agent at 7:30 am generates report from last available data; if configured to wait, the 8:05 am run uses the freshest Monday data
Report content reflects data state at the time of generation; no stale data from the previous week included; Notion page timestamp matches the Monday run date
T19
Full daily cycle: trigger through Slack escalation in one run
Asana sandbox contains 2 tasks overdue by 5 days with no owner response; daily 8:00 am trigger fires
Both tasks written to sheet, escalation flag set to TRUE for both; Owner Reminder Agent sends emails and posts Slack alerts; execution log records all steps in sequence
End-to-end execution completes within 5 minutes; all steps logged in sequence; Slack alerts contain correct task details for both milestones
T20
Full weekly cycle: trigger through Notion report and Slack digest
Test sheet contains 10 rows across 4 projects updated across the preceding week; Monday 7:30 am trigger fires for Report Builder Agent
Notion sandbox page created with all 10 milestones grouped correctly; Slack digest posted with accurate headline counts; no data from prior weeks included
Notion page contains exactly 10 milestone entries; project groupings match source sheet; Slack digest counts are arithmetically correct; prior-week rows excluded
T21
Escalation threshold boundary: 3 days overdue exactly
Asana sandbox task with due date exactly 3 days ago, no completion status, no owner response logged
Task written to sheet with escalation flag set to TRUE; Slack escalation posted
Escalation fires at day 3 boundary, not day 4; confirms threshold is inclusive at the configured value
T22
No overdue milestones: daily run produces no output
All Asana sandbox tasks have either a future due date or a complete status
No rows written to the tracking sheet; no reminder emails sent; no Slack messages posted; execution log confirms run completed with zero items flagged
Sheet unchanged after run; no outbound actions fired; log entry confirms zero-item result without error
Edge case tests: the following scenarios target conditions outside the happy path that could cause silent failures, duplicate actions, or data corruption if not handled explicitly.
ID
Scenario
Input
Expected Output
Pass Criteria
E01
Duplicate milestone entry: same task ID written twice
Milestone Monitor Agent runs twice in quick succession (e.g. manual re-trigger during the same day); same Asana task ID is processed in both runs
Deduplication logic identifies the existing row by task ID and updates it rather than inserting a second row; only one row per task ID exists in the sheet
Sheet contains exactly one row per Asana task ID after both runs; updated timestamp reflects the later run; no duplicate reminder emails sent
E02
Missing due date on Asana task
Asana sandbox task in scope has no due date set; all other fields complete
Task is skipped by the Milestone Monitor Agent; a warning entry is written to the execution log identifying the task ID and the missing field
Task not written to tracking sheet; log entry present with task ID; no email or Slack action triggered for this task; no unhandled exception
E03
Inconsistent milestone naming across Asana projects
Two sandbox projects use different naming conventions for milestone tasks (e.g. [MILESTONE] prefix vs no prefix)
Agent correctly identifies milestones in both projects based on the configured filter logic (task type, section, or tag rather than naming convention alone)
All milestone tasks identified regardless of naming prefix; zero false positives (non-milestone tasks included); configuration note flagged in execution log if naming inconsistency is detected
E04
Asana API unavailable at trigger time
Daily 8:00 am trigger fires; Asana returns a 503 Service Unavailable response
Automation retries up to 3 times at 2-minute intervals; if all retries fail, the run is skipped for that day and an error notification is sent to the FullSpec monitoring channel; no partial data is written to the sheet
Sheet contains no partial data from the failed run; monitoring channel receives the error notification within 10 minutes of the first failed attempt; next scheduled run proceeds normally
E05
Owner does not respond within 24 hours of initial Gmail reminder
Reminder email sent via Gmail at 8:05 am on day 1; no Asana status update or email reply received by 8:05 am on day 2; item is not yet at the 3-day escalation threshold
Owner Reminder Agent sends a second reminder email on day 2; execution log notes the non-response and increments the reminder count for that task
Second email sent to correct owner address with updated context referencing the previous reminder; reminder count in tracking sheet incremented to 2; no escalation Slack posted until the 3-day threshold is reached
E06
Human escalation review step times out: no stakeholder action after Slack alert
Slack escalation alert posted at 8:10 am; tagged stakeholder takes no action in Slack or Asana for 48 hours
Automation does not re-escalate automatically beyond the configured maximum reminder count; the item remains flagged in the tracking sheet; the execution log records the unresolved escalation status for manual follow-up
No infinite re-alert loop; item status in sheet remains escalated and unresolved; log entry timestamped at 48-hour mark; no further automated Slack posts for this item beyond the configured maximum
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use an internal test Asana project only, not any active client-facing or live project board. Set the test task owner email to an internal address controlled by the FullSpec team or the process owner so no real stakeholder receives an automated email or Slack message. After the production run is complete and all cases are confirmed passing, void the test task in Asana (mark complete and remove from scope), delete the test row from the live tracking sheet, archive the test Notion page, and confirm with the process owner that no residual test data remains in any live system. Do not proceed until this cleanup is confirmed.
ID
Scenario
Input
Expected Output
Pass Criteria
T23
Full production run: overdue milestone traced from Asana trigger to Slack escalation
A single Asana task in the internal test project with a due date 4 days prior, status not complete, assignee set to the internal test email address; production credentials active
Task detected at 8:00 am trigger; written to live tracking sheet with escalation flag TRUE; personalised Gmail reminder sent to internal test address; Slack escalation posted to the designated test channel with correct task details and Asana link
All four steps execute in sequence within 5 minutes; live sheet contains the test row with correct field values; Gmail received at internal test address; Slack message visible in the designated test channel; all field values match the Asana source task exactly
T24
Production Monday report: Notion page created and Slack digest posted
Live tracking sheet contains the test row from T23 plus at least 3 additional historical rows added manually for realism; Monday 7:30 am trigger fires (or is manually triggered at the agreed test time)
Report Builder Agent reads the live sheet and creates a formatted Notion status report page in the production Notion workspace under the designated test section; Slack digest posted to the leadership channel sandbox or a designated internal channel
Notion page created and accessible via the link; page contains all rows from the sheet grouped by project and status; Slack digest counts match the sheet data; Notion page link in the Slack message resolves correctly
T25
Error log verification: confirm all execution events are recorded
Review of the orchestration layer execution log following T23 and T24 runs
Execution log contains a timestamped entry for every step across both runs: Asana read, Sheets write, Gmail send, Slack post, Notion page create, Slack digest post; no steps show an unhandled error status
Every step has a corresponding log entry with a timestamp, status of success, and the relevant output ID or confirmation value; zero entries with an unhandled exception status; log is accessible and exportable for FullSpec records
T26
Process owner review and approval: final production sign-off
Process owner (Operations Manager) reviews the live tracking sheet, the Notion report page, the Slack escalation message, and the Slack digest produced during T23 and T24; FullSpec team presents the execution log summary
Process owner confirms that the tracking sheet data is accurate, the Notion report is correctly formatted and complete, the Gmail reminder received is appropriately worded and personalised, and the Slack messages match the expected format and routing
Process owner verbally or in writing confirms all outputs are acceptable; FullSpec team records the approval confirmation in the FullSpec project record with the process owner's name, role, and the date of approval; no separate sign-off block is required; this test case constitutes the formal process owner approval
Once T26 is confirmed passing and the FullSpec approval record is logged, the automation is cleared for ongoing production operation. Any subsequent configuration changes to escalation thresholds, Asana project scope, or Slack channel routing must be re-tested against the affected test cases before being deployed to production. Contact the FullSpec team at support@gofullspec.com for post-launch change requests.
Test and QA PlanPage 4 of 4