FS-DOC-06Quality
Test and QA Plan
Interview Scheduling Automation
[YourCompany.com] · HR Department · Prepared by FullSpec · [Today's Date]
This document defines the full testing programme for the Interview Scheduling automation, covering all three agents: the Availability Matching Agent, the Scheduling and Comms Agent, and the ATS Update Agent. Testing runs in three sequential phases, each gated on the prior phase passing in full. The FullSpec team owns all build and test execution through Phases 1 and 2. Your process owner joins Phase 3 to verify production behaviour and record formal approval before go-live.
01Testing approach
Testing is structured as three phases executed in sequence. No phase begins until every test case in the previous phase has reached a pass status. Phase 1 isolates each agent in a sandbox environment. Phase 2 validates agent-to-agent handoffs and edge conditions, also in sandbox. Phase 3 executes a full end-to-end run in the production environment with real credentials and a controlled test candidate record. The FullSpec team runs all phases; your process owner participates in Phase 3 to review outputs and record sign-off.
1Phase 1: Unit Testing
Each agent tested in isolation
Scope
Individual agent logic: trigger detection, data reads, output generation, and tool calls for each of the three agents tested separately.
Environment
Sandbox only. Greenhouse sandbox tenant, Google Calendar test accounts, Calendly test organisation, Gmail test address, Slack test workspace, Notion test database.
Credential rule
No production credentials used in Phase 1. All OAuth tokens and API keys must point to sandbox or test environments. Confirmed before each test run.
Pass condition
All test cases T01 through T12 must return a pass status before Phase 2 begins.
Who runs it
FullSpec team only.
2Phase 2: Integration and Edge Case Testing
Agent handoffs and failure scenarios
Scope
End-to-end data flow across agent boundaries: Availability Matching Agent to Scheduling and Comms Agent to ATS Update Agent. Plus structured edge cases covering missing data, tool unavailability, duplicate events, and human timeout scenarios.
Environment
Sandbox only. Same test-environment credentials as Phase 1. No production data or live candidate records.
Credential rule
No production credentials. All webhook endpoints must resolve to sandbox receivers. Slack notifications route to a dedicated test channel only.
Pass condition
All integration test cases T13 through T19 pass, and all edge cases E01 through E06 produce the expected exception handling behaviour or controlled failure state.
Who runs it
FullSpec team only.
3Phase 3: End-to-End Production Test
Live environment, process owner present
Scope
Full workflow run in production: real Greenhouse webhook, live Google Calendar and Calendly accounts, real Gmail send, real Slack channel, and live Greenhouse write-back. A controlled internal test candidate record is used and voided after each run.
Environment
Production. All credentials live. Test candidate email is an internal address controlled by your team.
Credential rule
Production credentials active. FullSpec confirms all scopes and permissions are correctly scoped before Phase 3 begins. No broad admin tokens used where scoped tokens suffice.
Pass condition
All production test cases T20 through T23 pass. Final case T23 records process owner approval in the FullSpec confirmation log.
Who runs it
FullSpec team executes; process owner reviews outputs and records approval for T23.
Test and QA PlanPage 1 of 4
FS-DOC-06Quality
02Phase 1 — unit test cases
Sandbox credentials only. Before running any Phase 1 test case, confirm that every connected tool (Greenhouse, Google Calendar, Calendly, Gmail, Slack, Notion) is pointing to a sandbox or test-environment account. Do not use production API tokens, OAuth grants, or live candidate data at any point in Phase 1. Any test that inadvertently reaches a production endpoint must be halted, logged, and restarted under correct credentials.
Availability Matching Agent: reads the Notion panel configuration, queries Google Calendar, and returns a ranked slot list. Test cases T01 to T04 cover this agent.
ID
Scenario
Input
Expected output
Pass criteria
T01
Happy path: two interviewers with overlapping availability
Greenhouse stage-change webhook; Notion panel doc with 2 interviewers; both Google Calendars show 3 common open windows in the next 5 business days
Ranked list of 3 overlapping slots, each meeting the configured interview duration (60 min default); Calendly scoped link generated
Slot list contains exactly 3 entries; each slot is free on both calendars; Calendly link is valid and scoped to those windows; execution time under 8 seconds
T02
Three interviewers, partial overlap only
Notion panel doc with 3 interviewers; one calendar is mostly blocked; 1 common window exists in the next 5 days
Single slot returned; Calendly link scoped to that slot only
List contains 1 slot; slot is confirmed free on all 3 calendars; agent does not fabricate additional slots
T03
Failure: no overlapping availability found
All 3 interviewer calendars fully blocked for the next 5 business days
Agent returns a no-availability exception; workflow routes to recruiter exception flag; no Calendly link generated
Exception event is logged with reason 'no_overlap_found'; Slack alert sent to recruiter test channel; no booking link emitted
T04
Failure: Notion panel doc missing or misconfigured
Greenhouse webhook fires; Notion database returns a 404 or empty record for the role
Agent logs a configuration error; workflow halts with an error state; no calendar query attempted
Error state captured with reason 'notion_panel_not_found'; no downstream action triggered; error visible in workflow run log
Scheduling and Comms Agent: sends the Calendly link, monitors for booking, creates the calendar event, sends Slack briefings, and queues the day-before reminder. Test cases T05 to T09 cover this agent.
ID
Scenario
Input
Expected output
Pass criteria
T05
Happy path: candidate books the first available slot
Valid scoped Calendly link; test candidate email address; Calendly booking confirmed event fires
Gmail sends Calendly link to candidate; on booking confirmation: Google Calendar event created with all participants; Slack message sent to each interviewer test account; day-before Gmail reminder queued 20 hours before event
All 4 downstream actions complete; Calendar event ID returned; Slack messages delivered to correct test channels; reminder scheduled at correct timestamp
T06
Candidate books then reschedules via Calendly
Initial booking confirmed; Calendly reschedule event fires with new slot within permitted windows
Calendar event updated to new time; updated Slack notification sent to interviewers; reminder rescheduled to new timestamp; original reminder cancelled
Old calendar event is modified (not duplicated); Slack message includes updated time; reminder reflects new event time; no orphan reminders remain
T07
Failure: Gmail send blocked by rate limit or auth error
Valid scoped Calendly link; Gmail API returns 429 or 401
Automation retries with exponential back-off (3 attempts, 30/60/120 second intervals); if all retries fail, error logged and recruiter alert fired
Retry logic executes correctly; final failure triggers recruiter Slack alert in test channel; no silent failure
T08
Failure: Calendly webhook not received within 48 hours
Calendly link sent; no booking event received within 48-hour timeout window
Workflow branches to no-booking exception; recruiter receives a Slack alert in test channel flagging the candidate name and role
Timeout fires at correct interval; Slack alert content includes candidate identifier and original slot list; workflow does not hang indefinitely
T09
Failure: Google Calendar event creation fails
Calendly booking confirmed; Google Calendar API returns 403 (insufficient scope) or 500
Error logged; recruiter Slack alert sent; Greenhouse record not updated; candidate receives no calendar invite
Error captured with HTTP status code; no partial calendar event created; downstream ATS update agent not triggered
ATS Update Agent: writes confirmed interview date, time, and panel details back to the Greenhouse candidate record. Test cases T10 to T12 cover this agent.
ID
Scenario
Input
Expected output
Pass criteria
T10
Happy path: successful Greenhouse write-back on first booking
Calendly booking confirmed; candidate Greenhouse ID; confirmed date, time, and interviewer list
Greenhouse candidate record updated with interview date, time, and assigned interviewers within 60 seconds of booking
Greenhouse sandbox record reflects correct date and time; all interviewer names written; no prior data overwritten; API response 200
T11
Write-back after a reschedule
Calendly reschedule confirmed; updated date and time for same candidate record
Greenhouse record updated to new date and time; previous scheduling note replaced, not duplicated
Record shows new date and time only; no duplicate entries; Greenhouse API confirms single update
T12
Failure: Greenhouse API rejects write-back (auth or plan restriction)
Calendly booking confirmed; Greenhouse API returns 401 (token invalid) or 403 (plan does not allow write-back)
Error logged with HTTP status; recruiter Slack alert sent noting ATS update failed; calendar event and notifications already sent remain intact
Failure does not roll back calendar event or Slack messages; error log entry contains candidate ID, HTTP code, and timestamp; recruiter alert received in test channel
Test and QA PlanPage 2 of 4
FS-DOC-06Quality
03Phase 2 — integration and edge case tests
Phase 2 validates data flow across agent boundaries and confirms that the full chain from Greenhouse trigger to ATS write-back behaves correctly when agents hand off to one another. All tests remain in the sandbox environment. IDs continue from Phase 1.
ID
Scenario
Agents involved
Expected output
Pass criteria
T13
Full happy-path handoff: Availability Matching Agent to Scheduling and Comms Agent
Availability Matching Agent, Scheduling and Comms Agent
Slot list output from Agent 1 is correctly consumed by Agent 2 without transformation errors; Calendly link scoped to the correct windows is sent to the test candidate
Slot payload schema matches the expected input schema of Agent 2; no data-type mismatches; Calendly link resolves to correct available windows only
T14
Full happy-path handoff: Scheduling and Comms Agent to ATS Update Agent
Scheduling and Comms Agent, ATS Update Agent
Calendly booking payload correctly passed to ATS Update Agent; Greenhouse record updated within 60 seconds
Candidate ID, confirmed date, confirmed time, and interviewer list all present in the payload; Greenhouse sandbox record matches exactly
T15
Three-agent full chain in sandbox
All three agents
Greenhouse webhook fires; slot list computed; Calendly link sent; booking simulated; Calendar event created; Slack messages sent; Greenhouse record updated
All 8 automated flow steps complete in sequence; no step skipped; total chain execution time under 3 minutes; all outputs verified against expected values
T16
Handoff with partial slot list (1 slot only)
Availability Matching Agent, Scheduling and Comms Agent
Agent 2 accepts a single-slot payload without error; Calendly link scoped to that one window; candidate email sent correctly
No array-length error; Calendly link valid; email delivered to test address; agent does not request more slots than available
T17
Reschedule chain: all three agents re-triggered
All three agents
Calendly reschedule event fires; Scheduling and Comms Agent updates Calendar event and Slack; ATS Update Agent writes new date to Greenhouse
Calendar event updated (not duplicated); Slack notification includes updated time; Greenhouse record shows new date; old reminder cancelled and new reminder queued
T18
Agent 1 failure: downstream agents do not fire
Availability Matching Agent (forced failure), Scheduling and Comms Agent
When Agent 1 returns an exception, Agent 2 does not receive a trigger; recruiter alert fires; no candidate email sent
Zero trigger events reach Agent 2; recruiter Slack alert received; workflow run log shows halted state at Agent 1
T19
Agent 2 failure mid-chain: ATS Update Agent does not fire
Scheduling and Comms Agent (forced failure after Calendar event created), ATS Update Agent
Calendar event may exist; ATS Update Agent does not fire; error log captures failure point; recruiter alert sent
Greenhouse record not updated; error log identifies step and agent; recruiter alert includes candidate identifier; no silent partial completion
Edge cases: the following table covers scenarios outside the normal booking flow, including duplicate records, missing data, tool outages, and human timeout situations. Edge case IDs use the E prefix.
ID
Scenario
Input condition
Expected behaviour
Pass criteria
E01
Duplicate Greenhouse webhook fires for same candidate
Greenhouse fires the stage-change webhook twice within 10 seconds for the same candidate ID
Automation deduplicates on candidate ID; second trigger is suppressed; only one Calendly link sent to candidate
Single email sent to test address; single Calendly booking page active; no duplicate calendar events created; deduplication log entry present
E02
Notion panel doc exists but interviewer list is empty
Notion database returns a record for the role but the interviewers field is blank
Agent returns a configuration error 'interviewer_list_empty'; workflow halts; recruiter Slack alert sent with role name
No calendar query attempted; no candidate email sent; Slack alert received with actionable error description
E03
Candidate submits booking outside the scoped Calendly windows
Candidate attempts to book a slot not within the permitted windows (simulated via Calendly test bypass)
Calendly enforces window restriction at the booking page level; if a rogue webhook arrives, the automation validates the slot against the original slot list and rejects it
Invalid booking payload rejected; error logged; no Calendar event created for out-of-window slot; recruiter alerted
E04
Google Calendar API unavailable during slot query
Simulate Google Calendar 503 response for all queries in a single run
Agent retries 3 times with back-off; after all retries fail, workflow halts with error 'calendar_unavailable'; recruiter Slack alert sent; no partial slot list returned
Retry attempts logged; final failure state captured; recruiter alert received; no candidate email sent during outage
E05
Slack API unavailable when interviewer briefing fires
Slack API returns 503 at the point of sending interviewer briefing messages
Automation retries Slack send up to 3 times; if all fail, error logged and fallback Gmail notification sent to each interviewer's email address
Fallback Gmail notifications sent to all interviewers in the test panel; Slack failure logged with HTTP code; Calendar event and candidate confirmation unaffected
E06
Human recruiter timeout: no-booking exception not actioned within 72 hours
No-booking exception alert sent to recruiter Slack; recruiter takes no action within 72-hour window
Automation sends a second escalation Slack alert after 72 hours flagging the candidate as 'scheduling overdue'; no automatic cancellation of the candidate record
Escalation alert fires at correct timestamp; alert content includes candidate name, role, and original stage-change date; no automated cancellation action taken
Test and QA PlanPage 3 of 4
FS-DOC-06Quality
04Phase 3 — end-to-end production test
Important before running Phase 3: use an internal test email address (owned and monitored by your team) as the candidate email for all production test runs. Do not use a real candidate's email address. After each production test run, void the test candidate record in Greenhouse immediately, cancel any calendar events created, and delete the Calendly booking. Confirm with the FullSpec team that all test artefacts have been removed before declaring Phase 3 complete. Running Phase 3 with live production credentials means real emails, real calendar invites, and real Greenhouse record changes will occur.
Phase 3 uses live production credentials across all connected tools. The FullSpec team executes each test case and records the output. Your process owner reviews each result in real time and records formal approval as the final test case. IDs continue from Phase 2.
ID
Scenario
Steps traced
Expected output
Pass criteria
T20
Full production run: happy path from Greenhouse stage change to confirmed booking
Greenhouse webhook fires on real tenant; Availability Matching Agent queries live Google Calendars; Calendly link emailed via live Gmail; test candidate books slot; Calendar event created; Slack briefings sent to live interviewer channels; Greenhouse record updated
All 8 automated flow steps complete on production infrastructure; candidate (internal test address) receives correct Calendly link; Calendar event appears on all interviewer calendars; Slack messages arrive in correct channels; Greenhouse record shows confirmed date and time
Each step confirmed in real tool interfaces, not just workflow logs; total time from webhook to Greenhouse update under 5 minutes; no manual intervention required; all outputs match sandbox results from Phase 2
T21
Production reschedule path
After T20 booking: test candidate triggers Calendly reschedule to a different slot within permitted windows; Scheduling and Comms Agent updates Calendar event and sends updated Slack notification; ATS Update Agent writes new date to Greenhouse
Calendar event updated (not duplicated) on all interviewer calendars; Slack messages updated with new time; Greenhouse record reflects rescheduled date; old day-before reminder cancelled and new reminder queued at correct timestamp
No duplicate calendar events exist after reschedule; Greenhouse record updated to new date and time only; Slack messages received by interviewers with correct new time; reminder timestamp verified in workflow scheduler
T22
Production error log verification
FullSpec team reviews the production workflow run log for T20 and T21 runs
Run logs show complete step-by-step execution trace for each run; all HTTP response codes from Greenhouse, Google Calendar, Calendly, Gmail, and Slack APIs are 200 or 201; no unhandled exceptions; retry events absent (confirming clean runs)
Log entries present for every automated step; all API response codes correct; no error states logged; log timestamps consistent with actual delivery times observed in connected tools
T23
Process owner review and approval
Process owner reviews all Phase 3 outputs: candidate emails received, calendar events created, Slack messages delivered, Greenhouse records updated, and error logs reviewed
Process owner confirms all outputs are correct and consistent with the agreed specification; approval is recorded in the FullSpec confirmation log with the process owner's name, role, and date of approval
Process owner sign-off recorded in the FullSpec confirmation log. No separate signature document is required. Approval entry must include: reviewer name, role, date, and a pass/fail statement for each of T20, T21, and T22. Phase 3 is not complete until this entry is confirmed by FullSpec.
After T23 is recorded, the FullSpec team will archive this Test and QA Plan alongside the full test run logs in the project delivery pack. If any Phase 3 test case returns a fail result, the affected test case is logged as a defect, resolved by the FullSpec team, and re-run from T20 before approval is requested again. Contact the FullSpec team at support@gofullspec.com for any queries during the testing process.
Test and QA PlanPage 4 of 4