Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

Software Licence Management Automation

[YourCompany.com] · IT Department · Prepared by FullSpec · [Today's Date]

This document defines the complete testing programme for the Software Licence Management automation, covering both the Licence Discovery Agent and the Renewal Coordination Agent. It is written for the FullSpec engineering team and provides every test case, environment rule, pass criterion, and edge-case scenario needed to certify the build before go-live. Testing runs in three sequential phases: unit, integration, and end-to-end production. No phase begins until the previous phase has fully passed. The process owner (IT Manager) joins the team for Phase 3 only.

01Testing approach

All testing follows a strict three-phase sequence. Each phase must reach a full pass state before the next phase begins. This prevents integration and production tests from masking unit-level defects and ensures every agent is certified independently before handoffs are validated end to end.

Phase 1: Unit Testing

Estimated duration: 2 to 3 business days

Scope

Each agent tested in isolation: Licence Discovery Agent (Microsoft 365 sync, Notion comparison, mismatch detection, renewal flag logic) and Renewal Coordination Agent (Slack alert dispatch, response capture, Notion write-back, Xero finance note creation, digest delivery).

Environment

Sandbox only. No production data, no live API write access.

Credential rule

Sandbox credentials exclusively. Microsoft 365 developer tenant, Notion test workspace, Slack test channel, Xero demo company. No production tokens may be loaded.

Pass condition

All cases T01 through T18 reach Expected Output with zero critical failures. Minor cosmetic deviations logged and triaged before Phase 2 begins.

Who runs it

FullSpec team only.

Phase 2: Integration and Edge Case Testing

Estimated duration: 2 to 3 business days

Scope

Agent-to-agent handoffs, tool chain integrity (Microsoft 365 to Notion to Slack to Xero), and all edge cases including duplicate records, missing data fields, unavailable tools, and human non-response timeouts.

Environment

Sandbox only. Simulated payloads used to trigger edge conditions not reproducible with live data.

Credential rule

Sandbox credentials only. No production tokens introduced at this phase.

Pass condition

Integration cases T19 through T26 and edge cases E01 through E08 all pass. Every failure path routes to the correct fallback or alert without silent data loss.

Who runs it

FullSpec team only.

Phase 3: End-to-End Production Testing

Estimated duration: 1 to 2 business days

Scope

Full production run using live credentials across Microsoft 365, Notion, Slack, and Xero. One complete workflow cycle traced from trigger to Slack digest and Xero note. Error log verification and process owner sign-off.

Environment

Production. Live credentials active. Internal test licence record used; voided immediately after the run.

Credential rule

Production credentials loaded. Least-privilege scopes confirmed before run begins. Rollback procedure confirmed active.

Pass condition

Cases T27 through T30 all pass. Process owner reviews output and records approval in the FullSpec confirmation. No unresolved critical or high-severity issues remain open.

Who runs it

FullSpec team executes; process owner (IT Manager) joins for review and sign-off at T30.

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

Sandbox credentials only. No production Microsoft 365 tenant, Notion workspace, Slack workspace, or Xero organisation may be connected during Phase 1. All data used in these test cases must be synthetic. Any case that inadvertently touches a production system is an automatic Phase 1 failure and requires a credential audit before testing resumes.

Licence Discovery Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T01

Scheduled trigger fires at configured time

Cron expression fires at 07:00 in sandbox scheduler

Workflow execution begins; Licence Discovery Agent initialises and calls Microsoft 365 user list endpoint

Execution log shows trigger timestamp within 60 seconds of 07:00; no timeout error

T02

Microsoft 365 user list retrieved successfully

Sandbox M365 tenant with 10 synthetic users (8 active, 2 disabled)

Agent returns structured list of 10 users with status (active/disabled) and assigned licence SKUs

Response contains all 10 user records; status field populated for each; no API error code returned

T03

Notion licence register read successfully

Sandbox Notion database with 12 licence records including renewal dates and seat counts

Agent retrieves all 12 records with correct field mappings: tool name, renewal date, seat count, owner Slack handle

12 records returned; all four mapped fields present and correctly typed; no null fields on mandatory columns

T04

Mismatch detected: disabled user still holds active licence

Two disabled M365 users each assigned a licence SKU that appears in the Notion register as active

Agent flags both records as anomalies with status 'Inactive user, licence active' and writes flags to Notion

Both flagged records appear in Notion with correct anomaly status; no other records altered

T05

Renewal within 60-day window flagged correctly

Notion register contains three licences: one expiring in 15 days, one in 55 days, one in 90 days

Agent flags the 15-day and 55-day records as upcoming renewals; 90-day record untouched

Exactly two records receive renewal flag; 90-day record status unchanged; flag includes days-remaining value

T06

No anomalies or renewals: agent exits cleanly

All M365 users active, no licences expiring within 60 days

Agent writes no flags to Notion; logs a clean-run confirmation

Notion database unchanged; execution log contains 'no anomalies detected' entry; no downstream agents triggered

T07

Microsoft 365 API returns 401 Unauthorised

Sandbox credential deliberately invalidated before run

Agent catches auth error, logs it with error code and timestamp, and sends an alert to the configured IT Manager Slack handle

No Notion write occurs; Slack alert delivered within 2 minutes; error log entry contains HTTP 401 and endpoint path

T08

Notion API returns 503 Service Unavailable

Sandbox Notion API mocked to return 503 on first two calls, then 200

Agent retries up to three times with exponential back-off; succeeds on third attempt; logs retry count

Execution succeeds; retry count of 2 recorded in log; no duplicate Notion writes created

T09

Licence register contains a record with no renewal date

One Notion record has renewal_date field empty

Agent skips renewal-window check for that record, logs a data-quality warning, and continues processing remaining records

Warning logged referencing the specific record ID; all other records processed normally; no workflow halt

Renewal Coordination Agent — Unit Tests

Scenario

Input

Expected Output

Pass Criteria

T10

Agent triggered by Notion flag written by Licence Discovery Agent

Notion record updated with renewal flag by upstream agent

Renewal Coordination Agent initialises, reads the flagged record, and begins Slack outreach flow

Agent execution starts within 30 seconds of Notion write; correct record ID referenced in execution log

T11

Slack renewal alert sent to correct tool owner

Flagged record contains owner_slack_handle = '@testowner'; renewal date = 30 days from today

Structured Slack message delivered to @testowner in sandbox channel including tool name, renewal date, seat count, and three-option prompt (renew/reduce/cancel)

Message received in sandbox Slack channel; all four data fields present; interactive prompt rendered correctly

T12

Owner responds 'renew' via Slack; decision captured and written to Notion

Slack interactive response payload: action = 'renew'

Decision written to Notion record as 'Renew confirmed'; next_review_date set to renewal_date plus 365 days; record status updated to 'Actioned'

Notion record shows correct decision, updated review date, and 'Actioned' status; no duplicate writes

T13

Owner responds 'reduce seats' via Slack

Slack interactive response payload: action = 'reduce', new_seat_count = 3

Notion record updated with new seat count of 3 and decision 'Reduce to 3 seats'; Xero finance note created referencing the change

Notion seat_count field = 3; Xero note body contains tool name, old seat count, new seat count, and effective date

T14

Owner responds 'cancel' via Slack

Slack interactive response payload: action = 'cancel'

Notion record status set to 'Cancel scheduled'; Xero finance note created with cancellation instruction; IT Manager receives Slack notification of pending cancellation

Notion status = 'Cancel scheduled'; Xero note created; IT Manager Slack DM received within 2 minutes

T15

Xero finance note created successfully

Renewal decision requiring a finance update passed to Xero integration step

Note posted to the correct Xero contact/bill record with structured body: tool name, decision, seat delta, effective date

Xero sandbox shows note on target record; all four body fields present; no API error

T16

Daily digest sent to IT Manager via Slack

Three actioned renewals, one unresponded alert, one pending mismatch in sandbox state

Slack digest message delivered to IT Manager handle listing: 3 actioned, 1 unresponded (with tool name and days remaining), 1 mismatch pending review

Digest received in sandbox channel; counts match seeded state; unresponded item includes days-remaining figure

T17

Slack API rate limit hit during alert dispatch

Sandbox Slack mock returns HTTP 429 on first call with Retry-After header of 5 seconds

Agent waits 5 seconds, retries, succeeds on second attempt; no alert dropped; log records rate-limit event

Alert delivered; retry recorded in log; execution time extended by at least 5 seconds; no duplicate message sent

T18

Xero API returns 400 Bad Request on note creation

Malformed contact ID passed to Xero note endpoint

Agent logs error with HTTP 400 and payload details; sends fallback Slack alert to IT Manager indicating finance note failed and manual follow-up required

No Xero note created; error log contains 400 and payload; IT Manager Slack alert received; workflow does not halt

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration Tests — Agent Handoffs and Tool Chain

Scenario

Input

Expected Output

Pass Criteria

T19

Full handoff: Licence Discovery Agent flags anomaly, Renewal Coordination Agent picks it up

Sandbox run with one disabled M365 user holding an active licence

Discovery Agent writes anomaly flag to Notion; Renewal Coordination Agent triggers within 30 seconds; Slack alert dispatched to tool owner

Both agents appear in execution log in correct sequence; Notion flag and Slack alert timestamps confirm handoff lag under 30 seconds

T20

Full handoff: renewal within 60 days triggers end-to-end coordination flow

Sandbox Notion record with renewal date 45 days away and valid owner_slack_handle

Discovery Agent flags renewal; Renewal Coordination Agent sends Slack alert; simulated 'renew' response captured; Notion updated; Xero note created; digest delivered

All five downstream actions complete in sequence; no step skipped; Notion and Xero sandbox records reflect correct final state

T21

Multiple simultaneous flags processed without cross-contamination

Three flagged records written to Notion simultaneously by Discovery Agent (two renewals, one mismatch)

Renewal Coordination Agent opens three separate execution threads; each resolves independently without writing to the wrong Notion record

Three Notion records updated correctly; each Slack alert references the correct tool and owner; no field values swapped between records

T22

Xero note correctly references the Notion record that triggered it

Renewal decision for 'Tool-A' with Notion record ID 'abc-123'

Xero note body includes Notion record reference alongside tool name and decision details

Xero sandbox note contains 'abc-123' or equivalent reference; finance team can trace note back to Notion source record

T23

Daily digest aggregates all same-day activity correctly

Five events processed in one sandbox day: two renewals actioned, one cancel, one unresponded, one mismatch pending

Single digest Slack message to IT Manager listing correct counts and status for all five events

One digest message sent (not five separate); all five events accounted for with correct status labels

T24

Discovery Agent run overlaps with in-progress Coordination Agent execution

Discovery Agent scheduled run fires while Coordination Agent is mid-execution on a previous flag

Discovery Agent completes its read and write cycle without corrupting the record currently being processed by the Coordination Agent

No data overwrites observed; both agents complete successfully; execution logs show no locking conflicts or duplicate flag writes

T25

IT Manager exception review path triggered when no anomaly flag is set

Decision node evaluates to 'no anomalies'; workflow routes to IT Manager exception review node

IT Manager exception node executes; no Slack renewal alerts dispatched; digest still delivered with 'no anomalies' summary

Execution log shows correct branch taken; zero renewal alerts sent; digest delivered with clean-run message

T26

Coordination Agent completes with no owner response before digest fires

Owner alert sent; no Slack response received within the configured response window before digest time

Digest includes the unresponded alert with tool name and days remaining; Notion record status remains 'Awaiting response'

Digest references unresponded item; Notion status unchanged from 'Awaiting response'; no auto-decision applied

Edge Case Tests

Scenario

Input

Expected Output

Pass Criteria

E01

Duplicate Notion flag written for the same licence record

Discovery Agent writes a renewal flag twice for the same record ID within one run (simulated retry scenario)

Coordination Agent deduplicates on record ID; only one Slack alert sent to owner; Notion record updated once

Single Slack message received by owner; Notion record shows one update timestamp; execution log records deduplication event

E02

Notion record missing the owner_slack_handle field

Flagged renewal record has owner_slack_handle = null

Agent cannot send Slack alert; logs a data-quality error referencing the record ID; sends fallback alert to IT Manager Slack handle with record details for manual follow-up

No alert sent to null handle; IT Manager fallback alert received; Notion record updated with 'Owner handle missing, manual follow-up required' note

E03

Microsoft 365 returns an empty user list

M365 sandbox mock returns an empty array for the user list endpoint

Agent detects empty response, logs a data-quality warning, does not clear existing Notion flags, and alerts IT Manager that user sync returned zero records

Existing Notion flags preserved; IT Manager alert received; execution log contains 'empty user list' warning; no downstream agent triggered

E04

Notion API completely unavailable during Discovery Agent run

Notion sandbox mock returns 503 on all three retry attempts

Agent exhausts retries, logs a critical error, sends an IT Manager Slack alert, and halts the current run without writing partial data

No partial Notion writes; IT Manager alert received with error code and retry count; execution log marks run as failed; next scheduled run proceeds normally

E05

Tool owner responds to Slack alert after the response window has closed

Late Slack response payload arrives after the digest has already been sent and Notion status set to 'Awaiting response'

Agent accepts the late response, updates the Notion record with the decision and a 'Late response' tag, and logs the event without re-triggering the digest

Notion record updated with decision and 'Late response' tag; no second digest sent; Xero note created if the decision requires one

E06

Renewal date field contains an invalid date string

Notion record has renewal_date = 'TBC'

Agent logs a data-quality warning for that record, skips the renewal-window calculation, and continues processing all other records

Warning logged with record ID; no renewal flag set on that record; all other records processed without interruption

E07

Slack interactive message payload arrives malformed

Response webhook delivers a payload with missing 'action' key

Agent logs a parse error, does not write a decision to Notion, and sends IT Manager a Slack alert requesting manual confirmation of the owner's intent

Notion record unchanged; IT Manager alert received referencing the affected tool and owner; error log contains parse failure details

E08

Human IT Manager timeout: exception item not reviewed within 24 hours

Exception item routed to IT Manager review node; no action taken for 24 hours

Automation sends a reminder Slack message to IT Manager at the 24-hour mark; Notion record status updated to 'Escalated, pending review'

Reminder message delivered at 24 hours; Notion status = 'Escalated, pending review'; no further automated action taken until IT Manager responds

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: Use an internal test licence record in the live Notion workspace (label it clearly as 'FULLSPEC TEST — DO NOT ACTION'). Use a dedicated internal Slack channel for test alerts rather than the live IT Manager channel. Void or delete the test Notion record and any associated Xero note immediately after the run is complete and sign-off is confirmed. Confirm that the rollback procedure documented in the Developer Handover Pack is active before loading production credentials. Do not run Phase 3 during a period when real renewals are due within 5 days.

Scenario

Input

Expected Output

Pass Criteria

T27

Full production trigger to digest: one renewal flagged and actioned

Production Microsoft 365 tenant; Notion workspace with a seeded test licence record expiring in 45 days; test owner_slack_handle pointing to internal test channel

Licence Discovery Agent runs at 07:00, detects the test renewal, flags the Notion record, triggers Renewal Coordination Agent, which sends Slack alert to test channel; simulated 'renew' response captured; Notion record updated; Xero note created in demo/sandbox Xero company; IT Manager digest delivered to test Slack channel

All six steps logged in production execution log with correct timestamps; Notion record shows 'Renew confirmed' and updated review date; Xero note body contains correct tool name and decision; digest message counts match seeded state; end-to-end latency from trigger to digest under 10 minutes

T28

Production mismatch detection: disabled M365 user with active licence

One real (or seeded test) M365 account set to disabled status, with a corresponding Notion record showing an active licence

Discovery Agent flags the mismatch, writes anomaly to Notion, Coordination Agent sends alert to test Slack channel, IT Manager receives mismatch notification in digest

Notion anomaly flag created with correct status 'Inactive user, licence active'; test Slack channel receives alert; digest references the mismatch; no production Notion data outside the test record is altered

T29

Error log verification: confirm all execution events are written to the audit log

Full production run from T27 and T28 completed

Execution audit log contains a complete chronological record of: trigger timestamp, M365 API call result, Notion read and write events, Slack dispatch events, Xero note creation event, digest dispatch event, and any retries or warnings

Audit log entries present for all eight event types listed; no gaps in the chronological sequence; log is accessible to the FullSpec team and exportable in JSON or CSV format

T30

Process owner review and approval: IT Manager confirms production output is correct

IT Manager (process owner) reviews the Notion test record state, the test Slack channel messages, and the execution audit log following T27 to T29

IT Manager confirms that: the correct licence record was flagged, alerts were correctly formatted and routed, Notion updates are accurate, the Xero note contains the right information, the digest is clear and actionable. Approval is recorded in the FullSpec confirmation log.

IT Manager approval recorded in the FullSpec confirmation log. All Phase 3 test records voided in Notion and Xero immediately after sign-off. No unresolved critical or high-severity issues remain open. Phase 3 is declared complete and the build is cleared for go-live.

After T30 is complete, the FullSpec team will void the test Notion record, delete the associated Xero note, archive the test Slack channel messages, and confirm deletion in the project log. Go-live is authorised only after all three phases show a full pass status and the FullSpec confirmation log has been updated. Any open issue rated critical or high must be resolved and retested before go-live is approved. Issues rated medium or low may be scheduled for a post-launch patch at the FullSpec team's discretion in agreement with the process owner. For questions at any point, contact the FullSpec team at support@gofullspec.com.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process