Test and QA Plan

How the build is validated before go-live: three sequential test phases, edge cases, and production verification.

4 pagesPDF · Quality

FS-DOC-06Quality

Test and QA Plan

New Tool Evaluation and Rollout

[YourCompany.com] · IT Department · Prepared by FullSpec · [Today's Date]

This document defines the complete test and quality assurance strategy for the New Tool Evaluation and Rollout automation. It covers three sequential phases: unit testing of each individual agent in isolation, integration and edge-case testing across agent handoffs, and a final end-to-end production run. The FullSpec team owns all test execution through Phases 1 and 2. Your team joins in Phase 3 to confirm the production run meets operational requirements and to record formal sign-off. No phase begins until every test case in the preceding phase achieves a passing status.

01Testing approach

Testing proceeds through three phases in strict sequence. Phase 1 validates each agent independently against known inputs and expected outputs in a sandboxed environment. Phase 2 confirms that data flows correctly across agent boundaries and that the workflow handles edge cases without silent failure. Phase 3 executes one complete production run with real credentials and verifies outcomes end to end. No phase begins until the previous phase fully passes, with all test case IDs recorded as green.

Phase 1: Unit Testing

Each agent tested in isolation against fixed inputs

Scope

Request Intake Agent, Evaluation and Approval Agent, Procurement and Rollout Agent — each tested independently

Environment

Sandbox only. All tool connections use sandbox or developer credentials. No production data.

Credential rule

Sandbox Jira project, Notion test database, Gmail test account, DocuSign sandbox envelope, Slack test workspace. Production credentials must not be loaded.

Pass condition

All cases T01 through T14 return expected outputs with no unhandled errors

Who runs it

FullSpec team

Phase 2: Integration and Edge Case Testing

Agent handoffs and failure modes validated across the full pipeline

Scope

End-to-end data flow across all three agent handoffs, plus structured edge cases including duplicates, missing fields, tool unavailability, and human timeout

Environment

Sandbox only. Real API calls to sandbox endpoints; no production records created.

Credential rule

Same sandbox credentials as Phase 1. Any changes to credential configuration must be applied to sandbox copies only.

Pass condition

Integration cases T15 through T22 pass; all edge cases E01 through E07 produce the defined fallback behaviour with an error or flag logged

Who runs it

FullSpec team

Phase 3: End-to-End Production Test

Single full run in the live environment with process owner present

Scope

One complete tool evaluation traced from form submission through to Slack rollout notification using production credentials and a pre-agreed test vendor record

Environment

Production. Live Jira project, live Notion software register, live DocuSign account, live Slack workspace.

Credential rule

Production credentials loaded via the secure environment variable store. The FullSpec team confirms credential values before the run begins. No sandbox fallback during this phase.

Pass condition

Cases T23 through T27 pass, including process owner review and approval recorded in the FullSpec confirmation log

Who runs it

FullSpec team executes; process owner (IT Manager) reviews outputs and records approval at T27

Test and QA PlanPage 1 of 4

FS-DOC-06Quality

02Phase 1 — unit test cases

All Phase 1 tests must be run against sandbox credentials only. Sandbox Jira project key: TOOLTEST. Sandbox Notion database ID: notion-sandbox-db-001. DocuSign sandbox account: demo.docusign.net. Do not load production credentials during this phase under any circumstances.

Request Intake Agent

Scenario

Input

Expected Output

Pass Criteria

T01

Happy path: new unique tool request

Valid form submission: tool name 'Figma', requestor 'Jane Smith', 5 users, $50/month budget, no matching entry in Notion register

Jira ticket created (status: Open), Notion register queried with no duplicate flag, Slack acknowledgement sent to Jane Smith

Jira ticket ID returned; duplicate_flag field = false; Slack message delivered within 30 seconds of trigger

T02

Duplicate tool detected in Notion register

Form submission: tool name 'Zoom', requestor 'Lee Park', existing Notion entry for Zoom with status Active

Jira ticket created with duplicate_flag = true; Slack message to requestor notes existing tool; no evaluation scorecard created

duplicate_flag = true on Jira ticket; Slack message body contains the word 'duplicate' or 'existing'; no Notion scorecard page created

T03

Form submission with missing required field (use case blank)

Partial form: tool name 'Loom', requestor 'Sam Ray', use_case field empty

Automation halts before Jira ticket creation; error logged to the orchestration layer; Slack error notification sent to IT Manager

No Jira ticket created; error event visible in run log; Slack error message delivered to IT Manager channel

T04

Jira API unavailable at trigger time

Valid form submission; Jira sandbox endpoint returns 503

Retry logic executes up to 3 times at 60-second intervals; after 3 failures, error logged and IT Manager notified via Slack

Exactly 3 retry attempts recorded in run log; Slack alert sent after third failure; no silent discard of the submission

T05

Notion API unavailable during duplicate check

Valid form submission; Jira ticket creates successfully; Notion API returns 503 on register query

Jira ticket created; duplicate check marked as 'check_failed'; Jira ticket comment added noting register could not be queried; IT Manager alerted

Jira ticket exists with check_failed flag; ticket comment present; Slack alert to IT Manager delivered

Evaluation and Approval Agent

Scenario

Input

Expected Output

Pass Criteria

T06

Happy path: Jira ticket moves to Evaluation In Progress

Jira ticket status updated to 'Evaluation In Progress' for ticket TOOLTEST-001

Security checklist page created in Notion and assigned to IT Manager; scorecard page created in Notion pre-populated with request fields

Both Notion pages exist with correct ticket reference; IT Manager assigned on checklist; scorecard fields populated from ticket data

T07

Approval request email sent on scorecard completion

Notion scorecard for TOOLTEST-001 marked Complete by IT Manager

Approval request email sent from IT Manager Gmail sandbox account to manager's sandbox address with evaluation summary attached

Email delivered to sandbox recipient within 60 seconds of scorecard status change; subject line contains ticket ID; attachment present

T08

48-hour reminder triggered when no approval response

Approval email sent; no response recorded after 48-hour wait (simulated in sandbox via time-offset trigger)

Follow-up reminder email sent to same approver; reminder event logged on Jira ticket

Second email delivered to sandbox recipient; Jira ticket comment timestamped at +48 hours from initial send

T09

Approval decision: rejected

Approver replies with rejection via sandbox email response or manual Jira status update to 'Rejected'

Jira ticket status set to Rejected; requestor notified via Slack; no DocuSign envelope triggered

Jira ticket status = Rejected; Slack message delivered to requestor; no DocuSign API call made

T10

Google Workspace API unavailable when sending approval email

Scorecard marked Complete; Gmail sandbox API returns 503

Retry up to 3 times; after failure, error logged on Jira ticket and IT Manager alerted via Slack

3 retry attempts in run log; Jira ticket comment with error detail; Slack alert sent

Procurement and Rollout Agent

Scenario

Input

Expected Output

Pass Criteria

T11

Happy path: approval recorded, DocuSign envelope sent

Jira ticket status set to 'Approved' for TOOLTEST-001

DocuSign sandbox envelope sent to pre-configured signatory; envelope ID logged on Jira ticket

DocuSign envelope status = sent within sandbox; envelope ID present on Jira ticket; no manual step required

T12

Notion software register updated on agreement signing

DocuSign sandbox envelope status changes to 'completed'

Notion software register entry created with tool name, licence count, renewal date, owner, and monthly cost from ticket fields

Notion register page exists; all five required fields populated; entry linked to Jira ticket ID

T13

Slack rollout notification sent after register update

Notion register entry confirmed as created

Slack message sent to requestor and named users confirming tool approval and linking to onboarding materials

Slack message delivered to correct channel or DM; message body contains tool name and onboarding link placeholder

T14

DocuSign API unavailable when envelope should be sent

Jira ticket status = Approved; DocuSign sandbox API returns 503

Retry up to 3 times; after failure, Jira ticket comment added with error detail; IT Manager alerted via Slack; Notion register not updated prematurely

3 retry attempts logged; Jira comment present; Slack alert sent; Notion register unchanged until envelope confirmed sent

Test and QA PlanPage 2 of 4

FS-DOC-06Quality

03Phase 2 — integration and edge case tests

Integration tests confirm that data passes correctly between agents at each handoff point. All tests in this phase use sandbox credentials and build on the Jira ticket and Notion records created during Phase 1 testing. Each case below must pass before Phase 3 begins.

Scenario

Input

Expected Output

Pass Criteria

T15

Handoff 1: Intake Agent to Evaluation Agent — ticket moves to Evaluation In Progress

Jira ticket created by Intake Agent (T01 output); status manually advanced to 'Evaluation In Progress' to simulate workflow trigger

Evaluation Agent picks up the correct ticket; Notion checklist and scorecard created with fields sourced from the Jira ticket created in T01

Notion page titles contain TOOLTEST-001; requestor name and tool name from T01 appear in scorecard fields; no orphaned pages

T16

Handoff 1: duplicate-flagged ticket does not trigger evaluation pipeline

Jira ticket with duplicate_flag = true (T02 output); status advanced to 'Evaluation In Progress'

Evaluation Agent skips checklist and scorecard creation; Jira ticket comment notes evaluation blocked by duplicate flag

No new Notion pages created; Jira comment present; no approval email sent

T17

Handoff 2: Evaluation Agent to Procurement Agent — approval recorded

Jira ticket from T07 with approval email sent; approver responds to sandbox email confirming approval; Jira status set to 'Approved'

Procurement Agent triggers DocuSign envelope within 2 minutes of status change; envelope ID written back to Jira ticket

DocuSign sandbox envelope status = sent; envelope ID on Jira ticket; elapsed time from status change to envelope send under 2 minutes

T18

Handoff 2: rejection path — no DocuSign triggered, Slack notification sent

Jira ticket status set to 'Rejected'

Procurement Agent sends Slack rejection notification to requestor; Jira ticket status remains Rejected; no DocuSign call made

Slack message body contains 'not approved' or equivalent; no DocuSign API call in run log; Jira status unchanged

T19

Handoff 3: DocuSign completion to Notion register update

DocuSign sandbox envelope for TOOLTEST-001 set to 'completed' via sandbox UI

Notion software register entry created automatically within 3 minutes; Slack rollout message sent immediately after

Notion entry present with all required fields; Slack message sent; elapsed time under 3 minutes

T20

Full sequential run: all three agents in order with no manual intervention

Clean form submission in sandbox with unique tool name 'TestTool-QA'

Jira ticket created, evaluation triggered, approval email sent (sandbox), approval recorded, DocuSign sent and completed (sandbox), Notion register updated, Slack notification sent

All seven automated steps produce expected outputs in sequence with no manual prompts; total elapsed time under 10 minutes in sandbox

T21

Reminder escalation: second reminder sent after 96 hours with no approval

Approval email sent at T=0; no response at T+48h (first reminder via T08); no response at T+96h

Second reminder sent at T+96h; Jira ticket comment updated; IT Manager receives Slack escalation alert

Second reminder email delivered at simulated T+96h mark; Jira comment timestamped accordingly; Slack escalation alert sent to IT Manager

T22

Data fidelity: field values consistent across Jira, Notion, DocuSign, and Slack

Form submission with known field values: tool='SandboxApp', users=8, budget=$120/month, requestor='Test User'

All downstream records (Jira ticket, Notion scorecard, Notion register, DocuSign envelope recipient field, Slack message) reflect exactly the submitted field values

Field-by-field comparison across all four systems confirms no truncation, casing errors, or field mismatches

Edge case tests cover inputs and conditions outside the happy path that the automation must handle gracefully without silent failure or data loss.

Scenario

Input

Expected Output

Pass Criteria

E01

Near-duplicate: tool name variation (e.g. 'MS Teams' vs 'Microsoft Teams')

Form submission with tool name 'MS Teams'; Notion register contains entry 'Microsoft Teams'

Duplicate check returns a fuzzy match flag; Jira ticket comment lists the potential match and prompts IT Manager to confirm; evaluation does not proceed until confirmation

Jira ticket has fuzzy_match_flag = true; comment lists matched Notion entry name; no scorecard created until IT Manager confirms

E02

Submission with all optional fields blank

Form: tool name 'Miro', requestor 'Alex Chen', all optional fields (integration needs, notes, trial link) left blank

Jira ticket created; optional fields recorded as null; no error thrown; Slack acknowledgement sent normally

Jira ticket created with null values for optional fields; no validation error in run log; Slack message delivered

E03

Notion database at row limit or read-only during duplicate check

Notion sandbox API returns 403 Forbidden on register query

Error logged; duplicate check marked as 'check_failed'; IT Manager alerted; evaluation pipeline paused pending manual confirmation

check_failed flag on Jira ticket; IT Manager Slack alert sent; pipeline does not proceed to scorecard creation

E04

DocuSign template missing or misconfigured for agreement type

Jira ticket approved; DocuSign API call fails with 'Template not found' error

Error logged on Jira ticket; IT Manager and FullSpec support alerted via Slack; Notion register not updated; no silent completion

Jira ticket comment contains DocuSign error code; Slack alert sent to IT Manager; Notion register unchanged; envelope status = failed

E05

Approval email bounces or is undeliverable

Gmail sandbox send returns a permanent delivery failure (5xx SMTP error)

Delivery failure logged on Jira ticket; IT Manager notified via Slack to send the approval request via an alternative method; reminder schedule suspended

Jira ticket comment contains bounce detail; Slack alert sent; no further automated email attempts; reminder timer suspended

E06

Human timeout: no approval or rejection recorded within 10 business days

Approval email sent; two reminders sent (T+48h and T+96h); still no response at T+10 business days

Jira ticket escalated to 'Stalled' status; escalation Slack message sent to IT Manager and Operations Lead; process owner prompted to intervene

Jira ticket status = Stalled after 10 business days; Slack escalation delivered to both IT Manager and Operations Lead channels

E07

Slack workspace unavailable when notification should be sent

Rollout notification step reached; Slack API returns 503

Retry up to 3 times at 60-second intervals; after failure, notification content logged to Jira ticket as a comment so IT Manager can send manually

3 retry attempts in run log; Jira ticket comment contains the full notification text; no silent discard

Test and QA PlanPage 3 of 4

FS-DOC-06Quality

04Phase 3 — end-to-end production test

Important before running Phase 3: use an internal test email address (e.g. it-test@[YourCompany.com]) as both the requestor and the approval recipient. Submit the request with a clearly labelled test vendor name such as 'QA-Test-Vendor-[Date]' so the record is identifiable. After the run is confirmed as passing, void or delete the Jira ticket, the Notion evaluation pages, and the DocuSign envelope using the sandbox void option. Do not allow the test record to persist in the live software register. The FullSpec team will confirm deletion before handover.

Phase 3 uses production credentials across all five connected tools. The FullSpec team configures and executes each step. The process owner, IT Manager, is present throughout to observe outputs in real time and to record the final approval at T27. All outputs are verified against the expected values defined in Phase 1 and Phase 2 before the process owner sign-off is logged.

Scenario

Input

Expected Output

Pass Criteria

T23

Production form submission creates Jira ticket and Slack acknowledgement

Intake form submitted with test vendor name 'QA-Test-Vendor-[Date]', requestor it-test@[YourCompany.com], 3 users, $75/month

Live Jira ticket created in production project; duplicate check runs against live Notion register; Slack acknowledgement delivered to test address

Jira ticket visible in production board within 60 seconds; duplicate_flag = false (test vendor is unique); Slack DM or channel message delivered

T24

Production evaluation pipeline triggered: Notion checklist and scorecard created

Jira ticket status advanced to 'Evaluation In Progress' by FullSpec team

Security checklist and scorecard pages created in live Notion database; IT Manager assigned on checklist; all fields populated from ticket

Both Notion pages exist in production database within 90 seconds; field values match ticket submission exactly

T25

Production approval email sent and DocuSign envelope triggered on approval

Notion scorecard marked Complete; approval email sent to test approver (it-test@[YourCompany.com]); approver replies confirming approval; Jira status set to Approved

Approval email delivered from production Gmail account; DocuSign production envelope sent to test signatory address

Gmail sent-items confirms delivery; DocuSign production dashboard shows envelope status = sent; envelope ID written to Jira ticket

T26

Production register update and Slack rollout notification after DocuSign completion

DocuSign production envelope voided/completed via test signatory action

Notion software register entry created in production database with all required fields; Slack rollout notification sent to test channel

Notion register entry present with correct tool name, licence count, renewal date, owner, and cost; Slack message delivered to designated channel within 3 minutes of DocuSign completion

T27

Error log verification and process owner review and approval

FullSpec team presents run log to IT Manager showing all production steps, timestamps, field values, and system responses for T23 through T26

IT Manager confirms all outputs match expected behaviour described in this document; no unhandled errors in run log; process owner records approval in FullSpec confirmation log

Run log contains a timestamped entry for each of T23 to T26 with pass status; IT Manager approval recorded in the FullSpec confirmation system; this constitutes the process owner sign-off for the automation going live. Contact support@gofullspec.com to request a copy of the confirmation record.

After T27 is recorded, the FullSpec team will void the test Jira ticket, delete the test Notion pages, void the DocuSign envelope, and confirm in writing that no test data remains in any production system. The automation is then cleared for live operation on all incoming tool requests.

Test and QA PlanPage 4 of 4

Test and QA Plan

Test and QA Plan

01Testing approach

02Phase 1 — unit test cases

03Phase 2 — integration and edge case tests

04Phase 3 — end-to-end production test

More documents for this process