Integration and API Spec

Integration and API Spec

Competitor Monitoring Automation

REST API v3. Used by Agent 1 (Data Collection Agent) to pull keyword position changes, estimated organic traffic, and new paid search keywords for each competitor domain on a weekly schedule.

// Endpoint called per competitor
GET https://api.semrush.com/?type=domain_ranks&key={{SEMRUSH_API_KEY}}&domain={{competitor_domain}}&database={{SEMRUSH_DATABASE}}
GET https://api.semrush.com/?type=domain_organic&key={{SEMRUSH_API_KEY}}&domain={{competitor_domain}}&database={{SEMRUSH_DATABASE}}&display_limit=10

// Response fields extracted
Db, Domain, Rank, OrKw, OrTr, AdKw, AdTr

// Staged output fields
semrush.domain | semrush.organic_keywords | semrush.organic_traffic | semrush.paid_keywords | semrush.rank

REST API v3 (api.ahrefs.com). Used by Agent 1 (Data Collection Agent) to retrieve new backlinks and recently published content for each tracked competitor domain.

// Endpoints called per competitor
GET https://api.ahrefs.com/v3/site-explorer/new-backlinks?target={{competitor_domain}}&date_from={{run_date_minus_7d}}&limit=20
GET https://api.ahrefs.com/v3/site-explorer/pages?target={{competitor_domain}}&date_from={{run_date_minus_7d}}&limit=20

// Response fields extracted
url_from | domain_rating_source | anchor | url_to | first_seen
url | title | words | traffic | value

// Staged output fields
ahrefs.domain | ahrefs.new_backlinks[] | ahrefs.new_pages[] | ahrefs.domain_rating_change

Public RSS feeds generated by Google Alerts. Used by Agent 1 (Data Collection Agent) to collect news and web mentions for each tracked competitor keyword since the last run.

// HTTP GET per competitor alert feed
GET {{google_alerts_feed_url}}

// XML fields parsed per entry
<entry>
  <title> | <link href> | <published> | <content>
</entry>

// Staged output fields (per item)
alerts.competitor_name | alerts.headline | alerts.url | alerts.published_at | alerts.snippet

Google Sheets API v4. Used by Agent 2 (Intelligence Summariser Agent) for two purposes: reading the competitor source list at the start of each run, and appending a new dated row of findings per competitor after the digest is generated.

// Read competitor list
GET https://sheets.googleapis.com/v4/spreadsheets/{{GOOGLE_SHEETS_SPREADSHEET_ID}}/values/Competitors!A2:C

// Append weekly log row per competitor
POST https://sheets.googleapis.com/v4/spreadsheets/{{GOOGLE_SHEETS_SPREADSHEET_ID}}/values/Weekly Log!A:J:append
Body: { valueInputOption: 'USER_ENTERED', values: [[run_date, competitor_name, organic_keywords, organic_traffic, new_backlinks, new_pages, alert_mentions, flag_significant, digest_excerpt]] }

// Staged input fields consumed
semrush.organic_keywords | semrush.organic_traffic | ahrefs.new_backlinks[] | ahrefs.new_pages[] | alerts.headline | ai_summariser.flag_significant | ai_summariser.digest_excerpt

Slack Web API (chat.postMessage). Used by Agent 2 (Intelligence Summariser Agent) to post the formatted weekly intelligence digest to the designated marketing channel, with highlights in the main message and per-competitor detail in a thread.

// Main digest message
POST https://slack.com/api/chat.postMessage
Headers: { Authorization: Bearer {{SLACK_BOT_TOKEN}} }
Body: { channel: {{SLACK_CHANNEL_ID}}, blocks: [...highlight_blocks], text: 'Weekly Competitor Digest - {{run_date}}' }

// Per-competitor thread reply
POST https://slack.com/api/chat.postMessage
Body: { channel: {{SLACK_CHANNEL_ID}}, thread_ts: {{parent_message_ts}}, text: '{{competitor_name}}: {{competitor_section_text}}' }

// Fields consumed from AI summariser
ai_summariser.digest_date | ai_summariser.highlights[] | ai_summariser.competitor_sections[]{name, body, flag_significant}

HubSpot CRM API v3 (Engagements / Notes). Used by Agent 2 (Intelligence Summariser Agent) to log a note against the marketing campaign or company record after the Slack post, keeping the CRM context current for sales and marketing alignment.

// Create note per competitor
POST https://api.hubapi.com/crm/v3/objects/notes
Headers: { Authorization: Bearer {{HUBSPOT_PRIVATE_APP_TOKEN}} }
Body: {
  properties: {
    hs_note_body: '{{note_body_text}}',
    hs_timestamp: '{{run_timestamp_ms}}'
  },
  associations: [{ to: { id: {{hubspot_company_id}} }, types: [{ associationCategory: 'HUBSPOT_DEFINED', associationTypeId: 190 }] }]
}

// Fields consumed
ai_summariser.competitor_sections[]{name, body} | HUBSPOT_COMPETITOR_RECORD_MAP | run_date | sheets_log_url

// Credential store contents
// All values are encrypted at rest. Reference by key name only in workflow steps.

SEMRUSH_API_KEY                 = "<semrush_account_api_key>"
SEMRUSH_DATABASE                = "us"                          // e.g. us, uk, au

AHREFS_API_TOKEN                = "<ahrefs_bearer_token>"

GOOGLE_ALERTS_FEEDS             = [                              // JSON array
  { "competitor_name": "CompetitorA", "feed_url": "https://www.google.com/alerts/feeds/<uid>/<alert_id>" },
  { "competitor_name": "CompetitorB", "feed_url": "https://www.google.com/alerts/feeds/<uid>/<alert_id>" }
  // ... one entry per tracked competitor
]

GOOGLE_SHEETS_CREDENTIAL        = "<service_account_json_key_or_oauth_refresh_token>"
GOOGLE_SHEETS_SPREADSHEET_ID    = "<google_sheets_file_id>"

SLACK_BOT_TOKEN                 = "xoxb-<slack_bot_token>"
SLACK_CHANNEL_ID                = "<slack_channel_id>"          // e.g. C01234ABCDE

HUBSPOT_PRIVATE_APP_TOKEN       = "<hubspot_private_app_token>"
HUBSPOT_COMPETITOR_RECORD_MAP   = {                              // JSON object
  "CompetitorA": "<hubspot_company_id>",
  "CompetitorB": "<hubspot_company_id>"
  // ... one entry per tracked competitor
}

AI_SUMMARISER_API_KEY           = "<llm_provider_api_key>"      // e.g. OpenAI or equivalent
AI_SUMMARISER_MODEL             = "gpt-4o"                      // or configured model name
AI_SUMMARISER_PROMPT_VERSION    = "v1.2"                        // increment on prompt changes

WORKFLOW_TIMEZONE               = "America/New_York"
WORKFLOW_SCHEDULE_DAY           = "Monday"
WORKFLOW_SCHEDULE_TIME          = "07:00"

// Do not hardcode any of the above values in workflow step configurations.
// Rotate tokens in the credential store only; update both agents if a shared token changes.