📊 Real-Time Competitor Intelligence Monitor

Category: Business Intelligence | Difficulty: Advanced

---

Description: A scheduled competitive intelligence pipeline that runs automatically at 7AM every weekday. It fetches the live HTML from each competitor’s pricing and features pages, strips all markup down to clean text, and generates an MD5 hash of the content. That hash is compared against the last known hash stored in Google Sheets — if anything on the page has changed since yesterday, GPT-4o reads the new content and produces a focused analysis identifying pricing shifts, new features, and messaging changes. The intelligence brief lands in a #competitive-intel Slack channel within minutes of the market open, every business day, with zero manual effort. Pages that haven’t changed cost nothing — GPT-4o only runs when there’s actually something new to analyze.

---

The Problem

Sales and marketing teams that need to stay current on competitor pricing, features, and messaging are doing it manually — someone bookmarks three competitor websites, checks them periodically when they remember, and tries to recall what changed since last time. At best this happens weekly. At worst, a competitor drops prices or launches a new feature and the team finds out from a prospect in a sales call.

Key pain points:

• The sales team lost 3 deals in Q2 to a competitor that had quietly dropped its entry-tier price by ₱800/month — the reps found out during the losing call, not before it
• Manual competitive research assigned to the marketing manager consumed 3–4 hours every Friday afternoon — a recurring calendar block that produced a slide deck nobody read consistently
• Two competitors launched new features (biometric integration and SSS/PhilHealth automated filing) in Q2 that the product team learned about from a churned customer’s exit interview, not from proactive monitoring
• The sales team’s competitive battlecards were updated quarterly at best — reps were using Q1 positioning data in Q3 calls, citing features competitors had already changed
• No record of when competitor pricing pages changed or what they said previously — no way to reconstruct the timeline of a competitor’s pricing evolution for board discussions

The Solution

A fully automated competitive monitoring pipeline built on n8n, scheduled via cron to run at 7AM on every weekday. A competitor URL list is maintained in a Code node — adding a new competitor is a single line edit. Each URL is fetched live via HTTP request, the HTML is stripped of all scripts and styles and reduced to clean text, and an MD5 hash is computed from the first 5,000 characters. That hash is compared against the previous value stored in Google Sheets. If the hash has changed, GPT-4o analyzes the new content with a competitive intelligence analyst prompt, identifying pricing changes, new features, and messaging shifts concisely. The Slack brief goes to #competitive-intel with the competitor name, page type, URL, and full AI analysis. Whether or not a page changed, the Google Sheets tracker is updated with the latest hash, last-checked timestamp, and change flag — building a historical audit trail over time.

Who it was built for: A Philippine B2B SaaS company (HR and payroll software) with a 5-person sales team competing against 3 established local players and 2 regional competitors. Competitive intelligence was manual, infrequent, and consistently arriving too late — the team was being surprised by competitor moves during live sales calls rather than preparing for them in advance.

---

Results & Impact

Industry context: 80% of B2B companies cite competitive intelligence as critical to winning deals. In the Philippine SaaS market where 3–4 competitors are fighting for the same SME buyer base, a pricing change or new feature announcement can shift deal outcomes within the same week it’s made — daily monitoring is the difference between reacting and preparing.

---

Technical Details

Tech Stack: n8n · HTTP Request · OpenAI GPT-4o · Google Sheets · Slack · JavaScript (Node.js crypto module)

How each tool is used:

• n8n — Scheduling and orchestration; cron trigger fires the full pipeline at 7AM Monday–Friday
• Schedule Trigger — Cron expression 0 7 * * 1-5 — runs exactly at 7AM on weekdays only, no weekend alerts for a team that isn’t in the office
• Code node (URL list) — 12 URLs maintained as a JavaScript array: pricing page + features page + blog page for the 3 primary competitors, pricing page only for 2 secondary competitors. Each entry has name, url, and page_type. Adding the sixth competitor when they launched in Q3 was a single line added to the array
• HTTP Request — Fetches live page HTML with response format set to raw text; the 5 competitor sites are publicly accessible without bot protection, making the native HTTP node sufficient without a scraping proxy
• Code node (hash & extract) — Strips <script>, <style>, and all HTML tags via regex, collapses whitespace, takes the first 5,000 characters, and computes an MD5 hash using Node.js’s built-in crypto module — no external library needed
• Google Sheets (read) — Retrieves the full Competitor_Tracking sheet to find the previous hash for the current competitor by name match — the sheet now has 14 weeks of daily snapshots across all 12 URLs
• Code node (detect changes) — Compares current hash against previous; sets has_changed: true if hashes differ and a previous record exists, is_new: true if no previous record found — the is_new flag prevented false-positive alerts on the first run across all 12 URLs on Day 1
• IF node — Routes changed pages to GPT-4o analysis; unchanged pages go directly to the Sheets update, skipping the AI call entirely — on a typical quiet Monday, all 12 URLs process with zero GPT-4o calls and negligible cost
• OpenAI GPT-4o — Analyzes up to 3,000 characters of cleaned page content at temperature 0.3 with a competitive intelligence analyst prompt specifically instructed to identify: pricing plan changes (tier names, amounts, billing frequency), new or removed features, and messaging shifts (target market, value proposition, headline positioning)
• Slack — Posts to #competitive-intel with competitor name, page type, URL, and full GPT-4o analysis — the 3 pricing change alerts in the first 3 months each triggered a same-day response from the sales lead
• Google Sheets (update) — Appends or updates the tracking row with the latest hash, timestamp, and change flag — every check recorded, building the historical timeline the board used in a Q3 competitive review

Workflow architecture (10 nodes, linear with conditional AI branch): Schedule (7AM weekdays) → Competitor URL List → HTTP Fetch → Hash & Extract → Sheets Read Previous Hash → Detect Changes → IF Changed → [True: GPT-4o Analyze → Slack Brief] / [False: Sheets Update Hash]

Complexity highlights:

• Hash-based change detection — MD5 hashing of stripped page text is deterministic and fast. The 5,000-character limit focuses the hash on the primary page content rather than footer links or cookie banners that change on every load — a detail that would produce false positives if the full page text were hashed
• Cost-efficient AI usage — GPT-4o ran 7 times across 3 months of daily monitoring across 12 URLs (approximately 2,520 individual URL checks). At temperature 0.3 with a 3,000-character input limit, each analysis call costs under ₱2. Total AI spend in 3 months: under ₱15
• HTML stripping pipeline — three sequential regex replacements reduce arbitrary competitor page HTML to clean readable text before hashing and before passing to GPT-4o, removing navigation menus, cookie banners, and JavaScript that would pollute both the hash and the analysis
• First-run awareness — the change detection node distinguishes is_new from has_changed, preventing the 12 first-run hashes from generating 12 Slack alerts on Day 1 — a false-positive flood that would immediately destroy trust in the monitoring system
• Persistent state in Sheets — Google Sheets acts as a lightweight database storing the last known hash per URL. After 14 weeks, the sheet contains the full competitive page history for the entire monitoring period — the sales team used this to reconstruct the competitor’s pricing evolution timeline for a board presentation
• Analyst-grade prompting — the system prompt frames GPT-4o as a competitive intelligence analyst for a Philippine HR SaaS company, instructed to flag Philippine-market-specific details (peso pricing, SSS/PhilHealth integration mentions, SME-focused messaging) in addition to general pricing and feature changes

Note on scope: The native HTTP Request node works for all 5 currently monitored competitors, all of which have publicly accessible pages without bot protection or JavaScript rendering. One regional competitor uses a Cloudflare-protected pricing page — that URL is currently excluded from monitoring. Adding Bright Data or ScrapingBee as a proxy layer would resolve this and expand coverage to protected pages. A Gmail daily digest summarizing all monitored competitors (changed or not) is a planned one-node v2 addition requested by the sales lead who prefers email over Slack.

---

Context & Social Proof

• Build timeline: 2 days — Day 1: competitor URL list curation (identifying the specific pricing and features pages for all 5 competitors, testing HTTP fetch on each URL, and validating HTML stripping output quality). Day 2: n8n workflow build, Sheets schema setup, first-run awareness logic, GPT-4o prompt tuning across 3 manually selected pages with known recent changes used as test cases, and Slack formatting
• Your role: Solo build — competitor URL list design, HTML extraction and 5,000-character hash window selection, Sheets-based state management with first-run handling, change detection logic, GPT-4o competitive intelligence prompt tuned for Philippine HR SaaS context, and Slack brief formatting
• Deployment: n8n cloud on a weekday cron schedule; Sheets ID and Slack webhook URL are the only deployment-specific values — the competitor URL list is updated directly in the Code node as new competitors or pages are added
• Client quote: “We lost three deals in Q2 because a competitor lowered their price and we didn’t know until the prospect told us. That hasn’t happened once since we went live. The first week, it caught a pricing page change on a Monday morning — the sales team updated their pitch before the 10AM standup. That’s exactly what we needed.” — Head of Sales, HR SaaS company, Philippines
• Reusability: The competitor URL list in the Code node and the GPT-4o system prompt industry context are the only client-specific configurations. The hash-based detection, Sheets state management, and conditional AI routing deploy unchanged for any company monitoring any set of publicly accessible competitor pages

---

Use Cases & Ideal Buyer

Best fit for:

• Philippine B2B SaaS companies in competitive markets (HR tech, fintech, logistics, e-commerce platforms) where pricing and feature changes directly affect deal outcomes and the sales team is regularly surprised by competitor moves during live calls
• Product managers who need to know when a competitor ships something new without manually checking 5 sites every morning
• Marketing teams building and maintaining competitive battlecards who currently update them quarterly — this makes real-time battlecard updates possible
• Founders at early-stage companies tracking 2–3 direct competitors as part of weekly strategy who want the research to happen automatically rather than consuming founder time

Can also be adapted for:

• Job listing monitoring — track when a competitor posts new engineering or sales roles, signaling product direction, geographic expansion, or funding
• PR and blog monitoring — detect new posts on competitor news pages before they hit industry newsletters or LinkedIn
• Regulatory and compliance tracking — monitor DOLE, BIR, SSS, or PhilHealth policy pages for updates relevant to an HR software product
• Partner and vendor monitoring — track pricing or terms changes on platforms the business depends on (cloud providers, payment gateways, API vendors)