Cogeco Promotion Tracker
Scrapes Cogeco.ca's business internet pages, extracts promotions with real Chromium to bypass Cloudflare Turnstile, and diffs runs to track price changes and offer expirations.
Automates promotion monitoring on Cogeco.ca by driving a real browser to bypass Cloudflare's bot detection, parsing structured offer data (speed, price, terms, equipment), and emitting diffs when offers change or expire.
- TypeScript
- Playwright
- Cheerio
- Zod
- Node.js
- Chromium
- Cloudflare Turnstile
Cogeco's website pages are protected behind Cloudflare, and their package/promotion details sit behind Turnstile challenges that block plain HTTP and headless browser requests. cogopd solves this by launching a persistent Chromium browser via Playwright with stealth techniques: a Chrome 148 User-Agent, matching client hints, anti-webdriver masking, realistic plugin/hardware declarations, WebGL spoofing, and a persistent `.cache/browser-profile` that maintains `cf_clearance` cookies across runs. A one-time manual Turnstile click seeds the session; subsequent headless runs reuse the cleared cookie. The scraper pauses 3–6 seconds between page requests to mimic human cadence.
On each page, the tool extracts structured promotion data using Cheerio: package names, speeds (Mbps/Gbps), regular and promo monthly prices, savings amounts, promos duration and commitment terms, equipment included (Wifi 6 pods, Plume Workpass, modems), eligibility language (new-customer-only, cancellation clauses), time-limit callouts, and expiry dates. It parses HTML via multiple fallbacks—package card elements, heading-first slicing, and bundle promotion tables—and cross-validates with structured legal disclaimers buried in the page text.
Each run snapshots all targets into `data/snapshots/<runId>.json` with ISO timestamps and complete page metadata. The diff engine compares runs by target ID, detecting new/removed/changed promotions by package name signature, and tracks new price mentions and time-limit language. A markdown report flags promotions that beat the user's current plan (a hardcoded baseline: $175.05/mo, 2 Gbps down, 150 Mbps up, 4 pods) on price, speed, or upload capacity. This is designed for existing Cogeco customers who see website promos but need to quote them when calling the retention team—the tool eliminates manual checking and surfaces what changed since yesterday.
- Cloudflare Turnstile evasion via persistent browser profile with stealth masking (navigator.webdriver, plugin array, WebGL spoofing, chrome.runtime restoration)
- Multi-mode extraction: package card elements, heading+sibling text slicing, bundle promotion HTML tables, and legal disclaimer regex parsing
- Scheduled snapshots with ISO-timestamped runs; append-only historical record; JSON diff engine tracking added/removed/changed promotions and price mentions per page
- Baseline-relative reporting (user's plan vs. all offers) with filters for price beats, speed beats, and upload improvements; negotiation checklist for retention calls
- CLI with four commands: snapshot (fetch all targets), diff (JSON output vs. previous), report (markdown with flagged offers), monitor (snapshot + report one-shot)