colab
Competitor Content Gap Analysis
Compare your site against a competitor's using embeddings and topic clustering instead of keyword lists. Free local version, or a few dollars for the LLM write-up.
A keyword list tells you what you’re missing. It doesn’t tell you whether a competitor has two pages on a topic or twenty, or whether your gap is structural or just a couple of posts. So these compare two crawls as topics instead.
Three tiers, each building on the one before it. Start with Rising Tide and move up only if you need what the next one adds.
| Notebook | Method | What it adds | Cost |
|---|---|---|---|
| Rising Tide | Sentence Transformers + K-Means | Topic clusters, breadth and depth charts, a UMAP map | Free, fully local |
| Open Water | BERTopic + capped LLM synthesis | Auto-labelled clusters, a prioritised gap report | ~$0.01 to $0.05 per run |
| Deep Sea | BERTopic + full LLM synthesis | No cluster cap, coverage quality, cannibalisation flags | A few dollars |
The real jump between tiers is K-Means to BERTopic. K-Means makes you guess a cluster count up front, which is exactly the thing you don’t know yet. BERTopic works it out from the data and names each cluster readably. The LLM step on top is what turns clusters into something you can hand to a content team.
One thing before you run Open Water or Deep Sea: those tiers send your content to an LLM. Your client probably didn’t agree to that, and your security team almost certainly has an opinion. (Rising Tide is fully local, so it’s the safe one to reach for while you go ask.)
You’ll need a CSV crawl of your site and one of a competitor’s.