Keyword research has a default output: a spreadsheet with keywords in one column, monthly search volume in another, difficulty in a third, sorted descending by volume. Someone picks from the top and writes content.
That process reliably produces content targeting terms with high volume, high difficulty, and unclear commercial value, while missing the terms that actually convert. The fix is starting from intent and treating volume as a tiebreaker rather than a sort order.
Four Intents, and Only Two Usually Matter
Search intent divides into informational, navigational, commercial investigation, and transactional. Every keyword belongs to one, and the type determines what content can possibly rank.
Informational queries want understanding. What is, how does, why does. Navigational queries want a specific destination and are usually branded. Commercial investigation queries want comparison before deciding: best, versus, alternatives, review. Transactional queries want to act: buy, pricing, near me, book.
For most businesses, commercial investigation and transactional queries produce the revenue, and informational queries produce the traffic. Those are different things and a keyword list that does not separate them will overweight the second.
The practical test for intent is not guessing from the phrasing. Search the term and look at what ranks. If page one is entirely guides and Wikipedia, the intent is informational and your product page will not rank there regardless of optimisation. Google has already decided what that query means.
Volume Is the Least Reliable Number in the Spreadsheet
Reported volume is an estimate derived from sampling, it varies substantially between tools for the same term, and it aggregates variants in ways that inflate some terms and deflate others.
More importantly, volume is a poor proxy for value. A term with two hundred monthly searches from people ready to buy is worth more than a term with twenty thousand searches from people writing a school assignment.
The correction is estimating value directly rather than inferring it from volume. For each candidate term, ask what a person searching it wants, how close that is to a purchase, and what a conversion is worth. A term with modest volume and clear buying intent beats a high volume term with none, and it is usually easier to rank for.
Difficulty Scores Are Directional at Best
Every tool calculates keyword difficulty from link metrics of currently ranking pages. That captures one input and misses several others: how well those pages match intent, how old and stale they are, and whether the SERP is dominated by formats you cannot compete with.
A term showing high difficulty where page one is filled with thin, outdated content is easier than the score suggests. A term showing low difficulty where page one is entirely major publishers is harder.
Read the SERP rather than the score. Ten minutes looking at what actually ranks tells you more about whether you can compete than any number.
Where to Find Terms Tools Do Not Surface
Keyword tools are built from search log data, which means they show terms people have already searched enough times to register. That is a lagging view and it misses the specific language your buyers actually use.
Sales calls are the strongest source most companies own and never mine. The words a prospect uses to describe their problem, before anyone has trained them on your product vocabulary, are the words they type.
Support tickets carry the post purchase equivalent, unfiltered by marketing language.
Community discussions where people describe a requirement and ask for recommendations show you both the problem framing and the vocabulary. The community signals guide covers why those platforms matter for visibility. They are also raw research material.
Your own comment sections and on-site questions carry the same value, per the comments guide, since every question a reader asks is a query your content left unanswered.
Search Console's query report shows terms you already rank for, including ones you never targeted. Filtering for queries with impressions and near zero clicks surfaces terms where you appear but the content does not match what the searcher wanted, which is usually a quick content fix rather than a new page.
Clustering Beats a Flat List
Individual keywords are not the unit of work anymore. Google resolves synonyms and related phrasings to the same underlying topic, which means one page can rank for dozens of variants and building a page per variant creates cannibalisation.
Group terms by the answer they want rather than by their words. If three phrasings would be satisfied by the same content, they belong on one page. If two phrasings look similar but the SERPs for them show different content types, they are different topics regardless of wording.
SERP overlap is the practical test. Search two candidate terms and compare page one. Substantial overlap means one page. No overlap means two.
This clustering is also what produces the topical structure covered in the topical authority guide, where a pillar page and its supporting content cover a subject comprehensively rather than scattering across unconnected posts.
Cannibalisation Is the Predictable Failure
Two pages targeting the same intent compete with each other. Google picks one, usually not the one you wanted, and both underperform what a single consolidated page would achieve.
This accumulates over years of publishing without a keyword map. Audit by searching your own site for a target term with a site query and seeing how many pages surface. More than one page genuinely competing means a consolidation decision, covered in the pruning guide.
The prevention is a keyword map: one document recording which page owns which topic cluster. It is unglamorous and it stops the problem existing.
Long Tail Terms Are Where Most Sites Should Start
Definitional terms deserve separate treatment here, since they behave differently from both, and the definition pages guide covers when a term warrants its own page.
Head terms carry volume and competition. Long tail terms carry specificity and intent, and they are usually winnable within a quarter rather than a year.
A new or mid authority site targeting a head term is choosing a fight it will lose slowly. The same site targeting twenty specific long tail terms will rank for several of them quickly, accumulate topical signal, and eventually have the authority to compete higher up.
This is not a compromise position. It is the correct sequencing, and sites that skip it typically spend a year producing content that never ranks.
How This Connects to AI Search
Worth a brief note rather than a full detour. AI search queries run considerably longer and more conversational than typed search queries, which means a meaningful share of them do not appear in keyword tools at all.
The practical implication for keyword research is that intent based clustering transfers well and volume based selection does not. Content built to resolve a situation comprehensively will match a wide range of phrasings including ones no tool reports. Content built to hit an exact phrase will not. The conversational search guide covers this in depth if it is relevant to your programme.
A Workable Process
Start with the problems your buyers describe, sourced from calls, tickets, and communities rather than tools.
Expand each into the phrasings people would use, then run those through a tool to find variants you missed rather than to generate the initial list.
Cluster by SERP overlap into topics rather than keeping a flat list.
Classify each cluster by intent and check what currently ranks, discarding any where the format is one you cannot produce.
Prioritise by proximity to revenue first and estimated difficulty second. Volume is a tiebreaker.
Record the map so the next person publishing knows what already exists.
Checking What You Actually Rank For
Search Console remains the only source of truth for your own performance, since it reports actual impressions and clicks rather than modelled estimates. Every third party rank tracker is approximating what Search Console tells you directly.
The NotionCue AI Answer Gap Finder covers the adjacent question of which sources answer your target topics in AI surfaces, which is increasingly worth knowing alongside classic ranking data for commercial investigation queries specifically.
Start your free NotionCue trial and check your highest value commercial terms across both surfaces. The gap between where you rank and where you are cited is frequently larger than teams expect.
Before your next keyword research project, pull the last twenty inbound enquiries and note the exact words those people used to describe what they needed. That list will contain terms no tool suggested and it will convert better than anything the tool did suggest.
Common Questions
How many keywords should one page target?
One topic cluster, which may contain dozens of phrasings. Targeting one exact phrase per page produces thin pages and cannibalisation. Targeting several unrelated topics on one page produces a page that ranks for none of them.
Are free keyword tools sufficient?
For a small site, largely yes. Search Console, Google's own keyword planner, and reading SERPs manually cover the majority of what matters. Paid tools save time on competitive research and variant discovery rather than providing information unavailable elsewhere.
How often should keyword research be redone?
The map should be maintained continuously as pages publish. A full refresh is worth running annually, or whenever the business changes what it sells, since that invalidates the intent mapping underneath everything.