Keyword research has long been a keystone of SEO, but in today’s world, keyword research must now evolve to meet the demands of AI-powered search, generative responses, voice queries, etc. That may sound overwhelming, but here is the good news – when keyword research in the AI era is done correctly and well, you can uncover more meaningful opportunities, align better with user intent, and future-proof your content strategy.

Here’s how to approach keyword research now—what’s changed, what still works, and how to do it successfully.

What’s Changed + Why It Matters

In its simplest form, keyword research was previously conducted by assessing short-tail and long-tail keywords related to your product or service to better understand how potential customers were searching for your offerings (i.e. searching “business attorney” or “business litigation attorney in Columbus, OH”). Those terms or phrases would then be assessed based on search volume and keyword difficulty and matched to user search behavior to create pages that matched that phrase.

Search engines are now increasingly using AI, machine learning, and natural-language processing (NLP) to determine intent, context, and conversational phrasing. This demands that keyword research move beyond simple terms and phrases and focus more on topic clusters, conversational prompts, user questions, and the way people are speaking when using voice or chat interfaces.

With this in mind, traditional metrics, like high search volume and low keyword difficulty, still matter, but they are less reliable on their own as many queries are now resulting in zero-click answers where users are receiving everything they need in AI overviews or chat responses without having to go to an actual website.

All of this means that keyword research in this era should blend traditional keyword research with emerging strategies geared for AI-search visibility.

A Step-by-Step Guide for Keyword Research in the AI Era

Now that we have a better understanding of why keyword research has to adapt, here is a practical workflow that you can follow:

Start with Audience & Intent

First, look at your ideal customer profile and consider the following questions. We will use a business law firm to answer these same questions to provide a concrete example.

What questions are they asking?

In the case of a law firm - What is due diligence in mergers and acquisitions deals?

What problems are they trying to solve?

A client might ask - How do I form an LLC?

What is their intent?

A potential search might be - Find a business attorney near me

Some of these questions can be answered internally by assessing support emails, social media comments, and surveys. You can also look at industry forums to see what questions commonly pop up to give you an idea of real-world language and conversational phrasing.

Use Traditional Tools + AI-Assisted Research

Once you have a better understanding of your audience, you should leverage keyword tools, like Google Keyword Planner, SEMrush, AnswerThePublic, or Keywords Everywhere to gather information on search volume, difficulty, and competition. Look for high-intent, low-competition conversational queries—these often drive more qualified traffic and may perform better in AI-first environments. Using the business law firm as an example, within SEMrush you would get keyword results like:

You can then incorporate AI-powered tools by asking prompts such as: “What conversational queries do people ask when considering [your service/product]?” or “What related questions might someone ask after they search [seed keyword]?” This helps uncover long-tail, conversational, and intent-rich queries.

Combine this data to create your working keyword/topic list.

Determine Your Content Clusters/Pillars

Now that you have a pretty good list going, these keywords should be organized into your content clusters or pillar content. These are the primary topics that your site represents, and they are crucial for helping search engines and AI better understand what your website is the authority on. So, for example, if you are a business law firm, your main pillar would be “business law,” supported by practice areas like litigation, mergers and acquisitions, and HR and compliance. Each of those practice areas should then have supporting content that aligns with what users are looking for.

To determine what content is best for your keywords, you should assess the intent behind the keywords (i.e. what is the user trying to find or accomplish with this search?). Traditional intent only considered navigational, commercial, and informational searches, but the overall understanding around intents is deepening. We need to go beyond the surface to better serve our audiences. Good intent mapping results in better visibility since search engines care more about intent alignment than exact phrasing.

Evaluate Content Opportunities + Gaps

This should be a two-pronged approach: Analyze your site and analyze your competition.

When looking at your site, audit your existing site content to determine what fits within your clusters, what is outdated, and what is not relevant. If your site is large, you can use tools like ScreamingFrog to crawl your site and look for opportunities.

For analyzing your competition, there are tools you can pay for, or you can look at AI summaries and overviews for your keywords and run your queries through generative engines like ChatGPT and Perplexity to see which sites are being referenced and cited. Look for content gaps - i.e. content that no one is covering in depth. This is an opportunity to get visibility!

Prioritize topics aligned with your business goals and expertise—not just what looks “easy.” Depth and authority are what matters.

Build Out Your Content Plan

By this stage, you should have a good list of keywords and topics that are well aligned with your audience and your business. These opportunities and gaps you discovered should inform your content plan moving forward.

Start with your main and supporting pillars (In the law firm example, we have business law, supported by litigation, mergers and acquisitions, HR and compliance)
Prioritize your topics by what is most important to your audience and most relevant to your business – here it is probably helpful to come up with a content calendar. It doesn’t have to be fancy, but you want to be able to have a schedule you can follow and a point of reference. You can start with this spreadsheet as a content calendar template.
Identify target keywords (and conversational prompts) for each pillar
Assign topics to your content pillars and build out those supporting pieces (blogs, FAQs, short-form videos, etc.)
If applicable, assign content to authors/creators with relevant expertise or experience to boost authenticity and authority
Incorporate internal linking strategy, which is linking from supporting content to the pillar, reinforcing topic cluster structure
Define metadata, schema markup, headings, and conversational hooks for each page, optimizing for both users, search engines, and AI/LLM systems. Make sure your content is well-sourced, in-depth, and provides sources when appropriate. Keep in mind - you do not want to over-optimize for exact keywords. Keyword stuffing is not beneficial for users, nor will it help you in discovery!

Monitor + Refine Your Content Efforts

Once you have implemented the pillar content and kicked off your strategy of developing supporting content, you need to measure performance. We recommend tracking the following metrics using Google Analytics and Google Search Console; however, if you pay for a subscription-based tool, incorporate that data as well! We offer a reporting template that compiles most of this data for you.

Traffic volume and sources

Organic
Referral + Referral sources
Direct
Social

Impressions
CTR (for queries and pages)
Ranking (for queries and pages)
Engagement on page

Time on page
CTAs
Returning visitors
Form fills

Branded keyword volume
If you have access - content appearing in AI/LLM answers

This data can then be used to refine your efforts. If you see that a piece is performing well, be sure to analyze what worked. If you notice a piece isn’t performing, assess if the content is well-aligned with the intent. Content strategies should be audited regularly as search behavior shirts and AI advances.

TL;DR

Keyword research in the AI era means enhancing the traditional methods with the new tools out there and understanding what is available. By developing a deeper understanding of conversational queries and user intent, you are bound to be able to create content that not only helps you get discovered but that also provides more value to your target audience. Like all things in the digital marketing world, you have to be adaptable and flexible as new technology emerges.

How to Conduct Keyword Research in the AI Era