Search versus chat…. are they the same or different?

Written By Michael Murphy



Perplexity, ChatGPT, and Claude are conversational interfaces.  They’re changing the way we search for information online.  But a question:

  • Is chat just a new interface for doing the same thing, i.e., search? 

  • Or is it an entirely new interaction altogether?

  • If chat and search are different, which one is better?

The answer is:  Chat is an entirely new interaction

  • But how?  What makes it different from search?  

First, let’s answer the easier question.  How are search and chat the same?  A simple answer is they both enable a user to find information.  But “to find information” is a broad use case.  Differences appear at a deeper level.

One way to distinguish them is by the scope of information that each tool retrieves.  The motivations that lead us to search for information online are the same as the motivations we have in the physical world.  In some cases you know what you want, so you look for that one thing.  In other cases, you’re not sure what you want, so you look for a selection. 

So, by one measure, the use cases for an info-seeking utility can be distinguished by the need for:

  • an answer, or

  • a selection of candidate answers




That doesn’t answer the question. Which is better?

The next question is whether — for each of these tasks — is one utility better than the other?   

That’s actually two questions. And there’s an answer for each case.  The answer to both is: ‘yes’.  One utility is better than the other in each case (but it’s different for each case).  The deciding factor is the kind of input each utility lends itself to. 

The search bar lends itself to keyword entry.  It strives to return relevantly matched documents from an index.  Its output is a results list that users browse.

The chat interface lends itself to natural language queries, i.e., questions and statements.  It strives to return one, best answer from a model.  If it fails, it seeks to use context and another query from the user to find a better answer.  The interaction is iterative.

Re-phrasing these descriptions:

  • Search finds candidates, i.e., multiple results that users browse.

  • Chat finds an answer, i.e., one result, in as few iterations as possible.

As further comparisons:

  1. for chat, a user enters iterative queries

  2. for search, a user enters one search term and browses multiple results

  1. for chat, a user seeks an experience that is as short as possible

  2. for search, a user may tradeoff the cost of a longer experience, to keep the opportunity to make comparisons themselves

  1. for chat, a user wants the utility to make the decision, i.e., know the answer

  2. for search, a user wants to make the final choice themselves


Google completed a study on the topic:  ChatGPT vs Google:  A Comparative Study of Search Performance and User Experience.   https://arxiv.org/pdf/2307.01135

But, ultimately, the answer is that chat and search are different. One is better than the other — depending on the task.

 

Which one is better for e-commerce?

Narrowing the use case to e-commerce, either chat or search can work.  But is one usually better? In the physical world, selection (and price) are primary factors in deciding where we shop.  It’s the same online. For most shoppers, selection is paramount. 

So, for e-commerce, the advantage leans strongly towards search. Search enables a shopper to make comparisons across a multitude of variables that change quickly and are often known only to them, e.g., design, fabric, color, fit.  People want decision-making control over these, because combinations of subtle variables can vary so broadly.

Even with the power of today’s LLMs, it’s still hard for models to capture and update these individual nuances effectively, especially at scale.  But a collection of search results lets us (humans) do that evaluation better and faster, assuming that the selection of choices we have are relevant.




But what about personalization?

Oops! Personalization changes everything. 

Interestingly, because chat interfaces retrieve info from LLMs, they can maintain context.  That means chat can excel at personalization, by retaining information about a shopper’s tastes and preferences. So, adding the condition that shoppers want personalized results, the advantage may shift from search to chat.   The value of personalization is that high.

But that raises the question:  is personalization a capability unique to chat, i.e., LLMs?  The answer is ‘no’, which points to an attractive technical opportunity.  Can search be personalized? 

The answer is ‘yes’.  That’s an interesting technical challenge. It’s definitely a tremendous business value. Advances in AI are making this capability more possible. Many major tech companies (and brands) are pursuing these capabilities now.  But there is plenty of opportunity to offer this capability to brands who don’t have engineers to build this themselves.

KLOA is working on that — offering personalized search-as-a-service to small and mid‑size brands (SMB).  If you’re an SMB, or a large enterprise without the engineering team to develop this, browse our site! See how KLOA can personalize your brand’s search experience for your shoppers.

 

Conclusion

Search and chat are not the same.  One is better, depending on the use case.  For e-commerce, search is preferred because the dominate use case prefers comparisons.  But personalization sharply shifts that balance.  Therefore, for commerce, shopper preference falls in this order:

  1. Personalized search

  2. Personalized chat

  3. Un-personalized search

Because personalized search isn’t a widely available capability yet, this points to a clear business opportunity.

Comparison of the attributes of chat and search

Chat

  • Conversational input

  • Intends to avoid the need to browse

  • Ideally yields one, best result

  • Ideally the final step in the experience

  • Eliminates a pain point (finding info)

  • Generally, not fun

Search

  • Keyword input

  • Intends to yield a browsing opportunity

  • Ideally yields multiple results

  • Expectation that experience continues (next step is browsing)

  • Can be rewarding (if some results are richly relevant)

  • Often fun

Diagrams of navigation for search and chat

 
 

For further information, enter these prompts into Perplexity…

  • Are there any reports on the utility of search vs chat for information discovery?

  • How do users perceive the effectiveness of chatbots versus search engines?

  • How does user experience differ between chatbots and traditional search engines?

The result from the first Perplexity query above is below:

 
 
Michael Murphy