GenAI for Complex Questions, Search for Critical Facts: How Users Choose Between AI and Traditional Search
In-depth discussion
Easy to understand
0 0 1
Nielsen Norman Group's study observes how users choose between generative AI chatbots and traditional search for real information tasks. Users turn to AI when starting with vague ideas, handling multiple constraints, or aggregating information from many sources, as it reduces clicks, keyword foraging, and working-memory load. They revert to traditional search for validation, specific pricing or key facts, high-stakes decisions, citation transparency, and specialized academic research, treating the two tools as complementary.
main points
unique insights
practical applications
key topics
key insights
learning outcomes
• main points
1
Grounded in an observational study with real user tasks and think-aloud sessions across varying AI experience levels
2
Clearly structures the decision criteria for when users choose AI versus search, with concrete participant quotes
3
Provides actionable insight into cognitive costs (keyword foraging, working-memory load, interaction cost) that explain user behavior
• unique insights
1
AI reduces working-memory load by consolidating comparisons that users would otherwise hold in memory or track with notes
2
Users treat genAI as helpful but fallible, using search as a validation mechanism, and citations do not fully resolve trust because aggregation obscures which claim comes from which source
• practical applications
Helps UX designers and product teams understand when AI is the preferred entry point versus when users need trusted, verifiable sources, informing design of AI-assisted search features, comparison tools, and citation/trust cues.
• key topics
1
User decision patterns between generative AI and traditional search
2
Cognitive costs of information seeking (keyword foraging, interaction cost, working-memory load)
3
Trust, validation, and citation challenges with AI-generated answers
• key insights
1
Empirical, task-based evidence of when users prefer AI versus search rather than opinion-based claims
2
Explains the mental model behind users' skepticism toward AI facts, pricing, and citations
3
Connects AI behavior patterns to concrete UX design implications for information-seeking products
• learning outcomes
1
Understand the task conditions under which users prefer generative AI versus traditional search
2
Recognize the cognitive costs (keyword foraging, working-memory load, interaction cost) that AI can reduce
3
Identify trust and validation issues in AI-generated answers and how they shape user behavior
Generative AI has changed how people look for information, but it has not replaced traditional search. A recent observational study of users with varying AI experience found that people deliberately choose between AI chatbots and search engines depending on the task. AI is preferred for exploring possibilities, juggling constraints, and synthesizing information across sources. Traditional search remains the choice when accuracy, control, and trusted sources matter most. Understanding this division helps designers and content strategists build experiences that support both modes of information seeking.
“ Why Users Turn to GenAI for Information Seeking
Users turn to generative AI for several reasons that trace back to the friction of traditional search. The study revealed that AI chatbots reduce the burden of formulating precise queries, aggregate information so users need fewer clicks, and help with tasks involving many interdependent requirements. These strengths make AI a natural first stop for many information-seeking journeys.
“ Starting Points When the Search Space Is Unfamiliar
Many participants began with AI when they did not yet know what to look for. Traditional search requires users to supply specific keywords, which presumes some familiarity with a topic. Chatbots accept broad or exploratory prompts and suggest directions. One user who had just bought a Kindle asked whether free books were available and received several options, including library apps and public-domain sites. He described the value of starting wide and narrowing down afterward. This approach also spares users from keyword foraging, the trial-and-error process of guessing search terms, allowing them to focus on clarifying their goals.
“ Handling Multiple Constraints Simultaneously
Information tasks often involve several constraints at once, such as budget, timing, and location. These are hard to express in a single search query. One participant planning a golf trip for four friends had requirements covering trip length, number of courses, a per-person budget, and proximity to his home. He began with a broad prompt in ChatGPT and then refined it by excluding destinations. He reported that chatbots handle three or four constraints better than conventional search, though he acknowledged that newer AI search features were underused due to low discoverability.
“ Reducing Interaction Cost and Aggregating Information
Participants repeatedly noted that AI saves clicks. Researching a topic through search often means visiting many websites and reading lengthy content. Chatbots aggregate relevant information into a single response, reducing both interaction cost and cognitive load. This is especially valuable for product reviews. One participant asked an AI tool to summarize car reviews rather than reading many technical discussions, wanting a quick answer to whether the vehicle was worth considering. Participants described the experience as talking to a knowledgeable salesperson who did not push them toward a purchase.
“ Lowering Working-Memory Load Through Comparison
Comparing options is cognitively demanding, especially when details are scattered across sites. Without AI, users must hold candidates in working memory or rely on external aids like notes. In a study on day hikes near Tucson, a participant first worked through the National Park Service site and AllTrails, writing down candidates and ratings over about twenty minutes. When she used Gemini for the same task, it returned summarized, sorted results more quickly. Users with higher AI literacy also requested comparison tables in their prompts, such as pros and cons for beaches, allowing them to scan multiple options at once.
“ Why Users Still Rely on Traditional Search
Despite AI's strengths, participants made deliberate choices to return to search. Six of nine participants moved back and forth between the two tools. Their behavior reflects a mental model of AI as helpful but fallible. Users wanted control over the process and confidence in the results, and they treated search as a complementary tool rather than a competitor.
“ Validation and Trust in AI Outputs
Search frequently served as a validation mechanism. The golf-trip planner read ChatGPT's summaries and then searched for each resort's name to confirm details on its official website. Participants also observed AI errors directly. In one case, a chatbot listed a New York resort in response to a Midwest query, prompting the user to remark that this was where the tool began to lose its charm. Such mistakes reinforce the need for verification.
“ Verifying Pricing and Key Facts
Participants were particularly skeptical of AI-provided prices and specific figures. Even when they expected AI to be directionally correct, they wanted to confirm exact numbers through search. Some worried that AI might draw on outdated sources or present misleading starting prices. This pattern shows that users treat AI as a guide for orientation but expect primary sources for precise, consequential data.
“ High-Stakes Decisions and Specialized Research
The likelihood of verifying AI output rises with the cost of error. A participant researching a car purchase wanted higher confidence than AI could reliably provide because the expense was significant, while a smaller purchase carried little risk. Similarly, a user researching omega-3 supplements preferred reading university or government health sites directly. Citations do not fully resolve this issue. Participants found it difficult to tell which claims were supported by a given source, since AI synthesis blends information in ways that obscure attribution. For specialized academic work, participants also avoided chatbots because they may fabricate papers or authors and cannot access paywalled literature, preferring tools such as Google Scholar or research-focused notebook applications.
“ Conclusion: Designing for Complementary Tools
Generative AI and traditional search serve complementary roles. Users rely on AI to explore possibilities, manage multiple constraints, and synthesize information, and they return to search and trusted sources when accuracy, verification, and accountability are essential. Designers and content creators should recognize this division: support AI-driven exploration while making sources transparent and easy to verify, and preserve clear paths to primary information for high-stakes decisions.
We use cookies that are essential for our site to work. To improve our site, we would like to use additional cookies to help us understand how visitors use it, measure traffic to our site from social media platforms and to personalise your experience. Some of the cookies that we use are provided by third parties. To accept all cookies click ‘Accept’. To reject all optional cookies click ‘Reject’.
Comment(0)