Original Summary

Hi, I&#x27;m trying to get much better at using search engines for technical research. If you just want the question, jump straight to <i>QUESTION</i> below. The rest is context for why I&#x27;m asking.<p>---<p>LLMs solve part of the problem today as it helps better information foraging quickly, for instance when what I need is a clear definition. But I have the feeling that there is still much more I could get out of web search if I knew how to use it properly. Serendipity is one obvious example. But I keep running into a problem that makes the whole process very inefficient: context explosion.<p>Take Apache Hadoop.<p>Let&#x27;s say I encounter the term somewhere and I don&#x27;t know what it is.<p>I search for it, land on Wikipedia, and get something like:<p>&quot;Apache Hadoop is a collection of open-source software utilities for reliable, scalable, distributed computing. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model.&quot;<p>Now the problem is that if I don’t already know the field, this definition creates several new questions and so on.<p>What exactly is MapReduce? What does &quot;programming model&quot; mean in this particular context? &quot;Programming model&quot; is the kind of expression where I understand both words, but that doesn&#x27;t necessarily tell me precisely what the author means by putting them together here. Then I end up with a lot of tabs open.<p>I&#x27;m deliberately using a slightly crude example. Obviously I can keep reading the Wikipedia page. There is an Overview section, HDFS and YARN get introduced, the architecture becomes more concrete, and after a while I can probably build a decent picture of what Hadoop is.<p>So my point isn&#x27;t that Wikipedia gives a bad definition, or that the information isn&#x27;t there. It&#x27;s more that there is a difference between eventually reconstructing the answer from all of that and getting something like: &quot;Hadoop is basically a system for turning a cluster of machines into distributed storage plus distributed compute for large datasets.&quot;<p>When I asked ChatGPT about Hadoop, it could give me something at roughly that level immediately. Maybe the answer here is simply that an LLM is better for that particular job. I don&#x27;t know and don&#x27;t care. I&#x27;m more interested in is whether I&#x27;m missing something on the web-search side.<p>---<p>However I&#x27;m not completely new to web search either. I know that different kinds of sources are useful for different things. I know about forums, aggregators, documentation, Stack Overflow, GitHub issues, Hacker News and so on. I also know enough of the technical web ecosystem that a SERP isn&#x27;t completely flat to me anymore: I recognize names like USENIX, CNCF, LWN, good company engineering blogs, vendor documentation, independent technical blogs, and so on, and I often already have some sense of which results are worth opening and which ones raise a red flag. I also know that useful searches are often indirect, and that one good result can lead to a good author, another site, another term, another discussion, etc.<p>And yet I keep running into the same problem: context explosion. So even though I feel like I already have quite a few of the pieces, I don&#x27;t feel that I have a precise method for this.<p>I&#x27;ve also had the opposite experience, where web search was much better than asking ChatGPT. Recently I was looking into some FastAPI questions around how to organize things — when something should be a query, when it should be an operation, how people actually structure that boundary. I ended up finding Stack Overflow discussions that made the issue much clearer to me than the answers I was getting from ChatGPT.<p>So there is obviously a larger question here about how to combine both efficiently, but that isn’t really the question I’m trying to ask in this post, its much narrower. Here it is:<p>---


  • 情报分类:工作与职业机会
  • 分类依据:内容涉及招聘、求职或职业发展
  • 信息来源:Hacker News 新项目
  • 发布时间:2026/9/27 00:26:43