Original Summary

i'm the founder of keylight. while reviewing our own saved answers, we found a bug in what counted as a brand mention one google ai overviews run had six recorded brand matches. four of those names appeared only inside web addresses or link destinations. the visible answer never named them a vendor's blog being used as a source had become a brand appearance in our numbers we changed the matching to exclude the addresses and keep the visible text. that includes link labels. if the label names a brand, a reader can see it, so removing the whole link would create another counting error that's one saved run. it doesn't tell us how often another tool makes this mistake another issue showed up in a small repeat test on 20 september. we asked the same email marketing question ten times per engine. mailchimp appeared in 4 of 10 perplexity answers, 10 of 10 chatgpt answers and 5 of 10 gemini answers a combined score would be about 63%. that hides the difference between the engines. these were automated checks in one evening, with a us setting for perplexity and chatgpt and no country setting available for gemini. they're not a measure of everyone's chats our tracker reports the engines separately, keeps the answers behind the scores and shows sample uncertainty as a margin of error. missing answers are flagged, and the current weekly comparison uses questions answered in both periods if you're building a product around a score, keep a few saved examples where the counting can go wrong. check the result against those examples whenever the rule changes. we could recheck the stored answers after the fix without paying to ask the engines again   submitted by   /u/abhijeetgupta [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SaaS
  • 发布时间：2026/9/28 03:39:10