Original Summary

Disclosure first: i work at schema labs this is our thing and i'm after a number more than signups. the failure i keep hearing about: an agent hits a table it has never seen, the column is called ref_4, and everything downstream is leaning on the header to work out what it is. strip the header and text-to-sql, schema linking and serialised-row embedding all fall over in the same place. ours reads the values instead of the name. returns what each column probably is, whether it resolves to a person including through a join with another table and a confidence band with an explicit unknown rather than a forced guess. the honest bit: benchmark is our own harness, nobody outside has replicated it. We stripped every column name off 20 public openml datasets, mean roc-auc stayed flat at 0.9230, comparable models drop to about 0.86. those same models beat us when the names are clean. what i actually want: take any table with nothing sensitive in it. a public dataset, a demo db, whatever. rename the columns to c1, c2, c3 so the names carry nothing. run it. tell me how many it got right and which ones it missed. I'd rather have your error rate than your signup and the design question i'm stuck on: would you put something like this in front of the llm as the step that reads the table or would you rather hand the llm the rows and let it work it out itself. SchemaLabs website   submitted by   /u/No-Plant-5234 [link]   [comments]

  • 情报分类：商业与市场研究
  • 分类依据：内容涉及商业、投资或市场动态
  • 信息来源：Reddit · SaaS
  • 发布时间：2026/9/15 02:43:40