Skip to content

An attribute's value is read in its datatype and unit, and a table cell or a figure written as an object is a value - #1252

Merged
WaylandYang merged 4 commits into
devfrom
fix/attribute-values
Oct 16, 2026
Merged

WaylandYang merged 4 commits into
devfrom
fix/attribute-values

Conversation

@WaylandYang

@WaylandYang WaylandYang commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Defining an attribute and extracting a two-paragraph company profile gave wrong values for the plainest facts. Measured on a fresh base (DeepSeek chat, bge-m3 embeddings), eight attributes on 公司 / 人物 (注册资本 number in 万元, 员工人数 number, 成立日期 date, 出生日期 date, 是否上市 bool, …), two documents:

before after
星河科技 注册资本 (“注册资本为5000万元”) nothing: 5000万元 became an entity 5000
星河科技 成立日期 (“成立于2015年3月12日”) nothing: the date became an entity 2015-03-12
星河科技 员工人数 (“员工人数为1200人”) nothing 1200
张伟 出生日期 (“出生于1978年5月”) nothing 1978-05
蓝山生物 注册资本 (“注册资本2.3亿元”, unit 万元) "2.3亿元" 23000 (store test; the second run's reply did not state it)
蓝山生物 员工人数 (“员工约350名”) "约350名" 350
李娜 出生日期 (“生于1982年”) "1982年" 1982
蓝山生物 是否上市 (“尚未上市”) "尚未上市" not written; the statement stays open

Three causes; the first two were left behind when extraction moved to the open graph (#736):

  1. A figure or a date written as the object. The same model and prompt put “成立于 2015年3月12日” in the value slot in one document and listed the date in e and wrote it as the object in the other. The signature is then a relation between two things, object_is_value is false, and no attribute can ever bind it; the document also gains entities named 2015年3月12日 and 5000万元. open::values_are_not_things now moves an object that is wholly a quantity (magnitude word, currency, percent or counter: 5000万元, 1200人, $5 billion) or a readable date (2015年3月12日, 2020年, 2015-03-12) into the value slot, and drops it from e when nothing else refers to it. A bare number moves only when the entity was not listed as named, since a named 1984 or 360 is more likely a book or a company. It runs on model replies only: pushed statements are the client's contract (0054) and are not rewritten. Each move is recorded as object_is_value in extraction_drops, so it shows how often the prompt rules still fail.

  2. Materialisation copied the document's words. normalize_attr_value was only called by the RDF tests after Extraction reads only the open graph #736, so a number attribute held "2.3亿元" and a date "1982年", which the export then wrote as "2.3亿元"^^xsd:decimal. Materialisation now reads the value by the property's datatype and unit (attribute_value): a magnitude is converted into the declared unit (2.3亿元 under 万元 → 23000; a bare 5000 is taken to be in the declared unit), a different currency is refused rather than converted, 约 / 左右 are dropped, YYYY年 is read at year precision. A value that cannot be read is not written; the statement keeps the document's words in the open graph. The same applies to rows an implication rule concludes. This is done in materialisation rather than at insert_fact_on: people and errata write relative dates ("relative": true) onto date attributes, which the gate would refuse.

  3. Tables ([Help]Extracted facts/triples consistently missing temporal information (timestamps/dates) #987). A requirements sheet shaped like the one in [Help]Extracted facts/triples consistently missing temporal information (timestamps/dates) #987 (16 columns, one row per section, attributes created as the UI creates them: state, single-valued) kept 9 and then 5 of the 17 values its rows state. The table rule said a cell is a statement whose phrase is its column heading, but not that the cell is the value, nor which column names the row's thing. So the model wrote cells as things (["待审核", "处理状态", 0] as the object of 处理状态), made the sheet name the subject of every cell, and once wrote [需求记录, 处理状态, 应用无法访问网络, 处理中], whose value the write path then drops because the object wins. The table rule now says a cell is the value unless it names a thing of its own, and that a row is about what its first (or name, title, ID) column names, not the table or sheet. For a statement quoted from a table row, the same pass also moves an object back to the value slot when the model's own kind word for it equals the statement's phrase and it is unnamed, or named but carries a digit (REQ-0019, listed as named with kind 需求编号), and drops it from e so no orphan entity is created. Two runs after the change: 17 of 17 in both.

utopia-store now depends on utopia-extract for these readers (no cycle; extract depends only on llm).

Not changed, and worth a decision:

  • “2023年实现营业收入8.6亿元” is bound to a state attribute with marks = none, so 0053's revision for A statement dated at one moment, bound to a state property, becomes a state that holds at no moment #966 writes no row. For an attribute observed over a year this loses the value; one option is to read a period-precision date under none as holding through that period.
  • “总部位于杭州”, “是首席科学家”: the extractor rightly treats 杭州 and 首席科学家 as things, so the signature has an entity object and cannot bind a text attribute.
  • Rows materialised before this keep their raw strings until their statements are recomputed.
  • An unreadable value is skipped, not recorded, so the due query selects it again on each materialise run. That is cheap while such rows are few; recording the skip like MOMENT_UNREAD would stop it.
  • The pure readers (scan_quantity, written_date, parse_time, attribute_value) would sit better in utopia-core than behind a store → extract dependency; moving them is a separate change.

Tests: unit tests for the readers and the parser move (a_figure_or_a_date_is_not_a_thing, plain_company_figures_are_read, a_value_written_as_an_object_becomes_the_value, a_table_cell_listed_as_a_thing_becomes_the_value); a store test that materialises 2.3亿元 / 2008年11月 / 5000万美元 against a real PostgreSQL. fmt and clippy pass. cargo test --workspace --no-fail-fast with UTOPIA_DATABASE_URL passes except seven pool, lock and worker-timing tests (review_locks::* ×3, concurrent_exports_on_a_minimum_pool…, the_embedding_gate_is_never_held…, a_queued_job_wakes_an_idle_worker, a_build_finishes_with_only_one_connection_available), plus closing_the_pool_cancels_writeback…, which passes on its own, that fail the same way on unmodified dev on the Windows machine this ran on; CI will tell.

🤖 Generated with Claude Code

WaylandYang and others added 2 commits October 16, 2026 11:17
…ritten as an object is a value

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
…st column names

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
@WaylandYang WaylandYang changed the title An attribute's value is read in its datatype and unit, and a figure written as an object is a value An attribute's value is read in its datatype and unit, and a table cell or a figure written as an object is a value Oct 16, 2026
WaylandYang and others added 2 commits October 16, 2026 20:32
…out of the shared parser

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: WaylandYang <wayland0916@gmail.com>
@WaylandYang
WaylandYang merged commit 7393e9d into dev Oct 16, 2026
6 checks passed
@WaylandYang
WaylandYang deleted the fix/attribute-values branch October 16, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant