Repository navigation
An attribute's value is read in its datatype and unit, and a table cell or a figure written as an object is a value - #1252
Merged
Merged
Conversation
…ritten as an object is a value Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: WaylandYang <wayland0916@gmail.com>
…st column names Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: WaylandYang <wayland0916@gmail.com>
…out of the shared parser Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: WaylandYang <wayland0916@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defining an attribute and extracting a two-paragraph company profile gave wrong values for the plainest facts. Measured on a fresh base (DeepSeek chat, bge-m3 embeddings), eight attributes on 公司 / 人物 (
注册资本number in 万元,员工人数number,成立日期date,出生日期date,是否上市bool, …), two documents:5000万元became an entity50002015-03-1212001978-05"2.3亿元"23000(store test; the second run's reply did not state it)"约350名"350"1982年"1982"尚未上市"Three causes; the first two were left behind when extraction moved to the open graph (#736):
A figure or a date written as the object. The same model and prompt put “成立于 2015年3月12日” in the value slot in one document and listed the date in
eand wrote it as the object in the other. The signature is then a relation between two things,object_is_valueis false, and no attribute can ever bind it; the document also gains entities named2015年3月12日and5000万元.open::values_are_not_thingsnow moves an object that is wholly a quantity (magnitude word, currency, percent or counter:5000万元,1200人,$5 billion) or a readable date (2015年3月12日,2020年,2015-03-12) into the value slot, and drops it fromewhen nothing else refers to it. A bare number moves only when the entity was not listed as named, since a named1984or360is more likely a book or a company. It runs on model replies only: pushed statements are the client's contract (0054) and are not rewritten. Each move is recorded asobject_is_valueinextraction_drops, so it shows how often the prompt rules still fail.Materialisation copied the document's words.
normalize_attr_valuewas only called by the RDF tests after Extraction reads only the open graph #736, so a number attribute held"2.3亿元"and a date"1982年", which the export then wrote as"2.3亿元"^^xsd:decimal. Materialisation now reads the value by the property's datatype and unit (attribute_value): a magnitude is converted into the declared unit (2.3亿元under 万元 → 23000; a bare5000is taken to be in the declared unit), a different currency is refused rather than converted,约/左右are dropped,YYYY年is read at year precision. A value that cannot be read is not written; the statement keeps the document's words in the open graph. The same applies to rows an implication rule concludes. This is done in materialisation rather than atinsert_fact_on: people and errata write relative dates ("relative": true) onto date attributes, which the gate would refuse.Tables ([Help]Extracted facts/triples consistently missing temporal information (timestamps/dates) #987). A requirements sheet shaped like the one in [Help]Extracted facts/triples consistently missing temporal information (timestamps/dates) #987 (16 columns, one row per section, attributes created as the UI creates them:
state, single-valued) kept 9 and then 5 of the 17 values its rows state. The table rule said a cell is a statement whose phrase is its column heading, but not that the cell is the value, nor which column names the row's thing. So the model wrote cells as things (["待审核", "处理状态", 0]as the object of处理状态), made the sheet name the subject of every cell, and once wrote[需求记录, 处理状态, 应用无法访问网络, 处理中], whose value the write path then drops because the object wins. The table rule now says a cell is the value unless it names a thing of its own, and that a row is about what its first (or name, title, ID) column names, not the table or sheet. For a statement quoted from a table row, the same pass also moves an object back to the value slot when the model's own kind word for it equals the statement's phrase and it is unnamed, or named but carries a digit (REQ-0019, listed as named with kind需求编号), and drops it fromeso no orphan entity is created. Two runs after the change: 17 of 17 in both.utopia-storenow depends onutopia-extractfor these readers (no cycle; extract depends only on llm).Not changed, and worth a decision:
marks = none, so 0053's revision for A statement dated at one moment, bound to a state property, becomes a state that holds at no moment #966 writes no row. For an attribute observed over a year this loses the value; one option is to read a period-precision date undernoneas holding through that period.MOMENT_UNREADwould stop it.scan_quantity,written_date,parse_time,attribute_value) would sit better inutopia-corethan behind a store → extract dependency; moving them is a separate change.Tests: unit tests for the readers and the parser move (
a_figure_or_a_date_is_not_a_thing,plain_company_figures_are_read,a_value_written_as_an_object_becomes_the_value,a_table_cell_listed_as_a_thing_becomes_the_value); a store test that materialises 2.3亿元 / 2008年11月 / 5000万美元 against a real PostgreSQL. fmt and clippy pass.cargo test --workspace --no-fail-fastwithUTOPIA_DATABASE_URLpasses except seven pool, lock and worker-timing tests (review_locks::*×3,concurrent_exports_on_a_minimum_pool…,the_embedding_gate_is_never_held…,a_queued_job_wakes_an_idle_worker,a_build_finishes_with_only_one_connection_available), plusclosing_the_pool_cancels_writeback…, which passes on its own, that fail the same way on unmodifieddevon the Windows machine this ran on; CI will tell.🤖 Generated with Claude Code