Skip to content

fixed bug - #2116

Open
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:Bug_fix_validator_goldens
Open

fixed bug#2116
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:Bug_fix_validator_goldens

Conversation

@niveditasing

@niveditasing niveditasing commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

This PR introduces a normalization step for properties that store list-like string values (e.g., "[A, B]" vs "[B, A]") before performing validation/comparison between input_nodes and golden_nodes.

Why is this change necessary?
During schema validation, certain fields represent multi-valued properties serialized as string-based lists (for example, "[AdministrativeArea, Country]"). The ordering of these elements can vary depending on upstream parsing logic, data source extraction, or database insertion sequence.

Currently, comparing these fields using strict string equality causes tests and validation checks to fail if the elements are ordered differently (e.g., "[B, A]" vs "[A, B]"), even though they are semantically identical.
example - https://storage.mtls.cloud.google.com/datcom-prod-imports/statvar_imports/us_bls/bls_ces_state/BLS_CES_State/2026_07_27T04_54_33_595590_07_00/input0/validation/validation_output.csv

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a normalization step for list-like string values in validator_goldens.py to ensure that element sequence does not affect the comparison between input and golden nodes. Feedback suggests removing the condition if ',' in v or v == '[]' to allow single-element lists with inconsistent spacing (e.g., "[ A ]" vs "[A]") to be consistently normalized as well.

Comment thread tools/import_validation/validator_goldens.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants