FEATURE: Respect globalScope and type, and find assets by metadata

[bwaidelich]

Makes the property declarations do something — globalScope and type were both parsed and then read by nothing — collapses the read API, and adds a way to find assets by their metadata.

Respect globalScope

MetaDataPropertyDefinition::$globalScope was parsed from the settings and then read by nothing. Properties declared with globalScope: true — copyright, out of the box — were localized like any other, so the same asset could end up with a different copyright notice per language.

Such properties now have exactly one value, shared by all dimensions. The dimension space point passed to a read or write is ignored for them, so callers never have to branch on globalScope: the UI can pass whatever dimension it is currently working in.

One read instead of four

The manager had four ways to read values, which existed because the UI needs three different answers:

Need	Previously
Edit field, without shine-through from the fallback	getMetaDataPropertyValues()
The "translate this" hint below that field	getMetaDataPropertyValuesOfParentWithFallback()
What a visitor sees	getMetaDataPropertyValuesWithFallback()

Those are three views of one answer rather than three questions — the third is just the first falling back to the second. They are now fields on the returned MetaDataPropertyValue:

$value = $metaDataManager->getMetaDataPropertyValue($assetReference, 'caption', $german);

$value->ownValue;        // 'Eine Katze', or NULL if only a fallback exists
$value->inheritedValue;  // 'A cat'
$value->inheritedFrom;   // the dimension space point it stems from
$value->value;           // ownValue ?? inheritedValue

Two side effects worth having:

• Provenance. inheritedFrom tells the UI which dimension a hint comes from, so it can render “EN: A cat”. The old API returned bare values with no way to find out.
• Half the queries. Own and inherited now come from the same query, so an inspector needs one query per property instead of two.

For a property with a global scope the value is shared and therefore never inherited: ownValue is that value and inheritedValue is always NULL, which makes the inspector render it as a plain field with no hint line without special-casing anything.

Respect type

type was the second half of the configuration that was parsed and then ignored — values were written and read exactly as provided, so an integer property held whatever a caller happened to pass and always read back as a string.

It is now enforced in both directions: on the way in, so that nothing but a value of that type is ever stored, and on the way out, so that MetaDataPropertyValue::$value means what its string|int|bool|null signature says.

Unambiguous conversions are applied, so that callers which only ever have strings — the command line, form input, Fusion — do not have to cast:

Type	Accepted	Stored as
string	anything	as provided
integer	an int, an optionally signed decimal string, or a boolean	decimal
boolean	a bool, "true"/"on"/"yes"/"1" and their negatives (any case), or 1/0	1 or 0

Anything else is rejected with an InvalidArgumentException rather than silently turned into a wrong value — "abc" is not 0.

Reading is deliberately more forgiving, because it meets values written before a property was given its current type: a value that cannot be interpreted reads as NULL. Such a value is skipped rather than treated as empty, so it does not shadow a fallback that is still readable.

Booleans are stored as 1/0 and integers in decimal, which is what was already being written, so existing values stay readable.

Finding assets

New: findAssets() returns the references of all assets that have a matching metadata value. Every criterion of a MetaDataAssetFilter is optional and they are combined with AND:

$filter = MetaDataAssetFilter::create(
    searchTerm: 'cat',
    dimensionSpacePoint: MetaDataDimensionSpacePoint::fromCoordinates(['language' => 'de']),
    propertyNames: MetaDataPropertyNames::create('caption', 'altText'),
);

foreach ($metaDataManager->findAssets($filter) as $assetReference) { … }

Criterion omitted	Means
assetSourceId	assets of every asset source
dimensionSpacePoint	the default dimension space point — as everywhere else in this package, not "any dimension"
searchTerm	every asset that has a value for the filtered properties at all
propertyNames	all defined properties

The interesting decision is what "matching" means. A value only counts if it is the one getMetaDataPropertyValue() would return for the filter's dimension space point, so the search agrees with what an editor working in that dimension sees. Given an asset whose English caption is A cat, searching for cat in German finds it as long as German inherits that caption — and stops finding it the moment a German caption of its own is set. The cheaper alternative (OR over the whole chain) would keep matching a value that German never actually shows.

Properties with a global scope are matched on their shared value regardless of the dimension space point, just as they are read regardless of it.

The result is lazily streamed, contains each asset at most once and is ordered by asset source id and asset id. It carries MetaDataAssetReferences — the identity of an asset within its asset source — not Asset objects; resolving those is up to the caller, this package never touches the asset model. There is deliberately no Media Browser wiring here and no pagination; a consumer intersects this with its own listing.

AssetMetaData.getMetaDataProperty()

The Eel helper could only fetch all properties. It now also reads a single one, which is what Fusion usually wants:

caption = ${AssetMetaData.getMetaDataProperty(asset, 'caption', {language: 'de'})}

Storage stays dumb

The adapter used to pick the winner of a read itself, via ORDER BY FIELD(dimension_hash, …) plus fetchOne(). "First match along the fallback chain wins" is a domain rule, so it moved to the manager: storage now returns every value of the requested scope, unordered, and the manager applies fallback priority.

findAssets() is the one place where that could not hold. Resolving precedence per asset in PHP would mean a query per candidate, so the ranking has to be applied inside the query — via a NOT EXISTS anti-join that looks for a stored value closer along the chain. The manager therefore hands the storage the chain ordered, and the storage applies a ranking it was given rather than deriving one. The interface says so per method: getMetaDataPropertyValues() keeps "the order is meaningless here", the search method states the opposite explicitly.

So FIELD() is gone from the read path but back in the search, alongside <=> and the existing ON DUPLICATE KEY UPDATE. The adapter stays deliberately MySQL-specific; the matching semantics of the search are those of utf8mb4_unicode_ci (case- and accent-insensitive, verified against MySQL 8.0 rather than assumed).

MetaDataStorage gained findAssets(). Listing and removing values regardless of scope, which only the repair command needs, lives in a separate optional MetaDataStorageMaintenance; a storage that does not implement it still works and the command says so.

Storage format and the scope invariant

No schema change, and none for the search either — LIKE '%…%' cannot use an index, but the table holds at most (assets × properties × dimensions) rows. If that ever bites, a FULLTEXT index is the way out and nothing in the public API would have to change.

Shared values are stored under the reserved dimension hash global — a real hash is always 32 hex characters, so the two cannot collide.

For a given asset and property the table therefore holds either one shared value or one value per dimension space point, never both. That cannot be expressed as a constraint (it is a cross-row invariant), but it does not need to be: every read looks up the scope a property is configured for, so values of the other shape are simply unreachable. Flipping globalScope can never make stale values surface — and the search honours the same rule.

assetmetadata:repair

Since the shape of stored data is derived from mutable configuration, flipping globalScope leaves values behind that no longer match. They are invisible rather than harmful, so this is hygiene:

./flow assetmetadata:repair                    # report only
./flow assetmetadata:repair --force            # apply
./flow assetmetadata:repair --force --prune    # also drop unreachable values

• Became global: the value the default dimension resolves to is kept as the shared one, the rest are removed.
• Became localized: the shared value is stored for the default dimension space point.
• Existing values are never overwritten — if a shared value is already there, it wins over stale localized ones.
• Values of dimensions or properties that are no longer configured are unreachable rather than contradictory, so they are only reported until --prune is given. Pruning is refused outright while no content dimension is configured, because a broken or half-loaded dimension configuration would otherwise look exactly like every value being obsolete.

This also cleans up after assetmetadatamigration:migrateexistingassetproperties, which wrote copyright to the default dimension while copyright ships as globalScope: true. The command itself needs no change — it routes through the manager and now lands on the shared value — but installations that already ran it are in the mixed state repair fixes.

Breaking changes

// removed
getMetaDataPropertyValuesWithFallback()
getMetaDataPropertyValuesOfParentWithFallback()

// changed
getMetaDataPropertyValue($ref, $name, ?MetaDataDimensionSpacePoint = null): MetaDataPropertyValue
getMetaDataPropertyValues($ref, ?MetaDataDimensionSpacePoint = null): MetaDataPropertyValues

// added
findAssets(MetaDataAssetFilter): iterable<MetaDataAssetReference>

MetaDataPropertyValues now maps a property name to a MetaDataPropertyValue instead of a scalar; ->toArray() gives the old shape. No consumer of any of this exists yet.

MetaDataStorage changed twice and third-party implementations have to follow: the read method returns all values of a scope instead of picking one, and findAssets() was added. Putting the search behind another optional capability interface was considered and rejected — searching is core, not an extra.

AssetMetaData.getMetaData() is unchanged for Fusion — it still returns a flat array of effective values.

Three behaviour changes to be aware of:

• assetmetadata:list without --dimension-space-point now applies fallbacks (it previously did not) and marks inherited values.
• AssetMetaData.getMetaData($asset) without coordinates used to throw in any dimensioned setup: it passed empty coordinates, which isDimensionSpacePointValid() rejects whenever dimensions are configured. Empty coordinates now mean the default dimension.
• Writing a value that does not match a property's declared type now throws instead of storing it. No shipped property is anything but string, and string accepts anything, so no default configuration can hit this.

Tests

The package had none, so this adds require-dev, autoload-dev and both suites. They are split along the seams the package is built on: one set for the MetaDataManager with its dependencies as test doubles, and one per implementation of those dependencies.

Test	Subject
MetaDataManagerTest	The resolution rules, storage and dimension provider mocked
MetaDataRepairTest	Which stored values contradict the configuration and what is done about them
MetaDataPropertyTypeTest	Coercing values to a declared type and back
MetaDataStorageProviderDbalAdapterTest	The MetaDataStorage implementation — functional
DimensionSpacePointProviderContentRepositoryAdapterTest	The DimensionSpacePointProvider implementation
MetaDataConfigurationProviderYamlAdapterTest	The MetaDataConfigurationProvider implementation

110 unit tests, 215 assertions. They state the stored values they resolve from rather than writing them first, so a test says what a rule is instead of demonstrating it through a round trip. Repair records its calls in order, which lets it assert outright that promotions happen before the deletions of the rows they came from.

./bin/phpunit -c Build/BuildEssentials/PhpUnit/UnitTests.xml --filter 'Neos\\MetaData'

30 functional tests, 53 assertions — the SQL that unit tests cannot reach: the upsert, lookup by scope, the shadowing anti-join, LIKE escaping, distinctness, ordering, and the MetaDataStorageMaintenance surface repair is built on. They are the only tests that need a database.

They still do not run in this distribution. The functional bootstrap fails before loading any test, because Flowpack\Media\Ui\Service\UsageDetailsService requires a ContentRepositoryRegistry that does not exist in Neos 8.4. It fails identically with a filter matching no test at all, so it is unrelated to this change. They were verified instead against MySQL 8.0 in a throwaway container — all 30 pass — but CI here has never executed them.

Not covered: the repair command controller's output rendering. The repair logic itself is fully unit-tested through MetaDataRepair.

Review notes

• MetaDataManager::resolvePropertyValue() is where the read rules come together, and MetaDataStorageProviderDbalAdapter::findAssets() is where the search does — worth reading first.
• TASK: Cover enumeration of dimension space points by preset value adds regression tests for 7d4c8f1 on the base branch, which had none. They live here because the base branch has no test infrastructure at all; if that fix is merged independently, the tests do not travel with it.
• Deferred: getMetaDataPropertyValues() still issues one query per property. It is pre-existing, unchanged by this PR, and worth an optional bulk-read interface once a consumer (a Media UI listing, say) actually feels it.

Draft, because based on #11