Skip to content

Defer the visible content of the view together with the notification (#115) - #125

Open
aetos382 wants to merge 40 commits into
Cysharp:masterfrom
aetos382:issue-115
Open

aetos382 wants to merge 40 commits into
Cysharp:masterfrom
aetos382:issue-115

Conversation

@aetos382

Copy link
Copy Markdown
Contributor

Background

When ToNotifyCollectionChanged() is given an ICollectionEventDispatcher, only the CollectionChanged invocation is deferred to the UI thread; the internal list (listView) is
updated immediately, on the mutating thread.

As a result, when the collection is mutated from a thread other than the UI thread, the content has already moved on by the time the UI thread receives the notification, so the indices carried by the
notification no longer match the content. A consumer that reads the collection right after the notification — such as the WinUI 3 ListView, which calls IBindableVector.GetAt(index) — crashes
there with an out-of-range access (#115).

Changes

1. Defer the visible content together with the notification (the core of #115)

Added Internal/DeferredViewList<TView>. When a dispatcher is specified, the view now holds both the list as the subscribers see it (published) and a queue of not-yet-raised changes (pending):

  • when a change happens, it is enqueued into pending before ICollectionEventDispatcher.Post
  • the change is applied to published at the moment the notification is raised

This keeps the content of the notification and the visible content consistent at all times.
this[int] / Count / IndexOf / Contains / GetEnumerator all read published.

The following are also guaranteed:

  • A dispatcher may raise some events synchronously and others asynchronously; a later event never overtakes the ones already queued (when an event is raised, all older pending changes are applied
    and raised first, in order).
  • Reset does not carry items, so the writer-side content is captured as a snapshot at Post time.
  • If there is no subscriber and nothing is pending, there is no notification to stay consistent with, so the change is applied on the spot. Nothing is enqueued, so a subscriber that attaches
    later does not receive past notifications.
  • After unsubscribing, deferral continues while changes are still pending — switching to immediate application mid-stream would reorder the applications and corrupt the content.
  • If a subscriber throws, the remaining pending changes are still applied and raised, and the exception is rethrown afterwards (AggregateException when there is more than one).
  • CollectionChanged / PropertyChanged became explicit events whose add / remove are guarded by gate, so that "there is no subscriber" can be decided atomically with applying a change.

2. Index translation for position-based writes on writable views

The indexer setter, Insert and RemoveAt of ToWritableNotifyCollectionChanged() passed the view index straight through to the source collection. This is now fixed:

  • On a filtered view, the index is translated back to a source index via AlternateIndexList.GetAlternateIndex (previously it was off by the number of filtered-out elements).
  • While notifications are pending, the visible index is translated to the writer-side index by replaying the pending changes (Add / Remove / Move).
  • An Insert at the tail of the view has no determined source index, so it is treated as AddToSourceCollection.
  • An out-of-range index is rejected with ArgumentOutOfRangeException, whether or not a dispatcher is used.
  • When the target element itself has been removed by a pending change, or when a pending Reset makes any index unresolvable, an InvalidOperationException is thrown, with the two reasons (element removed vs. collection reset) worded distinctly.
  • The setter publishes a Replace so that the visible content is never mutated without a notification.

3. Thread check in SynchronizationContextCollectionEventDispatcher

The condition for raising synchronously was "SynchronizationContext.Current is not null", so in a process with more than one thread that has a SynchronizationContext, a mutation from a UI thread
other than the bound one was raised directly on that thread. The check is now "is Current the same context this dispatcher is bound to".

public static readonly ... Current = current.Value; was also changed to => current.Value.
Previously the Lazy was evaluated during static initialization of the type, so if a thread without a SynchronizationContext touched the type first, the result was a TypeInitializationException.

4. Notification inconsistencies (found while investigating #115)

  • Move on a filtered view raised the notification with the translated indices and then fell through and raised the original one as well — notifying the same move twice, the second time with untranslated indices.
  • When a filter evaluates differently at add time and at remove time, a remove notification arrives for an element that is not in the view. Since listView is unchanged, no notification must be raised. (Enqueuing a notification with an unknown index (-1) into the deferral queue makes it inapplicable at raise time, permanently breaking consistency.)
  • A range remove with OldStartingIndex == -1 raised, for each element, the arguments of the whole batch, and with an uninitialized index of -1. It now raises a single-item notification carrying the position actually removed.

5. Documented the limitation of ToNotifyCollectionChangedSlim()

By design ToNotifyCollectionChangedSlim() shares the data with the source, so its visible content cannot be deferred and the same inconsistency remains. This is accepted as the trade-off for its performance, and is documented instead: the XML doc remarks and the README now state that when it is combined with a dispatcher, the collection must be mutated on the dispatcher thread.

Behavioral changes

  • A view with a dispatcher holds one additional internal list for the visible content (one copy at construction time).
  • With a dispatcher, a change is not reflected in Count or the indexer until the notification is raised — even as seen from the mutating thread. This is intentional; consistency with the notification is the point.
  • Position-based writes on a writable view now throw InvalidOperationException for an index that cannot be resolved while notifications are pending.
  • SynchronizationContextCollectionEventDispatcher.Current changed from a static readonly field to a property, so binary compatibility is not preserved.

Known bugs not fixed in this PR

  • A writable view writes to the source without calling the converter when typeof(T) == typeof(TView), so Add / Insert / Remove ignore the converter even for cases that need conversion between identical types (split out into a separate branch; out of scope here).
  • SynchronizationContextCollectionEventDispatcher.Current is pinned to the SynchronizationContext of whichever thread reads it first, and Lazy<T> caches exceptions too, so if a thread without a context reads it first, it throws InvalidOperationException forever for the rest of the process.
    (Making it a property avoids the static-initialization failure, but the pinning and the cached exception remain.)
  • ToNotifyCollectionChangedSlim() still carries the same inconsistency as described above; only documentation was added.

aetos382 and others added 11 commits August 31, 2026 12:48
When notifications are deferred by ICollectionEventDispatcher, the internal
list was updated immediately on the mutating thread, so a subscriber on the
dispatcher thread observed a state ahead of the notification it received.

- add DeferredViewList<TView>, which holds the list as subscribers see it and
  applies each change at the same time as the notification is raised
- translate visible indexes into writer-side indexes on positional writes
  (setter, Insert, RemoveAt) by replaying the pending changes
- fix SynchronizationContextCollectionEventDispatcher.Post to compare against
  the bound SynchronizationContext instead of testing Current for null
- fix duplicated notification on Move in the filtered view
- document the limitation of ToNotifyCollectionChangedSlim, which shares the
  actual data and therefore can not defer its content

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only the head is ever taken, so a queue expresses the intent and drops the
O(n) List.RemoveAt(0). Draining n pending changes in order becomes O(n) instead
of O(n squared), which matters because the queue grows whenever the mutating
thread runs ahead of the dispatcher thread.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The setter updated the visible list without a notification, so its content
diverged from the notification stream and never recovered. Route the change
through Publish, and notify only when the converter rejects the source write
because otherwise the Replace of the source carries the same content.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The drain loop of InvokeChangedEvent raises an older pending event before the
event it was called for. A subscriber exception escaped the loop and left the
newer events pending, so their content and notification were never delivered
until the next change happened, if any.

Collect the exceptions, finish the drain, then rethrow. A single exception is
rethrown with ExceptionDispatchInfo so the behavior stays as it was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
When the source does not provide the index, the view looks the element up by
value. A filter that is not pure can make the lookup miss, and the resulting
notification carried -1 as its index. Such an event can not be applied to the
deferred list, so it threw inside the gate after being dequeued and broke the
consistency between the visible content and the notifications for good.

The multiple items branch also never updated the index and raised the whole
range once per element.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ToListViewIndex is the only entry point of the indexer setter, Insert and
RemoveAt, so validate the index there before translating it.

Filtable fell back to appending when the translated index was past the end of
listView, which was meant for the insertion point at the tail but also swallowed
an out of range index. Insert(Count + 1, x) appended silently instead of
throwing, with or without a dispatcher.

UntrackableIndex is -1 as well, so an index of -1 passed by the caller was
mistaken for an untrackable one. Insert(-1, x), RemoveAt(-1) and this[-1] = x
threw InvalidOperationException with a message about another thread, only when a
dispatcher was configured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The message blamed another thread for changing the element, but a dispatcher
defers a change of the same thread as well, and the change is never a Replace,
which does not move any element. It is either a remove of the element itself or
a reset of the whole content. Return the reason from ToWriterIndex and tell
which one it is, and what to do about it.

A pending reset makes every index unresolvable, so do not report an index for it
as if another one could work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adding the tests did not add any project to the solution, so its only real
change was the version of Visual Studio that happened to open it. The rest of
the diff was the line endings being rewritten from LF to CRLF.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bumping it to net10.0 has nothing to do with issue Cysharp#115. Restore net6.0 and stop
using a collection expression, which the resulting C# 10 does not have.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@aetos382
aetos382 requested review from neuecc and a balanced review from Copilot August 31, 2026 08:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Defers synchronized-view state alongside dispatched notifications to keep UI-visible content and event indices consistent.

Changes:

  • Adds deferred view state and writable-index translation.
  • Fixes dispatcher thread detection and notification inconsistencies.
  • Adds documentation and extensive regression tests.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
README.md Documents slim-view threading limitations.
src/ObservableCollections/ICollectionEventDispatcher.cs Corrects synchronization-context dispatching.
src/ObservableCollections/Internal/DeferredViewList.cs Implements deferred visible state.
src/ObservableCollections/ObservableList.OptimizeView.cs Documents slim-view behavior.
src/ObservableCollections/SynchronizedViewList.cs Integrates deferral and index translation.
tests/ObservableCollections.Tests/DeferredNotificationConsistencyTest.cs Tests notification-state consistency.
tests/ObservableCollections.Tests/DeferredNotificationTest.cs Covers deferred behavior and writes.
tests/ObservableCollections.Tests/MultipleUiThreadTest.cs Tests multiple UI contexts.
tests/ObservableCollections.Tests/NotifyCollectionChangedContractTracker.cs Adds notification contract validation.
tests/ObservableCollections.Tests/QueuedCollectionEventDispatcher.cs Adds queued dispatcher fixture.
tests/ObservableCollections.Tests/QueuedSynchronizationContext.cs Adds synchronization-context fixture.
tests/ObservableCollections.Tests/TestUiThread.cs Adds simulated UI thread.
tests/ObservableCollections.Tests/ToNotifyCollectionChangedTest.cs Cleans unused imports.
tests/ObservableCollections.Tests/WritableViewIndexTranslationTest.cs Tests writable index translation.
Suppressed comments (3)

src/ObservableCollections/SynchronizedViewList.cs:1167

  • When the converter rejects the source write (setValue == false) and no dispatcher was supplied, this condition skips Publish, so the notify collection changes its visible item without raising CollectionChanged. Emit the synthetic Replace for the inline path as well; Publish already supports it.
                    if (deferred != null)

src/ObservableCollections/SynchronizedViewList.cs:1076

  • Breaking as soon as the callback's own event is reached leaves any newer pending changes untouched. If this callback's subscriber throws, the exception is rethrown and can abort the dispatcher pump before those callbacks run, so remaining pending changes are not applied and raised before rethrow. Drain all changes already pending at callback entry before propagating collected exceptions.
            if (ReferenceEquals(applied, e)) break;

src/ObservableCollections/SynchronizedViewList.cs:1174

  • Suppressing this synthetic notification assumes SetToSourceCollection will always deliver the source Replace. An ObservableList.CollectionChanged subscriber registered before the view can throw before the view handler runs; the setter then throws after listView was changed, and this queued change is applied silently with no source notification to replace it. Ensure the synthetic update is notified or rolled back when the source update fails.
                            IsInvokeCollectionChanged = !setValue, // the Replace of the source carries the same content

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

aetos382 and others added 2 commits August 31, 2026 18:30
The previous commit routed the set of a writable view through Publish, but it
still updated listView first and then suppressed the notification when the
source write was going to carry the same content. Both halves were wrong.

When the write goes through, the parent view recomputes the view from the
selector, so the value put into listView is overwritten by selector(newOriginal)
a moment later. The deferred list applied the change twice, the first time
without a notification. When the source write did not come back - either
SetToSourceCollection throws, or a CollectionChanged subscriber registered
before the view throws so the view never sees the Replace - that first
application was all that was left, and the visible content changed with nothing
to say so. Let the Replace of the source be the only change.

When the converter rejects the write there is no source notification at all, so
the set has to notify by itself. It was guarded by a dispatcher being
configured, so without one the visible content changed silently. That was the
behavior before this branch as well.

Both were pointed out by Copilot on the pull request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
IsPending walked the queue on every TryApplyNext. In the normal case the target
sits at the head and the walk ends on the first comparison, but when a
synchronous notification overtakes the queued ones - the case the drain loop
exists for - the target is at the tail, so draining N changes compared
N + (N-1) + ... + 1 times while holding the gate.

The queue already applies the changes strictly in the order they were enqueued
and no event is enqueued twice, so an increasing sequence per change answers the
same question by itself: an event is pending exactly while its sequence is above
the one applied last. An event that was never enqueued keeps the default 0 and
is never reported as pending, so the ApplyWithoutNotification path is unaffected.

Hoisting the check out of the drain loop instead would not work, because the loop
releases the gate to raise the notification and another callback may dequeue the
target in the meantime.

Pointed out by Copilot on the pull request.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 25, 2026 06:22

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 25, 2026 06:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

aetos382 and others added 2 commits October 28, 2026 20:38
…stance

WPF installs a new DispatcherSynchronizationContext for every dispatcher
operation, so comparing SynchronizationContext.Current with the bound
instance posted changes made on the UI thread itself, and the view showed
stale content right after the change.

The dispatcher now records the thread that the context dispatches to. When
it is created off that thread, it posts a probe to learn the thread, and
posts every event until the probe runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tests/ObservableCollections.Tests/ToNotifyCollectionChangedTest.cs
aetos382 and others added 17 commits October 29, 2026 16:48
A view given a dispatcher keeps each change until its event is invoked, so a
dispatcher that stops invoking, for example after its UI thread has shut down,
makes the view keep every later change for as long as it is alive. State the
contract on ICollectionEventDispatcher and in the README, and ask to dispose the
view before its dispatcher stops.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…en(false)

DeferredViewList.TryApplyNext set its non-nullable out parameter to null!
when the event had been applied already, and its caller tested it for null.
AlternateIndexList.TryGetAtAlternateIndex said MaybeNullWhen(true), the
reverse of what it does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- the lock order is why gate is not held while raising; on the synchronous
  path the caller already holds the source lock before gate
- the visible list is deferred whenever a dispatcher is given, and is updated
  when the notification is raised, not by the Replace of the source
- say which handler reads the old view from the parent
- document how an insertion point is translated across a pending Add or Move
- drop the issue number from the comments in the source, and fix the grammar
  of the remarks of ToNotifyCollectionChangedSlim
- the context is installed first because the constructor learns the bound
  thread, not because of a static initializer
- the tracker reads the target element by element so that it does not depend
  on the CopyTo under test, which is now implemented
- move a comment to the assertion it explains, and name the issue (Cysharp#142) that
  a test refers to

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- TestUiThread.Invoke and Dispose time out, so that a deadlock fails the test
  instead of hanging the whole run, and Invoke on the thread itself throws
  instead of waiting for itself forever
- QueuedSynchronizationContext.CreateCopy keeps the queue; the base one posts
  to the thread pool
- the tracker exposes its recorded actions and violations as read-only lists

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- add the missing summaries, and rewrite the ones that only pointed at another
  test or described the mechanism instead of the guarantee
- name the parameter of CopyToCopiesPublishedContent after what it switches,
  the implementation, as it attaches no filter
- subscribe explicitly where a tracker was created only to have a subscriber,
  so that it no longer reads as a forgotten assertion

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- an insertion point across pending Removes, a range of them included
- the setter, a rejected setter and Insert on a filtered view while pending
- a filter change while pending
- positional writes across pending range changes and a pending Move
- views over a dictionary, a queue, a stack and a ring buffer
- a subscriber of PropertyChanged alone
- Dispose while notifications are pending
- no subscriber exception lost when callbacks run out of order
- concurrent changes from a worker and the UI thread, with and without a
  subscriber coming and going

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n applied meanwhile

A callback that raises older events ahead collects the exceptions that it cannot defer to their own
callbacks. When a nested message loop in a subscriber applies the own event of the callback meanwhile,
the callback returned on finding it applied, and dropped what it had collected. Throw them together with
the exception deferred to the own event instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lock order is not kept when a filter change raises synchronously, because a filter change takes only
the lock of the parent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The concurrent write tests waited 200 ms and took the other writer to have finished by then unless it had
to wait, so a loaded machine could let a missing lock pass. Wait until the other writer has either
completed or is blocked instead, and check that it was blocked.

Also drop a flag of the Insert test that was not needed, and check the view list as the other tests do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t need

- Remove the two tests that make random changes on two threads and hope for contention. Whether they catch
  a defect depends on the interleaving. Unsubscribed_KeepsDeferringWhileNotificationIsPending covers the
  switch between applying at once and queuing deterministically.
- Make the changes directly instead of in Task.Run(...).Wait(). QueuedCollectionEventDispatcher queues a
  change from any thread, and the test thread is not the thread of a TestUiThread, so the changes are
  posted all the same, and a deadlock can no longer hang the run.
- Reproduce ExceptionsAreNotLostWhenCallbacksRunOutOfOrder with a nested message loop on one thread instead
  of a dispatcher outside the single-thread contract.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is bound to the thread that gets it first

Also correct the wording of the README paragraph that introduces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…that repeat the code

The XML documentation of ICollectionEventDispatcher, SynchronizationContextCollectionEventDispatcher and
ToNotifyCollectionChangedSlim said what README already says. Keep it in README only, as the rest of the
public API does, so that the two cannot drift apart. Also drop the comments that only restate a name or
another comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he tests that no longer use a worker thread

The test of an older exception said that a nested message loop runs the newer callback. With a UI thread
dispatcher, the callback that raises ahead holds the lock of the source, so the newer event can only come from a
change that the UI thread makes in the nested loop, and is raised within that change. CompleteOwnCallback said that
the newer event has raised the event already, while it may still be raising it, which is why an exception thrown
after that can not be deferred.

Since 082356f the deferred tests make their changes on the test thread, so describe them by the dispatcher that
posts the changes instead of by a worker thread.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… XML documentation had

4c9fce4 moved the contract to README, but two parts of it did not reach README: a dispatcher raises each
notification only once, and no other thread has the context of SynchronizationContextCollectionEventDispatcher as
its SynchronizationContext.Current. Also stop calling Current itself the default implementation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It makes the same changes under the same conditions as the ui1 side of EachViewNotifiesItsOwnUiThread, and checks
the same things: a dispatcher created on its own thread raises a change on that thread synchronously, and the
content matches the notifications.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aetos382
aetos382 requested a balanced review from Copilot October 30, 2026 10:20

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A filter-change race can update the wrong visible element, and IList.Add returns incorrect insertion indices while notifications are queued in both writable implementations.

2 open issues

Review effort: Balanced

aetos382 and others added 2 commits November 2, 2026 12:36
…e element is not there

The Count - 1 of the base reads the visible Count, which does not include an add whose notification is pending, so
it pointed at another element, and the operations that take an index would have hit that element. The writable
lists now return the index only when the element is visible, and -1 while the add is pending, since the other
operations take an index into the visible content.

Two cases were wrong on master as well: an element that the filter excludes got the index of the last visible
element, and another writer could add between the add and the read of Count. The position is now determined under
the lock of the source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… filtered view

The setter translated the index into a position in listView before calling the converter, and used that position
after it. A filter change takes only the lock of the parent, not the one of the source, so the converter, or another
thread while the converter runs, can rebuild listView in between, and the setter then replaced the element that had
moved into that position. Look the element up again by its source index, which cannot move while the lock of the
source is held, and leave the visible content unchanged when the new filter excludes the element.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed because converter-driven source mutations can overwrite the wrong item, and notification publication can deadlock with concurrent filter resets.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants