Skip to content

drm: propagate modeset failures and tighten framebuffer ownership - #125

Merged
FengHeting merged 4 commits into
ViudiraTech:developfrom
FengHeting:fix/drm-kms-core
Nov 2, 2026
Merged

FengHeting merged 4 commits into
ViudiraTech:developfrom
FengHeting:fix/drm-kms-core

Conversation

@FengHeting

Copy link
Copy Markdown
Contributor

Summary

  • mode_set and atomic_disable now return int, and the atomic commit aborts before publishing the new state when either fails. simpledrm and virtgpu report the scanout-programming result.
  • drm_framebuffer_cleanup() reports whether it actually unregistered the framebuffer, so rmfb and the file-release fallback drop the registration reference exactly once. Only the file that created a framebuffer may remove it; kernel-created framebuffers remain removable from the release path, and an unauthorized rmfb fails with -ENOENT instead of tearing down another client's scanout.
  • probe() takes an optional bool *display_owned. The GPU walk stops at the first driver that took over the display, and the framebuffer fallbacks run only when no driver did. virtgpu unregisters when KMS setup fails so simpledrm can take over, and simpledrm reports ownership of its scanout.

This is the groundwork for the virtio scanout fixes and the bochs driver, which rely on the new callback contract and the display-ownership handoff.

Testing

Default configuration builds. With CONFIG_VIRTIO_GPU=y this branch builds together with the stacked drm/virtio branch.

The CRTC helpers returned void, so a driver could not tell the atomic
commit that programming the scanout failed: the commit still published
the new state and freed the old framebuffer even though the hardware
stayed on the old mode.  Make mode_set and atomic_disable return an int
and abort the commit before the state is published when either fails.

The atomic commit callers already free the state on error, so the failure
path leaves neither a leaked state nor a half-applied commit.  simpledrm
and virtgpu now return the scanout-programming result.

Signed-off-by: MicroFish <2609948707@qq.com>
drm_framebuffer_cleanup() can be called on a framebuffer that was already
unregistered by another path, but callers still dropped the registration
reference.  Return whether this call actually unregistered the object and
let drm_mode_rmfb() and the file-release fallback drop that reference
exactly once.

Only the file that created a framebuffer may remove it; kernel-created
framebuffers have no owning file and remain removable from the release
path.  An unauthorised rmfb now fails with -ENOENT instead of tearing
down another client's scanout.

virtgpu's resource creation error paths use drm_gem_object_put() so the
object goes through the normal refcount instead of the driver free hook.

Signed-off-by: MicroFish <2609948707@qq.com>
A zero probe return only says that something attached; it does not say
whether the driver took over the display.  The bus therefore kept
probing after a display-owning driver and a second driver could take
the global console away from it, and a driver that attached render-only
still suppressed the framebuffer fallbacks.

Give probe an optional out-parameter reporting display ownership, stop
the hardware walk at the first owner, and run the framebuffer fallbacks
only when no driver took the display.  virtgpu now unregisters when KMS
setup fails and reports ownership after registering, so the walk can
move on to simpledrm instead of keeping a render-only device in front
of it.

Signed-off-by: MicroFish <2609948707@qq.com>
@FengHeting
FengHeting merged commit d0b5148 into ViudiraTech:develop Nov 2, 2026
2 checks passed
@FengHeting
FengHeting deleted the fix/drm-kms-core branch November 2, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant