Skip to content

rfbserver: send the ExtDesktopSize pseudo-rect only once per client - #743

Open
slab-tsuchiya wants to merge 1 commit into
LibVNC:masterfrom
slab-tsuchiya:fix-extdesktopsize-once
Open

rfbserver: send the ExtDesktopSize pseudo-rect only once per client#743
slab-tsuchiya wants to merge 1 commit into
LibVNC:masterfrom
slab-tsuchiya:fix-extdesktopsize-once

Conversation

@slab-tsuchiya

Copy link
Copy Markdown
Contributor

Problem

When a client enables the ExtDesktopSize pseudo-encoding, every non-incremental
FramebufferUpdateRequest sets cl->newFBSizePending, and a pending new-FB-size makes
rfbSendFramebufferUpdate() emit only an ExtDesktopSize rectangle and return without
sending any pixels.

Most viewers send an incremental request after the first full one, so they receive the
size once and then get pixels. But a viewer that requests a full update again after
each ExtDesktopSize — notably UltraVNC Viewer in its "new ultra server" mode — never
converges: it loops forever receiving only size rectangles, and the screen never updates.

Reproduce

Connect an UltraVNC Viewer to a libvncserver-based server that has ExtDesktopSize enabled
(the viewer advertises the pseudo-encoding automatically). The server log fills with

Sending rfbEncodingExtDesktopSize for size (1024x768)
Sending rfbEncodingExtDesktopSize for size (1024x768)
... (every few ms, indefinitely)

and the framebuffer is never painted.

Fix

Track whether the initial ExtDesktopSize has been sent to a client
(cl->extDesktopSizeSent) and only re-assert it from a full-screen request until then.

Genuine resizes are unaffected: they set newFBSizePending through rfbNewFramebuffer(),
not through this per-request path, and the send path re-arms the flag. Viewers that only
ever needed the size once behave exactly as before.

Small and self-contained: one new client flag plus a guard. Verified against a real
UltraVNC Viewer (screen now paints normally) and with a protocol probe that mimics the
"request a full update after every ExtDesktopSize" behaviour: the size rectangle is sent
exactly once, then pixels follow.

Every non-incremental FramebufferUpdateRequest from an ExtDesktopSize-capable
client set newFBSizePending, and a pending new-FB-size makes the server emit
an ExtDesktopSize rectangle and return without sending any pixels. Viewers
that ask for a full update again after receiving that rectangle -- notably
UltraVNC in its "new ultra server" mode, which sends non-incremental requests
repeatedly -- therefore loop forever receiving only size rectangles, and the
screen never updates.

Track whether the initial ExtDesktopSize has been sent to a client and only
re-assert it from a full-screen request until then. Genuine resizes still
re-send it: they go through rfbNewFramebuffer(), which sets newFBSizePending
directly, and the send path re-arms the flag. Other viewers are unaffected
(they only ever needed the size once).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant