libs-back is missing support for on-screen rendering

[triplef]

#31 added support for a "headless" GUI backend, which @qmfrederik explained here:

The headless backend allows you to use gui components without having them actually render anything to your screen. In our case, we have an application which runs both in command line as well as full GUI mode. Even when running in command line mode, some code from libs-gui is invoked. The headless backend allows us to run those applications.

There may be other use cases, too. For example, I think this would allow you to run code which relies on libs-gui in e.g. a container environment, for, say, unit testing purposes as part of your CI pipeline.

We should add support for a full Win32 backend to allow on-screen rendering.

I think both @gcasa and @qmfrederik have worked on this in the past. I wanted to open this in order to document the current state of GUI support.

[arctic-marmoset]

Out of curiosity, I tried compiling gnustep-back for Win32, but ultimately ran into several linker errors about missing symbols for __objc_ivar_offsets from various gnustep-gui classes (gnustep-back-link-log.txt). Specifically, classes that have ivars declared in their interfaces, and from which the Win32 backend implements derived classes.

However, dumpbin /exports gnustep-gui.dll shows that those symbols are being exported (gnustep-gui-exports.txt).

Not sure where to go from here. Let me know if you need more info.

Clang version:

> clang --version
clang version 18.1.8
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

These are the changes I made to get to this point: changes.patch

[triplef]

@arctic-marmoset I believe this linker issue was fixed with llvm/llvm-project#77797, but I’m not sure if that patch made it into Clang 18. @qmfrederik do you know?

[qmfrederik]

I'm pretty sure it did, and it works on MSYS2. @arctic-marmoset are you sure you using the LLVM linker? Are you using MSYS2 for building?

[triplef]

Thanks. I believe @arctic-marmoset is trying to build for MSVC, not MinGW (if that was the question), and they are using lld-link:

lld-link: error: undefined symbol: __objc_ivar_offset_NSView._rFlags.{_rFlagsType☻b0I1b1I1b2I1b3I1b4I1b5I1b6I1b7I1b8I1b9I1}
>>> referenced by C:\Users\dev\tools-windows-msvc\src\gnustep-back\Source\win32\WIN32Server.m:2834
>>>               ./win32/./obj/win32.obj/WIN32Server.m.o:(process_mouse_event)

lld-link: error: undefined symbol: __objc_ivar_offset_NSView._cursor_rects.@
>>> referenced by C:\Users\dev\tools-windows-msvc\src\gnustep-back\Source\win32\WIN32Server.m:2836
>>>               ./win32/./obj/win32.obj/WIN32Server.m.o:(process_mouse_event)
...

Do you know if these errors should be fixed by the LLVM patch?

[qmfrederik]

This looks like it's related to ivar visibility, which should have been fixed via llvm/llvm-project#77385 (and it appears it has, if dumpbin lists the symbols)? @arctic-marmoset Could you try this on msys2/mingw64 first; the code (used to) compile on that platform.

[triplef]

Looks like for MinGW the changes were backported (msys2/MINGW-packages#19767).

@arctic-marmoset you could also try the latest Clang 19 release candidate, which should for sure have these fixes.

[qmfrederik]

@triplef They were backported but all backports were removed from msys2 once clang 18 made it into msys2.

[qmfrederik]

I ran some tests. With LLVM 18.1.8, compilation works just fine in an MSYS2 environment, but fails when you use a Windows-native toolchain.

I can't really explain the difference just yet; I assume that somehow the linker behaves different in both environments?

[qmfrederik]

FWIW, the same linking issues is present in libs-gui, too, but is sidestepped via this patch: https://github.com/gnustep/tools-windows-msvc/blob/master/patches/gnustep-gui-remove-files.patch

[qmfrederik]

... looks like the difference is that MSYS2 appends -Wl,--whole-archive to the linker command, which seems to help mitigate this. Need to dive a bit further, though.

[qmfrederik]

@arctic-marmoset So, it looks like there's an issue with the codegen for ivar references on Windows. They appear to be marked as extern but are missing the __declspec( dllimport ) declarator (to be confirmed).

It works on MinGW/MSYS2 because lld behaves slightly different on that platform -- the -lldmingw flag is enabled, which in turn enables the -auto-import and -runtime-pseudo-reloc linker flags, masquerading that problem.

Can you try to compile libs-back with the -auto-import and -runtime-pseudo-reloc linker flags and let me know if that helps? For example, you should be able to pass LDFLAGS='-Wl,-auto-import -Wl,-runtime-pseudo-reloc -fuse-ld=lld to ./configure.

[arctic-marmoset]

@qmfrederik I gave your suggestion a shot, and it seems to have resolved all the undefined symbol errors, except for one:

lld-link: error: undefined symbol: __objc_ivar_offset_NSView._cursor_rects.@
>>> referenced by C:\Users\dev\repos\extern\tools-windows-msvc\src\gnustep-back\Source\win32\WIN32Server.m:2836
>>>               ./win32/./obj/win32.obj/WIN32Server.m.o:(process_mouse_event)

If I understand correctly, this is actually because _cursor_rects has @package visibility.

If I make it @public, then that error is resolved.

Unfortunately, I'm now running into the following error:

lld-link: warning: runtime pseudo relocation in ./win32/./obj/win32.obj/WIN32Server.m.o against symbol __objc_ivar_offset_NSView._rFlags.{_rFlagsType☻b0I1b1I1b2I1b3I1b4I1b5I1b6I1b7I1b8I1b9I1} is too narrow (only 32 bits wide); this can fail at runtime depending on memory layout
...
lld-link: error: output image has runtime pseudo relocations, but the function _pei386_runtime_relocator is missing; it is needed for fixing the relocations at runtime

There are many more similar warnings about various gnustep-gui ivar offsets being too narrow.

Clang version:

> clang --version
clang version 19.1.0-rc4 (D:/a/llvm-project/llvm-project/clang 0c641568515a797473394694f05937e1f1913d87)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

I've attached the full log here: gnustep-back-runtime-pseudo-reloc-error.txt.

[gcasa]

#31 added support for a "headless" GUI backend, which @qmfrederik explained here:

The headless backend allows you to use gui components without having them actually render anything to your screen. In our case, we have an application which runs both in command line as well as full GUI mode. Even when running in command line mode, some code from libs-gui is invoked. The headless backend allows us to run those applications.

There may be other use cases, too. For example, I think this would allow you to run code which relies on libs-gui in e.g. a container environment, for, say, unit testing purposes as part of your CI pipeline.

We should add support for a full Win32 backend to allow on-screen rendering.

I think both @gcasa and @qmfrederik have worked on this in the past. I wanted to open this in order to document the current state of GUI support.

I had this working at one point while I was at Keysight but I don't think I merged the changes in because the patch made some changes that didn't seem correct. One was that it needed to add setters/getters to certain ivars which were being accessed directly by the backend, among other changes.

As I understand it that change may have been necessary at that time because of a bug in clang they had since been fixed.

I will try to find the changes so you guys can take a look. That being said it seems like you're on the right track so don't wait up for me. ;)

[qmfrederik]

@arctic-marmoset Thanks, that confirms my theory that the dllimport attribute is missing on all those ivar symbols. On MinGW (or with the -auto-import and -runtime-pseudo-reloc flags enabled), the linker detects this and introduces these pseudo-relocations. That masquerades the issue on MSYS.
For a native Windows build, that results in the error you are seeing, as the _pei386_runtime_relocator is implemented in the MinGW runtime. Since you're doing a native Windows build, you're not linking with that runtime and that symbol is not available.

Long story short, this is a compiler bug and we need to fix the clang compiler so that it marks those ivars with dllimport. I have an experimental fix for that here: llvm/llvm-project@main...qmfrederik:llvm-project:objc-ivar-dllimport .

Regarding __objc_ivar_offset_NSView._cursor_rects.@, yes, that's also a clang-oddity. You should be able to work around by that by updating https://github.com/gnustep/libs-gui/blob/3c160c9bb8b7fc8d5c2bc106873a78a993350fe8/Headers/AppKit/AppKitDefines.h#L75 to check for _WIN32 in addition to __MINGW32__.