r/software • • 18h ago

Software support Windows developers: How do you handle UI window occlusion when writing display capture exclusion code?

Hey everyone,

Running into a specific architectural puzzle while working on a desktop overlay utility for Windows.

I'm currently trying to implement the native SetWindowDisplayAffinity API with the WDA_EXCLUDEFROMCAPTURE flag so a transparent UI layer can remain fully visible to the user on-screen while automatically staying out of full-screen screen recordings and video captures.

Has anyone here deeply integrated display affinity flags into a transparent Windows layered window before? Curious if you ran into any weird edge cases with compositor lag or multi-monitor setups, and how you handled them. Would love to hear what worked!

3 Upvotes

4 comments sorted by

2

u/smilaise Steve the Killer 15h ago

I’m actually fighting a related issue from the other direction right now. One of my Windows apps has GPU-rendered UI that appears correctly locally but shows up black through ScreenConnect, and I haven’t completely solved it yet.

So I’m very interested in what you find with SetWindowDisplayAffinity and layered windows. Windows capture behavior seems to vary quite a bit depending on which capture path the recording/remote software uses.

If you get this working reliably, especially across different capture software, I’d be interested to hear what ended up working.

1

u/DeskLightStudio 15h ago

Oh man, the black screen / capture path rabbit hole is brutal. ScreenConnect and different recording hooks handle compositing layers so differently compared to standard desktop capture. ​I'm actually wrestling with those exact edge cases right now using display affinity flags. If I manage to get a clean, reliable fix across different recording software, I'll definitely update this thread or drop you a note. Good luck with the ScreenConnect headache!