Ideas on modernizing the open-source desktop

For me it was a major revelation when it was pointed out that, for someone who doesn't want his desktop changed, I did install all sorts of crazy things when I was starting with Linux (Compiz plugins, anyone?). And while Gnome 3 pushed me into Mate, it wasn't because it was different but rather because it was bad. So I'm excited about someone trying something new, particularly someone who realizes that the Desktop is not a tablet.

Here's an idea that I believe I read in HN as "the original purpose of the desktop": add a Desktop Manager that manages which icons are on your desktop and makes it easy to switch.

The desktop is kind of weird, in the sense that it's prime real estate but, at the same time, there's an expectation that you shouldn't put too much stuff there, and then it ends up cluttered anyway. I think it's worth trying a stronger separation between "Desktop" and any other regular folder, emphasizing that the Desktop is where you place links to the files you need right now, but with a mechanism to clear it completely (without losing data) and another to switch which icons you see right now, kind of like browser tab groups. That way, when you're working on a project you place the files you need on the desktop and when you're done you clean it up.

I would replace the start menu with a Raycast clone. Most people want to open files, ask a question or run an app they know the name of, not scroll a list of things they might never use. If you add in semantic search users don't even need to know the app name.

It's also a perfect use for local System One models that can classify prompts and forward them to the right module - calculator for math, distrobox for shell commands, LLM for natural language questions and normal search for everything else. Not using a fixed classifier because plugins should be able to add their own question types.

The article mentioned WinFS and relational filesystems. That's another good idea, integrate metadata so you can support use cases like finding all pictures of your dog or all playlists with Ed Sheeran songs (to remove them). Everything bound to the file entries instead of ID3 tags inside the file or app specific databases. It would be a lot more reliable and probably faster.

IIRC bcachefs is already a relational database but I don't know how good the query APIs are. If we can't do full extendable tags then at least add user defined groups/tags like Work and Games and Photos to make finding things easier. People remember what a file is better than where it's stored.

> He talked about the desktop building blocks, basically the familiar WIMP model plus the desktop clipboard, as the way that users move data around. Expert users can move data around quickly, he said, but there's a problem: the desktop is stateless. He called this the curse of direct manipulation; there is no working memory to the desktop. ""If you copy a few too many things to the clipboard, oh sorry, it's gone"". > > Jenson theorized that the reason there are so many desktop environments, window managers, and Wayland compositors for Linux—as well as alternative clipboard managers—is because the desktop isn't quite working right for users. He thought it was really about working memory: ""How do I manage my data in such a way that I can remember it across all of these things and actually use it."" His perspective shifted to asking ""what would we do to working memory to improve the desktop UX?""

I feel this so much.

I think that there's a "better" windowing system where all of your stuff sticks around way more than it used to. Clipboard history, but also just "oh you have these windows open while working on this task".

People have really leaned into coding harnesses + chat windows as a way of grouping together work. I think there's a more general idea in this space. Don't make me explicitly keep track of things, just hold onto it all for me!

There's some universe where my desktop is basically a Notion-like thing, and I can just throw things into semi-persistent workspaces (that I close out when I'm done). Desktop Environments that make this work nicely will be quite cool IMO

(A simple and dumb version of this is the BeOS "group various windows together, even though they aren't from the same application" stuff. Very cool stuff IMO

> The final, particularly strong, objection from the tech community is "don't touch my stuff". People get angry when they have worked hard to get their desktop environment just as they like it and something changes to disrupt that.

Yeah, this can be a little extreme, and I feel we can benefit from being a little more open to trying new things.

The part about designing specifically for large displays and widescreen monitors was interesting. We can also reverse it, though: right now I'm away from home, and I only have this old laptop which is mostly great but has a shitty touchpad as the mouse, and I have to think this must be really common. However, this situation seems to be completely ignored by all desktop UX design. We seem to be pretty good at coming up with ways to use the mouse in our programs, but when the mouse is hard to control and unreliable, it causes pain. I guess I can use ratpoison or something, but that seems to be giving up too much.

Regarding the AI going deep into the filesystem and organizing my data...I'm extremely skeptical of inserting any kind of inherent unpredictability into how I interact with the computer. Giving a command or input should produce a well-defined and predictable (in principle) action. Getting rid of that paradigm fundamentally changes the relationship between the user and the computer in a way that, in my opinion at least, greatly diminishes its usefulness as a tool.

one thing I'll keep on beating the drum on is that microsoft missed an opening with windows 8. WinRT UI was great for mobile, but as noted desktop is different and they tried to force it onto it.

What we need (and to some extent we get in web, i.e. gmail is different based on resolution) is different UIs based on the interface one is using. a single app that can switch between UIs.

I strongly believe the lack of this is why convertibles have remained more of a gimmick than anything else, as in practice you either keep using the app the same way you always did as you switch usages or you use different apps in the the different usage scenarios because they are tailored to that specific usage scenario. Neither are things that really sell the convertible as something with strong usage.

The closest we've gotten is things like DeX on samsung and perhaps what google is rolling out with android.

People have different ideas of what's good UX, because everybody comes from their own tradition of using desktops. Some people don't see anything wrong worth evolving from with mouse, point, click. The kind of thinking from the article, from someone that only worked on big companies, is the thinking from someone looking for the next big product, which I look with suspicion. At some point this stops being about good UX and starts being about a good product that makes good profit.

True revolution in UX will come from making customisation accessible, so everyone can decide what's best for them without losing their minds in endless customisation rabbit holes or having to rely on big and clumsy organizations.

I watched the video form of this and it was pretty informative, but I left it not really feeling anything was missing from the current desktop in terms of window management. It’s simple and old enough to be pretty well designed already.

What I do find is really lacking is UX inside of many foss apps. So many of them feel like windows xp programs with toolbars that look like junk drawers.

I watched the full video and, as with the previous talk, absolutely loved it.

I hope Scott keeps spreading the good word and inspiring others to take different perspectives

Ha I reported that drag and drop bug 20 years ago. Windows does it in an even smarter way - it raises the window on mouse-down but only if the target isn't a drag source.

As I recall X11 doesn't really support that at all so there was no way to implement that on Linux. Doing it on mouse up seems sensible though. I couldn't find my bug report but I bet it was met with "I like it broken".

Once more- allow people to build "AI"-pipes, by recombining programs and agents via GUI.

[Agent: Fetch cat pictures from the internet] >GetFromFolder> [Agent: Filter for Calicos] >> [Agent: in Gimp paint Cat-Ears on Cats ] >> [Agent: Generated Videos] >> [Agent: Reupload to Youtube]

If you can make a "Explorer" with zero code and which allows to store and share building blocks and pipes, congrats.

What I found interesting is desktop ui hasn’t changed much from either a windows layout or Mac layout. Isn’t there any other ui which is as useful as those two? Is there no ongoing research on os ui?

When I first met Linux desktop in the nineties, I was blown away - and I was completely convinced that Linux would overtake Windows as the preferred desktop within a few years. Little did I know that what we would end up with was a thousand mediocre variants of it, instead of a few awesome ones.

My own Linux desktop experience peaked with Compiz, and I have been on macOS ever since.

Good talk.

Even if the desktop turns out to be done though, UX polish like that drag thing is still a good idea. Like why does no major desktop let you just put icons on your desktop the way classic Windows does?

I want something like Os/2 Warp. Something with a design principle other than “make it look slick and hide stuff”. Look up CUA too.

Please give me the option to not use the, "Innovations."

  > He thought that it would be possible to do the things he was demonstrating with Wayland as it is today.

Yeah, with that AND The cooperation and rewrite of every app to actually support usecases like "if dragged to side of screen, get smaller and smaller eventually becoming a widget"

Don't touch my stuff objections run strong. Three things: it's a chicken & egg problem, agents help fix this, and third the 'some assembly required' issue.

Some backdrop, a thesis: tech used to be exciting because there was apparent motion, things changing. The dread hadn't set in, and I think that dread set in because tech shifted from close intimiate and touchable technologies we could touch, to far off systems running in some cloud data center/keep. Change was fun when we got to be on for the ride and feel some ownership but change happening to us was annoying.

So lately, it's hard for anyone to know what their preferences really are. We aren't trying a ton of human empowering things. The organizational systems Scott talks about here? We see some absolutely ferverously dedicated people enjoying Niri, enjoying new infinite space tiling systems. That's a rare example though! A couple people stumble their way into it. There's not that WinCustomize era where it's apparent people are active and busy and rebuilding, remixing their world on and on. People aren't in a receiving trying mode, because in part there's not a ton to try, and because there's no scene, people aren't kicking out new things either. The social poverty of the area & lack of things to look forward to mean people aren't looking, and people trying stuff are lucky to get traction, even when it's good.

This admits some of the "don't touch my stuff." Because we haven't been seeing as much upside to change. But also: it's pathetic! These peole have had their spirits crushed and have no peers out there who are having fun, so they can just wallow in their grumpiness. And we need to kind of ignore these users, because they don't know: they think they want not even faster horses but merely horses that stop breaking down or which start only turning left unless you pay premium. Let these naysayers and grumps alone. But more than anything: show the people you can reach good stuff! Activate interests! Try to restore a sense of motion & exciting change! Create more powerful, richer, more beautiful systems!

Agents help though. Change sucks when it gets in the way. The downsides though are lower than ever, if you have systems flexible enough to let agents work them. If something changes in a way you don't like, an agent may well be able to make quick work of it! Tell it, and it'll go fix it.

Last: the granularity of build it yourself systems is not great. Downloading Sway, a desktop shell atop it,then a launcher, then a host of other small utilities to cobble together is a lot of decision making to do, and most of these options for these roles have lots of competitors with little interoperability. The tools that makeup the desktop stand alone, small pieces, and it's up to you to pick each one and loosely couple it, with it's bespoke means of doing so. These patterns of assembly suck. There's significant money heading towards a new omakase system, and I don't think it's necessarily the authoritarianism/fascism per se, the desperation for authority to make sense of the world, but given how scattered the pieces are, I have some sympathy for why they want someone to say they've built the beautiful well hewn omakase thing. And that your agent can customize it.

One of my favorite examples for a system that does compose nicely is astronvim on neovim. It takes some well known very great core plugins for nvim, and builds a little zellij like mini-framework to nvim's tmux atop that: a visual leader key, with menu system builtin. It creates a framework for more to plug into. And then the community has lots and lots of plugins or packs of plugins that it takes from upstream plugins everywhere, and it packages them for astronvim. Hundreds if not thousands of plugins. Most require not that much extra to hook in nicely in to the framework, to make them more apparent, visual, seen in the broader system. The system composes capabilities into itself nicely, integrates. Where-as when you get a regular nvim plugin, it usually can assume either nothing or way too much about what kind of nvim it runs on. It has to define it's own world. I think astronvim is an incredible model for making common layers for assembly, and it's packs show something inbetween a single plugin and an app: it's a unit of software that aggregates a couple other modules, yet is shy of being a full app. That kind of composability of experiences is unique and special: not having to assemble every piece ourselves, and yet not having an entire massive system dropped on us. Figuring out how to deliver better units of capabilities, composable subsystems, is really missing on the desktop and is much needed.

No, thanks, I require no modernization. I just need stable, but customizable UI, so that I can customize it once and use it my entire life without needing to adopt to new stuff constantly.