The complexity that lives in the GUI(blog.royalsloth.eu) |
The complexity that lives in the GUI(blog.royalsloth.eu) |
hA! Hit the nail on the head :P
However, React introduces a lot of complexity to avoid unnecessary DOM updates, which makes me wonder about the viability of an immediate mode GUI in the browser using canvas.
The value of essentially creating and mutating a tree structure like the DOM is that things like screen readers and UI automation tools can read it. With canvas they just see a big bitmap.
- https://github.com/Erkaman/pnp-gui
- https://github.com/jnmaloney/WebGui
I guess accessibility is what takes the biggest hit with these implementations.
- it is badly designed. If state is too hard to manage it means often that the design is bad
- state is not the most important part of the UI. Interaction and being elastic to changes are the most important things
- frontend is often overengineered in a bad way. When you use bad solutions like React or Redux there is no change there won't be any problems.
If you have a problem synchronizing your views with a light, that's because the light should exist outside of your views.
MVC was invented in the 70s. Why do we have to act that this is not a solved problem?
The home page is a bit ugly, but it contains a range of examples that are commonly awkward to implement in other UI libraries:
No programming paradigm can stand up to rushed/flawed mental models.
The domain can become quite complex; it is wishful thinking to believe that a single approach could drain it of all complexity.
How is the click propagated recursively through every component and is the position compared repeatedly at every step?
Win* GetChildAt(Win* w, int x, int y)
{
size_t i;
if (x >= w->ScreenX1 && y >= w->ScreenY1 &&
x <= w->ScreenX2 && y <= w->ScreenY2) {
for (i=0; i < w->ChildCount; i++) {
Win* child = GetChildAt(w->Child[i], x, y);
if (child) return child;
}
return w;
}
return NULL;
}
Call this on the root window and you have the deepest child at the given coordinates.(though there is usually a bit extra logic for handling, e.g., invisible and disabled windows)
Well no. There are applications with nice well thought out GUIs.
>"Congratulations, a large amount of your effort will go towards resolving weird message bus problems as opposed to writing the business logic of your app"
Sorry but I do not resolve "weird message problems". I use my own publish-subscribe mostly asynchronous message bus for my GUI apps (actually I use it also for non GUI parts as well). Components (visible or not and including running threads) can subscribe to events. It does not exhibit any performance / memory problems and in combination with the global app state object makes programming interactions a piece of cake.
Simply not true. I recently had to salvage business rules from a codebase written by single person over the course of many years. It was probably one of the messiest code I've ever seen.
>"How big is your codebase and how many people work on it"
Depends on a project. On some I work alone. Some had 2-3 persons. The biggest team I've ever had to lead was about 35 people (not all developers). Properly organizing and splitting work and using mostly experienced people the resulting code was very decent and quite possible to grasp. Also well documented.
1. Mixed concerns and a lack of application layering. In React code bases and others like them, its not uncommon for me to find business logic embedded directly inside the components themselves. This makes the component inherently coupled to the specific implementation its being used and can only be re-usable in other contexts by passing in modifier flags to the props etc. In my opinion, components should be mostly of the stateless flavor and delegate anything not directly related to their immediate UI concerns elsewhere. This increases the likelihood of re-usability of your components and makes testing business and component logic much much more straight forward.
2. This might just be my personal experience, but I've noticed a bit of a dismissive attitude around design patterns and traditional computer science principles among my front end brethren. YAGNI and all that. While I think its fair that the UI != to the back end, I think frequently the baby gets thrown out with the bath water. For instance, I frequently leverage dependency injection in my front end work as a way to make my code more flexible and testable. It's hard to sell the benefits of something like DI until your application grows in complexity to the point that you are dealing with things like nested mocks in your tests and/or need to make a non trivial change to some piece of functionality. I've been seeing the winds start to shift a bit on this which is encouraging.
3. Most of the time there is little to no overall _conceptual integrity_ to a given front end code base. Its uncommon for me to come into an existing code base and have anyone be able to tell me what the architecture is comprised of. Things like what logic lives where and why. I'm not saying people _don't_ do this, but in the more gnarly code bases I've encountered, this "accidental architecture" is more common.
4. Front end is still see as "easy" and the place you stick the less experienced engineers first. I sincerely hope this doesn't come off like I am gatekeeping or anything. I work with some absolutely brilliant people who are only a year or two into their career and much smarter than me. IMO its less about skill and more about having been burned enough times to know where unexpected complexity lie.
I love front end. Its challenging and interesting and maddening. My hope is that it will continue to mature and grow to the point that it gets the same respect as other disciplines within this field. These days I work full stack so its not all I do, but it will always be my favorite. :)
As a rule practical and manageable UIs use all these approaches at the same time.
Components, class based and/or functional (fe: UserSection, InventoryTable ), are in principle loosely coupled. They may not know about each other and use messages to communicate.
The Flight by Twitter ( https://flightjs.github.io/ ) was/is probably the first practical thing that pioneered that approach.
The Flight is not even a framework but rather a design principle.
In fact it is an anti-framework - it postulates that in practice each component may have its own optimal internal architecture. One component is better to use React-ivity, another - Angular-ish data binding, etc. On some, immediate mode drawing is the most natural.
Smaller the task (component) - greater the chance to find silver bullet for it.
I see OP doesn't mention this option, but is slightly related to both option 1 (connect the box) and 3 (message bus / event emitter). This option is similar to how OS provides an API to user space application program. For example Windows provides an API to an application to flash its window without exposing all of its state like mentioned in option 1. https://docs.microsoft.com/en-us/windows/win32/api/winuser/n...
Here's the detail:
A self-powered object, let's call it workingIndicatorObject, can be introduced to work on the working indicator. It provides 1.) `getState() -> WORKING/NOT-WORKING` a function to get its state, and 2.) `register() -> None`, a function to register user activities. These functions are dependencies for UserSection component and InventoryTable component respectively. In term of lifetime and hierarchy, it precedes and outlives both components.
The workingIndicatorObject MUST have its own lifecycle to regulate its internal state. Its core lifecycle MUST NOT be managed under the same event loop as the GUI components, assuming the program is written on top of a framework that has it (React, Vue, etc). This is to assure that it doesn't directly affect the components managed by the framework (loose coupling). Although, a binding MAY be made between its state and the framework-managed event loop, for example, in React environment, wrapping it as a React hook. An EventEmitter can also be provided for a component to listen to its internal event loop.
Injecting it as a dependency can use features like React Context or Vue's Provide/Inject pattern. Its consumer must treat it as an optional dependency for it to be safe. For example, the UserSection component can omit drawing the "working" indicator if it doesn't receive the `getState` function as a dependency.
Internally, the workingIndicatorObject can use a set. The set is used to store a unique value (a Symbol in Javascript). `register` function creates a Symbol, stores it in the set, and deletes it after $TIMEOUT, while firing event both at addition and deletion. `getState` function is a function that returns `set.size()`. When bound to the component, an addition to the set fires an "update" command to the component, which redraw the component along its internal state. This is just one example of implementation and there are other simpler ways to have the same behaving workingIndicatorObject.
This approach allows UserSection and InventoryTable to only know `getState()` and `register()` function and nothing else, and having both of them optional. Using static type system such as TypeScript can help a lot in this since we can pin down the signatures of both function to `null | () => WORKING | NOT_WORKING` and `null | () => void`, where we can enforce both dependent components to check if it exists and to call it with the correct types, otherwise the compiler yells.
Small point, but it grated.
"the scrimers of their nation, / He swore, had had neither motion, guard, nor eye, / If you opposed them."
King James Bible:
"For I am persuaded, that neither death, nor life, nor angels, nor principalities, nor powers, nor things present, nor things to come, nor height, nor depth, nor any other creature, shall be able to separate us from the love of God, which is in Christ Jesus our Lord."
Rudyard Kipling:
"But there is neither East nor West, Border, nor Breed, nor Birth, / When two strong men stand face to face, tho’ they come from the ends of the earth!"
Charles Dickens:
"I had youth and hope. I believe, beauty. It matters very little now. Neither of the three served or saved me."
Jane Austen:
"Mary's ailments lessened by having a constant companion, and their daily intercourse with the other family, since there was neither superior affection, confidence, nor employment in the cottage, to be interrupted by it, was rather an advantage."
Thomas Hardy:
"Half an hour passed yet again; neither man, woman, nor child returned."
Samuel Johnson:
"Among these, Mr. Savage was admitted to play the part of Sir Thomas Overbury, by which he gained no great reputation, the theatre being a province for which nature seems not to have designed him; for neither his voice, look, nor gesture were such as were expected on the stage"
The authors above are not cherry-picked, except that I happened to remember the Kipling and St Paul quotations. Not one of the notable English writers I looked up failed to provide me with at least one example of "neither" applying to more than two things.
1. Library authors optimize aggressively for the beginner. They specifically sell their tools as being all in one and you can do everything in the view layer. Maybe its because they legitimately want to set their users up for success or maybe they'd rather gain users even if they are setting them up for issues in the long run
2. The kind of people working on web frontend aren't necessary coming from a programming background. You have the web designer/developer types who were doing a great job when they were building out static templates that got integrated into some server rendering back end. Now that everything has become an app, the idea of handling all the associated complexity is daunting. A lot of self taught developers also come in through web UI and the last thing they want to think about is proper architecture.
I'd go so far as to say there is a current of anti-intellectualism in terms of frontend application design. There is a large section of developers who want a small number of tools to do everything for them and it seems they are unwilling to even consider creating their own architecture of which individual libraries are their own self contained piece.
> I'd go so far as to say there is a current of anti-intellectualism in terms of frontend application design
I try to walk a line here between relying on my experience and being open to the idea that there is possibly a better way to do things that is just unfamiliar to me. "Beginner mindset" and all that. React hooks are a great example. At first I hated them because I didn't really understand what problems they solved well. To my eyes they encouraged mixing concerns in the component layer in ways that made testing and re-usability way harder. It was when I started considering hooks as "headless components" that I started understanding where they fit into a front end architecture. I think they solve a specific class of problem that react had no answer for previously. I don't use them for everything but now that I consider them as a special kind of component, they fit into a larger set of tools that I have at my disposal.
I also think there are different kinds of front end developers and we all get lumped into the same bucket right now. There are those that work on flashy UI stuff (which I suck at) and those that do more "middle tier" type stuff where they are building out web apps that solve business problems (more my sweet spot). Those two skill sets are way way way different from one another to the point that I think they should be classified as different disciplines.
As per dependency injection, you inject stuff the other party needs. As per "dependency inversion principle", the thing you inject stuff into owns the interface for what it needs. That means a concrete object you want to inject in many places should be made to conform to interfaces defined by the "recipients" of the injection, which may - and likely will be - different from each other. And in particular, an interface may be as simple as a function.
So, for instance, if a component needs a log sink to write to, and it only ever writes a single type of messages, you do not need to ship the whole logger object (and with it, the knowledge of what a logger is). All it needs is a function `(String) => void`, so all you need to inject is your logger's Info() method.
Another core point is actually the architecture of things, which lives/dies first/last, who depends on who, and how to connect them that each party is "happy" with what's on their plate.
> incidentally, when I built my own DI system for a game
Indeed systems in game are fun to play with! I've had my share of designing a game engine once and I learnt a lot from it too. I'm intrigued, what game were you working on?
https://floooh.github.io/sokol-html5/imgui-highdpi-sapp.html
There are also demos for other immediate-mode UI systems on the parent page (Nuklear and microui):
https://floooh.github.io/sokol-html5/
As I wrote in another thread, Immediate Mode UI doesn't imply how the UI rendering is implemented, only how the API works.
But a system that may be clean and elegant for a 1-person gig may end up falling to bits if you attempt to have 100 people working on it.
I get what you mean about hooks, I was initially apprehensive and like you I realized the problem they solved. I think the problem we have now, is they are the answer to everything, along with Context. Code that could have been a plain Javascript module is a hook that uses Context to pass refs between components.
I think a big issue is reactivity outside of a reactive view library isn't talked about much. You have libraries like RXJS and Effector, which I'm a big of, that let you handle reactivity separately from a view but they're still full black boxes themselves. I'm not sure many people think about reactivity as a quality separate from any library.
The libraries react-query and svelte-query are an interesting manifestation of this. I haven't diffed the code or anything but they share a core that is essentially the same. Observer based reactivity to handle async calls. Yet there are two different repos with duplicated code providing different frontends based on React and Svelte. I don't want to pick on how the code architecture or anything but it feels like the reason there isn't just a base "query" library that exposes its own Observer API with bindings to React and Svelte is because of the general refusal to accept splitting out responsibilities and there should be a "React way" of doing things.
I'm hoping the TC-39 Observer proposal along with Svelte's idea of a store contract might open a few eyes and show people that the frontend doesn't have to be the closed off black box that it currently is but a lot of people already seem set on the idea that Observers are some RXJS stream based thing that they want no part of.
Anyway sorry for the long reply. I've been thinking about this stuff a lot in trying to establish my course through all this.
It's an unpublished roguelike that I'm working on, slowly, on hobby time. ASCII graphics, focused on distance combat, and with a hybrid ECS/event-based architecture, where the entire game state is stored in an in-memory SQLite database. More of a research project into the benefits of such architectural decisions than an actual game.
The DI system kind of grew out of my design goal to make the game logic testable in headless mode, without rendering or input (I have only so much time to write and maintain it, and I sometimes code over SSH). I started architecting things with dependency inversion in mind, and then figured I could have the whole game's object graph instantiated in a single place. I decided to make it declarative and composable (pretty much concatenative) by writing code that automatically instantiates the objects in correct order, and then I realized I've reinvented a DI container.
I wonder though what's the upside of storing the game objects in SQLITE?
John Milton:
"Into this wilde Abyss, / The Womb of nature and perhaps her Grave, / Of neither Sea, nor Shore, nor Air, nor Fire, / But all these in thir pregnant causes mixt / Confus’dly, ..."
Daniel Defoe:
"And now I thought myself pretty well freighted, and began to think how I should get to shore with them, having neither sail, oar, or rudder, and the least capful of wind would have overset all my navigation."
William Wordsworth:
"Neither vice nor guilt, / Debasement undergone by body or mind, / Nor all the misery forced upon my sight, / Misery not lightly passed, but sometimes scanned / Most feelingly, could overthrow my trust / In what we may become"
Thomas Macaulay:
"Neither blindness, nor gout, nor age, nor penury, nor domestic afflictions, nor political disappointments, nor abuse, nor proscription, nor neglect, had power to disturb his sedate and majestic patience."
Jonathan Swift:
"I find likewise that your printer has been so careless as to confound the times, and mistake the dates, of my several voyages and returns; neither assigning the true year, nor the true month, nor day of the month"
George Bernard Shaw:
"But there are men who can neither read, write, nor cipher, to whom the answer to such sums as I can do is instantly obvious without any conscious calculation at all; and the result is infallible."
G K Chesterton:
"He takes us into the schools of inhumanist learning, where there are neither books nor flowers, nor wine nor wisdom, but only deformities in glass bottles, and where the rule is taught from the exceptions."
Since the above are all notable English (or in two cases Irish) writers, here are a few notable Americans.
Herman Melville:
"But neither great Washington, nor Napoleon, nor Nelson, will answer a single hail from below, however madly invoked to befriend by their counsels the distracted decks upon which they gaze"
H L Mencken:
"I am thus neither teacher, nor prophet, nor reformer, but merely inquirer."
Mark Twain:
"It is not pleasant to see an American thrusting his nationality forward obtrusively in a foreign land, but Oh, it is pitiable to see him making of himself a thing that is neither male nor female, neither fish, flesh, nor fowl--a poor, miserable, hermaphrodite Frenchman!"
I didn't actually check exactly what OP had said. I find that OP now doesn't use the word "neither" at all, but I guess the sentence that now says "none of the presented options are without problems" used to say "neither". Which I agree looks very strange. (The corrected text is arguably still wrong: "none" = "not one", and you would say "not one of them is ...", not "not one of them are ...". But almost everyone, almost all the time, uses "are" rather than "is" with "none", so I'm reluctant to call it wrong; correctness in language is determined by how actual language-users actually speak and write.)
So, my apologies: I made what I now think was probably a wrong guess at what the article had done that you didn't like, and my examples aren't really to the point.
I had a look in the OED. With the neither/nor/nor/... construction, it regards any number of branches as normal (as one would hope given the examples above). In "neither of them" and similar usages, its main definition says specifically "of two". It does explicitly countenance using it with more options, but describes this as "Now rare".
So I still wouldn't go so far as to say that what OP originally wrote is outright wrong, but I do agree that it was odd and changing it was probably a good move.
Neither sleet, nor snow, nor gloom of night..
The hardest part of dealing with state in a complex system is maintaining consistency in different places. Some instances of this this can be avoided by creating single-sources-of-truth, but in other cases you can't unify your state, or you're dealing with an external stateful system (like the DOM), and you have no choice but to find a way to keep separate pieces of state in sync.
I should probably write a blog post on this
I've been developing a convention in React over the last year that uses this idea and it's very very nice. I'm also trying to write a platform-agnostic UI language specifically for designers that makes this a first class concept.
That mental model has been the most natural for me as well. It really clicked for me when reading Trygve Reenskaug's descriptions of MVC [1], particularly this paragraph (emphasis mine) and the illustration that follows:
> The essential purpose of MVC is to bridge the gap between the human user's mental model and the digital model that exists in the computer. The ideal MVC solution supports the user illusion of seeing and manipulating the domain information directly. The structure is useful if the user needs to see the same model element simultaneously in different contexts and/or from different viewpoints.
[1]: https://folk.universitetetioslo.no/trygver/themes/mvc/mvc-in...
How does the UI get notified of changes? (like the article discusses some changes might come from different part of the UI, or might even come from external like in a chat client)
How do you handle actions of the user? (Of course in the controller in MVC, but how does it work exactly?)
See this: https://flutter.dev/docs/get-started/flutter-for/declarative
Absolutely. The error is in trying to treat GUI elements as an actual data model. That and premature optimization trying to only update things as they change.
Once you take that approach the only remaining hard part is that for complex applications, events can trigger state dependent behavior. But that should be at a layer under the GUI toolkit.
This complete example will toggle the checkbox every second (1000ms), or the user can click to update the variable. The checkbox watches variable "v".
#!/usr/bin/wish
checkbutton .button -variable v -text "Click me"
pack .button
proc run {} {
global v
after 1000 run
set v [expr !$v]
}
runCasey Muratori first spoke about IMGUI back in 2005, but didn't actually implement it practically: https://www.youtube.com/watch?v=Z1qyvQsjK5Y
In around 2013, Omar Cornut wrote an incredibly high quality practical implementation of Muratori's concept called _Dear ImGui_: https://github.com/ocornut/imgui Using both Cornut's library and Muratori's mindset is incredibly powerful and liberating. Things that would require days of work and multiple files in Qt or Cocoa can be finished in four hours and a couple of hundred lines of IMGUI code. It also uses an order of magnitude less CPU/memory resources (which was an issue for us as we were rendering and displaying RAW video footage in real-time).
I find it amazing that this way of thinking hasn't completely dominated frontend programming by now. There is literally no downside to using the IMGUI concept - entire classes of bugs disappear, and your code is simultaneously smaller, easier to maintain and more logical in flow. It's also a shitload more fun to write (something I think that SWE culture overlooks too much) - you spend the majority of your time writing code that directly renders graphics to the screen, rather than fixing obscure bugs and dealing with the specifics of subclassing syntax.
There's a big downside, the performance penalty. For a GUI that renders moderately complex objects, the cost of not caching becomes overwhelming, the equivalent of losing 20 years in GL architecture advances. Pushing the VBO to the GPU each frame is the same as losing indexing. My own application doesn't render at 60 fps in immediate mode.
I find that people who believe that IMGUI is somehow faster than RMGUI are game developers who have been taken in by marketing, because basic knowledge of GPU programming (i.e. what is indexed rendering) is enough to see that this couldn't possibly be true. Their UIs are usually simple enough that the performance penalty is not important. And many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier.
Though the bigger reality for most front-end devs in 2021 is that most of us are building for the web, and the web doesn't really allow for immediate-mode
I think this is what Elm solved (and then by plagiarising the same approach, React and Vue). Make the interface declarative rather than imperative and voila - state is automatically in sync.
Good GUIs have way too many requirements to be controlled (via a controller) by the model. As a typical example, the fact that a button should only be activated if there is data in a textbox has usually nothing to do with the underlying model. The model should only ever contain valid and consistent data.
In the aggregate, what ends up being most effective for me, is rapid prototyping, and “paving the bare spots.”[0]
I find that I do a terrible job of predicting user mental models.
The rub with prototypes, is that they can’t be lash-ups, as they inevitably end up as ship code. This means that a lot of good code will get binned. It just needs to be accepted and anticipated. Sometimes, I can create a standalone project for code that needs to go, but I still feel has a future.
So there’s always a need for fundamental high quality.
What has been useful to me, is Apple’s TestFlight[1]. I write Apple software, and TestFlight is their beta distribution system.
I start a project at a very nascent stage, and use TestFlight to prototype it. I use an “always beta” quality approach, so the app is constantly at ship quality; although incomplete.
It forces me to maintain high quality, and allows all stakeholders to participate, even at very early stages. It’s extremely motivating. The level of enthusiasm is off the charts. In fact, the biggest challenge is keeping people in low orbit, so they don’t start thinking that you are a “WIZZARD” [sic].
It also makes shopping around for funding and support easy. You just loop people into the TestFlight group. Since the app is already at high quality, there’s no need for chaperones or sacrifices to The Demo Gods.
I like that it keeps development totally focused on the actual user experience. They look at the application entirely differently from me.
[0] https://littlegreenviper.com/miscellany/the-road-most-travel...
Statecharts is currently probably the most undervalued tool when it comes to programming GUIs with state. Statecharts are a continuation of state machines, but with less footguns and better abstractions to be able to build larger systems.
In the end, you either build a GUI that are using state machines implicitly, or explicitly. Tends to be less prone to bugs if you do so explicitly.
If you're interested, here is some starting points (copied from an older comment of mine):
Here is the initial paper from David Harel: STATECHARTS: A VISUAL FORMALISM FOR COMPLEX SYSTEMS (1987) - https://www.inf.ed.ac.uk/teaching/courses/seoc/2005_2006/res...
Website with lots of info and resources: https://statecharts.github.io/
And finally a very well made JS library by David Khourshid that gives you lots of power leveraging statecharts: https://github.com/davidkpiano
While we're at it, here are some links to previous submissions on HN regarding statecharts with lots of useful and interesting information/experiences:
- https://news.ycombinator.com/item?id=18483704
- https://news.ycombinator.com/item?id=15835005
- https://news.ycombinator.com/item?id=21867990
Many complicated views have their own internal models for things like where they are scrolled, what columns are shown, or what elements of a tree are expanded. But those compound views are written so that, from the outside, they appear exactly the same as any other view.
> I’d love to hear what the functional programming camp has to say about this problem
It's called functional reactive programming.
> a view updating itself from the model should never trigger transactions on the database
i.e. a view rendering from state/messages, should never re-emit new state/messages.
The separation of an underlying data model from any particular way of presenting its current state is a powerful idea that has proven its value many times. We’ve used it to build user interfaces, from early GUIs when we didn’t yet have the kinds of libraries and platform APIs we enjoy today, through games with unique rendering requirements and a need to keep frame rates up, to modern web and mobile app architectures. The same principle is useful for building non-user interfaces, too, in areas like embedded systems where you are “presenting” by controlling some hardware component or distributed systems where you are “presenting” to some other software component as part of the larger system.
But I found that it's almost impossible to describe to someone used to event-/callback-driven UIs why exactly that is. You really need to try it yourself on a non-trivial UI to "get it".
This approach isn't without it's own challenges of course. For example it is sometimes hard to keep track of what is going on in complex applications with cascades of signals and slots. Some people also hate the fact that signals and slots use auto generated code, but I have never really found that to be a problem in practise.
I'm optimistic about Qt 6's QProperty (I don't know how it compares to FRP or, as someone else mentioned, MobX), but Qt 6 currently does not have KDE libraries, or Linux themes to fit into desktop environments or distros.
One thing you have to watch out for it that programmatic changes can fire signals. If you don't want this you have to add QObject::blockSignals(true);...QObject::blockSignals(false); around your call.
He did miss out a pretty significant flaw of message busses - the producers and consumers of messages are completely decoupled, which makes debugging really annoying because you don't have anything like a stack trace. You just know that your component received a message, and good luck trying to figure out where it was sent from and why.
That's also a big problem with magical reactivity systems like Vue. Your stack trace is pretty much "yeah something changed somewhere so we're updating this component. Don't try and figure it out."
This is actually a really good point. I don't know why you were downvoted.
A solution I’ve always had is to build your message bus with logging in mind initially
I think you misunderstood. Of course there's always a stack trace; you're still executing code. But with message buses and magic reactivity systems your stack trace always just goes to `mainEventLoop()` or `processEvents()` or whatever.
It doesn't go to the thing that actually caused the change as it would if you used direct function calls. I'm not saying it's a deal breaker, it's just a notable downside of those architectures.
IMO, a great solution here is along the lines of "Lift the state up" / "MV(X)". But... there is a vital detail which is usually missed when deciding how exactly the state in your M gets passed to your V for display: you must refresh your entire V when M changes in any way, not just the bit of V that you think changed. It's the only way to completely remove these difficult to test, hard to spot edge cases that the article discusses.
This is almost impossible to talk about without specific examples, so a while back I wrote such an example that I think distills the core problem, and demonstrates how refreshing the entire V not only solves the problem but typically takes less code to do it: https://dev.to/erdo/tutorial-spot-the-deliberate-bug-165k
> Immediate Mode GUIs somehow never reached the critical mass and you are probably not going to find it outside of the games industry.
Isn't this what React is built on? I think this was part of the 'original' selling point by Pete Hunt: https://youtu.be/x7cQ3mrcKaY (2013). Around 20:00 in that video he compares React with the Doom3 engine.
Agree with both of these points. You can no longer treat assignment simply as the way you mutate data, you also have to anticipate its effects in the data-binding system.
I imagine the circular event problem could be addressed with static analysis, but I don't know of any framework that does this.
That’s why in some framework I´m working on, the only events are those from input devices (keyboard, etc.) and windowing, views don’t generate any and are just here to paint the state and indicate to the controller where things are on the screen.
Often there is a price paid in brevity, but I believe it is worth it. It may seem annoying to propagate a click explicitly through 5 parent components just to sum clicks into a count widget, but as soon as a short circuit is made, you've created a graph, and you lose the ability to isolate GUI sub-trees for testing/debugging.
struct LightState {
var isLightOn: Bool
}
class LightViewController {
let lightView = UIView()
}
class StateDirector<
LightState,
LightViewController> {
let lightState: LightState
init(state: LightState) {
self.lightState = state
}
func bind(
view: LightViewController
) {
// Every time is turned on changed call this
LightState.add(
listener: self,
for keyPath: \.isTurnedOn,
handle: .method(Self.handleLightChange)
}
func handleLightChange(isOn: Bool) {
view.lightView.backgroundColor = isOn ?
.green : .red
}
}
This allows clear separation of State, View and Changes.You can then just model your state in your reducer and “simulate your views” inside Unit tests
We're in the age of 4K, 60 FPS rendering. If any GUI application has a message bus that's strained enough to impact performance, then either the application isn't made for humans (because if all that stuff is doing anything it'd result in a screen updating far faster than it could be read), or there's some horrible bug somewhere that produces a flood.
Or is that really the same as immediate mode ?
For the example given in the article, the update function could look something like
def View.update():
if model.lightTurnedOn:
self.backgroundColor = red
else:
self.backgroundColor = ibmGray
This way, all view property changes happen in one place, where you can read the code and understand how the view will appear in each possible state. Circular listener loops are impossible, and view properties for animations can even be computed by calling update twice (once before and once after the state change).The answer for me has been pervasive use of MobX computeds everywhere. See https://mobx.js.org/getting-started
<REDACTED> - - [14/Feb/2021:22:26:45 +0100] "GET /posts/the-complexity-that-lives-in-the-gui/ HTTP/1.1" 200 16798 "-" "HackerNews/1391 CFNetwork/1220.1 Darwin/20.3.0"
<REDACTED> - - [14/Feb/2021:22:26:45 +0100] "GET /posts/the-complexity-that-lives-in-the-gui/ HTTP/1.1" 200 16798 "-" "HackerNews/1391 CFNetwork/1220.1 Darwin/20.3.0"
<REDACTED> - - [14/Feb/2021:22:26:45 +0100] "GET /posts/the-complexity-that-lives-in-the-gui/ HTTP/1.1" 200 16798 "-" "HackerNews/1391 CFNetwork/1220.1 Darwin/20.3.0"
The requests usually come from a certain ip multiple times until fail2ban bans it. It's not just one offender, there are multiple behaving like that.As for why it occurs so often in quick succession, perhaps there's a bug in the app causing it to fetch several times instead of once.
If anybody knows which app is that, please tell the maintainer that they have a serious bug. It's night here, so I am logging off.
And that's a good thing, because so far, AFAIK, no one has implemented accessibility (e.g. for screen readers) in an immediate-mode GUI. I hope to work on that problem sometime soon.
Modern React, with hooks and functional components, solves the problem posed in the article by choosing option 2, lift the state up. The hypothetical problem (change the avatar background when the “working” light is on) is a non-issue. You wouldn’t add an event listener for the light’s state change, you’d simply pass the “isWorking” prop to both the light and the avatar, and they would each render based on the value of “isWorking”. There is no reason for one component to know about the other; they don’t actually do anything except sit there and look pretty, correctly.
The UI is always backed by a data model. The more explicitly you express that model—by keeping it all in one tree like Redux does, for instance—the simpler your UI becomes to reason about. React won’t (can’t) stop you from doing bad things like hitting random services when a component loads, but its design, especially recently, guides you away from that pitfall.
1) Define a domain model + service(s) that fundamentally addresses the logical business functionality without any notion of a specific UI or its stateful nature. This service should be written in terms of a singleton and injected as such. This is something that should be able to be 100% unit testable, and could be exposed as a prototype in a console application or back a JSON API if necessary (i.e. pretend you are writing a SAAS).
2) Define UI state service(s) that take dependency on one or more domain services. These should be scoped around the logical UI activity that is being carried out, and no further. The goal is to minimize their extent because as we know more state to manage means more pain. These would be injected as Scoped dependencies (in our use of Server-Side Blazor), such that a unique instance is available per user session. Examples of these might be things like LoginService, UserStateService, ShoppingCartService, CheckoutService, etc. These services are not allowed to directly alter the domain model, and must pass through domain service methods on injected members. Keep in mind that DI is really powerful, so you can even have a hierarchy of these types if you want to better organize UI event propagation and handling.
3) Define the actual UI (razor components). These use the @inject statement to pull in the UI state services from 2 above. In each components' OnRender method, we subscribe to relevant update event(s) in the UI service(s) in order to know when to redraw the component (i.e. StateHasChanged). Typically, we find that components usually only inject one or 2 UI state services at a time. Needing to subscribe to many UI state events in a single component might be a sign you should create a new one to better manage that interaction.
This is our approach for isolating the domain services from the actual UI interactions and state around them. We find that with this approach, our UI is quite barren in terms of code. It really is pure HTML/CSS (with a tiny JS shim) and some minor interfacing to methods and properties on the UI state services. This is the closest realistic thing I've seen so far in terms of the fabled frontend/backend isolation principle. Most of the "nasty" happens in the middle tier above, but it is still well-organized and easy to test. By enforcing immutability on the domain services, we ensure discipline with how the UI state services must consume the model. Blazor then goes on to eliminate entire classes of ridiculous bullshit by not forcing us to produce and then consume an arbitrary wire protocol to get to our data.
> By enforcing immutability on the domain services, we ensure discipline with how the UI state services must consume the model
Could you expand on that part? Do you mean that your domain services use an append-only approach to manage state?
Second, with this type of modeling on the separation of GUIs and system, what type of movement do you think there will be in Microsoft and Google going towards an even more minimal computer, almost entirely reliant on the cloud space for compute power. Google's machines are practically there, but from what you've mentioned above seems like a more fully "realized" approach than Google's.
Yuck. Use continuations with anonymous fns that tell the GUI what to do next. The state is in the closure implicitly!
I wish I had an articulate counterargument, but just look at this: https://xstate.js.org/docs/#promise-example
Statecharts are a formalism for modeling stateful, reactive systems.
Beautiful is in the eye of the beholder, and never moreso when the programmer is blind to the cost of repetition and state.
Arc got it right. http://www.paulgraham.com/arcchallenge.html
Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying "click here." When that is clicked it should lead to a third page that says "you said: ..." where ... is whatever the user typed in the original input field. The third page must only show what the user actually typed. I.e. the value entered in the input field must not be passed in the url, or it would be possible to change the behavior of the final page by editing the url.
(defop said req
(aform [onlink "click here" (pr "you said: " (arg _ "foo"))]
(input "foo")
(submit)))I don't necessarily agree with all the implementation details of xstate, in particular to where the logic tend to be located in practice, and the reliance on the Actor model for many things in the wild. I rather try to guide people to Statecharts as a paradigm overall, and if you happen to use JS, I think xstate is probably the most mature library there. But as all libraries/frameworks, they can be over-relied upon.
If you're in the Clojure/Script world, which is where I mainly locate myself, then https://lucywang000.github.io/clj-statecharts/ is all you need and so far the library I've had the best luck with.
https://www.edx.org/course/programming-for-everyone-an-intro...
Accessibility is hard. The OS needs to know visual tree to be able to conclude things like “this rectangle is a clickable button with Hello World text”.
Complex layouts are hard. Desktop software have been doing fluid layouts decades before the term was coined for web apps, because different screen resolutions, and because some windows are user-resizable. Layout can be expensive (one reason is measuring pixel size of text), you want to reuse the data between frames.
Animations are hard. Users expect them because smartphones introduced them back in 2007: https://streamable.com/okvhl Note the kinetic scrolling, and soft “bumps” when trying to scroll to the end.
Drag and drop is very hard.
There's nothing in the idea that forbids immediate mode UI frameworks from keeping any amount of internal state between frames to keep track of changes over time (like animations or drag'n'drop actions), the difference to traditional UI frameworks is just that this persistent state tree is hidden from the outside world.
Layout problems can be solved by running several passes over the UI description before rendering happens.
For accessibility, the ball is mainly in the court of the operating system and browser vendors. There need to be accessibility APIs which let user interface frameworks connect to screen readers and other accessibility features (this is not a problem limited to immediate mode UIs, but custom UIs in general).
1. Application data
2. Presentation data
3. Presentation rendering
The first is the single source of truth for your application state, what we often call a “model” or “store”. It’s where you represent the data from your problem domain, which is usually also the data that needs to be persistent.
The second is where you collect any additional data needed for a particular way of presenting some or all of your application data. This can come from the current state of the view (list sort order, current page for a paginated table, position and zoom level over a map, etc.) or the underlying application data (building a sorted version of a list, laying out a diagram, etc.) or any combination of the two (finding the top 10 best sellers from the application data, according to the user’s current sort order set in the UI). This is often a relatively simple part of the system, but there is no reason it has to be: it could just as well include setting up a complicated scene for rendering, or co-ordinating a complicated animation.
The final part is the rendering code, which is a translation from the application and presentation data into whatever presentation is required. There isn’t any complicated logic here, and usually no state is being maintained at this level either. The data required for rendering all comes ready-prepared from the lower layers. Any interactions that should change the view or application state are immediately passed down to the lower layers to be handled.
The important idea is that everything you would do to keep things organised around the application data also applies to the presentation data. Each can expose its current state relatively directly for reading by any part of the system that needs to know. Each can require changes of state to be made through a defined interface, which might be some sort of command/action handler pattern to keep the design open and flexible. Each can be observable, so other parts of the system can react to changes in its state.
It just happens that now, instead of a single cycle where application data gets rendered and changes from the UI get passed back to the application data layer, we have two cycles. One goes from application data through presentation data to rendering, with changes going back to the application data layer. The other just goes from the presentation data to the rendering and back.
I have found this kind of architecture “plays nicely” with almost any other requirements. For example, data sync with a server if our GUI is the front end of a web app can easily be handled in a separate module elsewhere in the system. It can observe the application data to know when to upload changes. It can update the application data according to any downloaded information via the usual interface for changing the state.
I have also found this kind of architecture to be very flexible and to fit with almost any choice of libraries for things like state modelling, server comms, rendering the actual view, etc. Or, if your requirements are either too simple to justify bringing in those dependencies or too complicated for a ready-made library to do what you need, you have a systematic overall structure in place to implement whatever you need directly, using all the same software design and scaling techniques you normally would.
Running = -1
WHILE Running
PRINT "1) Add record"
IF HasRecords THEN
PRINT "2) Delete record"
PRINT "3) Edit record"
END IF
PRINT "H) Help"
PRINT "X) Exit"
INPUT "Choice: ", I$
IF I$="1" THEN AddRecord
IF (I$="2") AND HasRecords THEN DeleteRecord
IF (I$="3") AND HasRecords THEN EditRecord
IF I$="H" THEN ShowHelp
IF I$="X" THEN Running = 0
WEND
...which, sure, it is ridiculously simple (like imguis) but it can very quickly become unwieldy the more you ask from your user interface. There is a reason why UIs moved beyond that.[1] https://github.com/0xafbf/aether/tree/master/addons/godot_im...
Does it? Computers are FAST. Webdev stacks are many things, but it ain’t fast or performant.
Has anyone built a moderately complex UI in a good immediate mode UI and in a good retained UI? I’d be very curious to know the actual results.
That’s how react works. It’s effectively an ‘immediate mode’ abstraction above a component based UI.
That avoids the component connecting and wiring problems, and creates the simple determinism that makes immediate mode systems easier to reason about.
I like the current trend of going back to renderless components as well. This way you separate the state changes from the way it looks like. Feels like each component is a miniature MVC framework with front and a back.
In fact, it can actually be less work, since you're coalescing changes into a single update() call rather than sprinkling them across observer callbacks. Also, if your update function starts running too slowly, you can always make it more precise by keeping track of which states have changed internally to the view. For example, if setting the background color takes a long time for whatever reason, you can do something like this:
def View.update():
if self.lightWasTurnedOn != model.lightTurnedOn:
if model.lightTurnedOn:
self.backgroundColor = red
else:
self.backgroundColor = ibmGray
self.lightWasTurnedOn = model.lightTurnedOn
Now backgroundColor will only be set if lightTurnedOn actually changed since the last update.Several toolkits and frameworks provide for "after all other events have been processed" hooks/events for such logic, e.g. Delphi has the TApplication.OnIdle event and later versions as well as Lazarus/FreePascal have dedicated controls for this event and "idle timers" meant to be used for updating any invalidated parts of the UI after all other events have finished. Similarly wxWidgets has wxEVT_UPDATE_UI and i'm almost certain that Qt has something similar too - though i can't find it now.
Note that in this case, most modern framework actually focus on what Microsoft called MVVM in Silverlight (Model View View-Model), a lesser strict "clean controller" approach where view and controller are tangled in each other since it's throw away code for the most part, while the model stays clean.
Focus is the hardest one, because that can move around really erratically. But even that sometimes has to be done in state. IE11 doesn’t have descendant focus selectors so at my last job I wrote a hook for our dropdown children to dump their focus state into a React context so our renderers could stay pure and not rely on CSS and therefore DOM state.
Just last week I implemented a hook to reposition a tooltip based on the mouse cursor.
You do need to think about DOM state a little when doing these things. But I would argue that’s somewhat separate from the activity of building a React UI. It’s pretty rare you can’t just render purely based on the React state, using off-the-shelf hooks.
So, I think that you're both not wrong: State tends to be one of the (if not the) fundamental problems in most of programming. Often, looking at things from another perspective is very helpful. And we should be uncovering new strategies to do that for all places where we haven't, including the GUI.
Of course, that also means that we arrive at the meta-problem of synchronizing the state backing various systems at one point. Which mostly encapsulates Karlton's "two hard things": Cache invalidation and naming things. :)
> The fundamental challenge of GUIs is that they have state as a core concern. Unlike most systems, state is not an implementation detail that can be refactored away.
I do talk about "unifying state"; eliminating "accidental" state. But any UI that allows for interaction also has essential state which can't be refactored away.
We don't play around with event sourcing right now, but it might be feasible at and above the UI state services if we wanted to do things like replay a user interaction with our app. The only caveat is that our domain services are very very messy in terms of side effects, so the practical utility is bounded to isolated UI testing.
I really enjoyed learning about the hierarchical state machines using statecharts.
There are just so many problems with MobX. For example reactions - ever tried figuring out why some value changes in a bigger app? Not to mention abysmal tooling (compared to Redux DevTools). But the biggest problem is that everything is... sprinkled with magic. Just give me Redux(-toolkit), thank you very much, I can understand these much more deeply. /rant
If I sound confrontational, sorry about that... I just had the misfortune of inheriting a MobX app. Magic indeed.
I introduced MobX (along with HTML5 UI, React, TypeScript, etc) to a large team and it went swimmingly. Debugging/stepping-through was no more tedious than regular Chrome DevTools debugging.
I recommend using MobX’s strictest setting (where changing reactive objects can only occur in named actions), and restricting reactivity to a monolithic “model” (mdl) object and its children.
It really does make writing fast, correct & good UI a breeze, in my experience.
If it comes to the question. I think you have answered yourself by writing React won't stop you from making bad design decisions.
I can state from experience that using Dear ImGui over Qt massively sped up the application we developed (Media player with non-trivial UI that decoded RAW video in real-time using CUDA/Metal/OpenCL). Framerate increased by around 10%, and there was no longer any significant difference between the consumer-oriented software and an internal engineering tool that was literally nothing but an SDL window and some direct API calls.
Perhaps caching was performed driver-side, or perhaps maintaining indexing is not a huge issue on modern GPUs. I couldn't tell you, honestly, but I saw the performance gains with my own eyes. Entire UIs would chew up less resources than a blank QWindow.
The RMGUIs (should?) be much more efficient when the contents are both hard to render and don't change much -- think large amount of styled text, or detailed SVG diagram, or a graph.
(and blank QWindow consumes 0% CPU when there are no screen updates -- this would be hard to beat with IMGUI's even with the best GPU)
I already mentioned this: "many RMGUIs have heavy styling, visually incomparable to their IMGUI counterparts, which makes the average RMGUI far heavier". Dear IMGUI will be faster than the average RMGUI implementation. It's doing way less work. In addition to being cleaner internally.
For simple use cases, Dear IMGUI behaves like an RMGUI internally. (flohofwoe mentions this in another thread.) So its only performance impact is some minor caching overhead, which is far smaller than extra GPU work.
This is actually a fascinating example: https://lucywang000.github.io/clj-statecharts/docs/get-start...
... because it highlights precisely my criticism.
;; define the machine
(def machine
(fsm/machine
{:id :lights
:initial :red
:context nil
:states
{:green {:on
{:timer {:target :yellow
:actions (fn [& _]
(println "transitioned to :yellow!"))
}}}
:yellow {:on
{:timer :red}}
:red {:on
{:timer :green}}}
:on {:power-outage :red}
}))
I just ... don't understand why anyone would do it this way. The code itself already says what to do. Adding this sort of data only subtracts from clarity with no additional flexibility.You might argue that the data model makes it flexible. But I look at that and go, that's what a function is for. The only thing you need is `(println "transitioned to :yellow!")` inside of a function called transition-to-yellow, or if you're feeling adventurous, a function called transition-to which takes yellow as an argument.
It's much easier for me to introspect, and I can easily build dynamic state-machines by changing a data structure, take a look at interceptors[0] as an example. There the stack is dynamically alterable based on what is within the request and each piece of middleware can look at the current context, analyse it and behave accordingly.
I write a lot of workflow based systems with dynamically changing functionality based on user input. This sort of thing is invaluable in that context.
If you're just printing some text to the console when doing one thing, then surely one function is enough. Once you start having some state and different views depending on values, normal state machines might be enough. But eventually, your application might grow so much in scope that you want to have state machines nested in other state machines, and you want to avoid the classic "state explosion" that you end up with when using state machines, then Statecharts are a very useful tool.
But before that, do the simplest thing that can work for your use case, complexity is evil and all that... Statecharts are simply a tool for handling complex and intertwined state, not for simple stoplights. Unless those stoplights are all connected to each other and also control/read other state machines, then Statecharts might be right for you.
Not sure if you took a look at the links I put before, but the https://statecharts.github.io/ website has a "Why should you use statecharts?" section with small descriptions and links to more elaborate information as well. Might give you a better view than what I can give.
And it's nothing but a long list of functions that use closures.
I did take a look at your examples. I just ... well, we'll have to agree to disagree, and I'll try not to be so harsh in my criticisms of what I perceive to be ugly ideas. Just because I think an idea is bad, doesn't mean it's bad.
But there is one specific critique: the state machine approach will make programs longer, and consciously choosing to make programs longer seems fraught with danger. Every additional character you type is an additional character for bugs to hide in.
This moves the subjective "I don't like the style" to something concrete: What's the shortest program you can write with a state machine? My argument is that it's "significantly longer than the equivalent program without a state machine."
Circular dependencies are an unfortunate fact of life, and I wish we had tools to deal with them, instead of desperately trying to avoid them.
A good test for a UI paradigm is, how do you handle a UI like this:
A = [50] [====| ]
B = [10] [| ]
C = [60] [=====| ]
A + B = C
Where [===| ] things are sliders, all three are changeable, and the relation A+B=C must always hold.The easiest solution is to maintain a fixed order of preference, something like:
values = [a:0, b:0, c:0]
fn onChange(key, value) {
values[key] = value
for (k,v) in values {
if (k !== key) values[k] = adjust(k)
}
}
fn adjust(key) {
switch(key) {
case 'a': return max(0, values[c] - values[b])
case 'b': return max(0, values[c] - values[a])
case 'c': return values[a] + values[b]
}
}
The alternative is to maintain the latest selections made by the user and use that as the iteration order.Whatever approach you go with, the "single source of truth" approach of react/vue/svelte + state (whether it is state in hooks, redux, mobx or whatever) holds. The "values" above is the source of truth, and the components just reflect that.
In other words: from a state point of view you don't have three sliders each with a value but a "three-valued component that happens to be shown as three sliders".
Then that some people make React components change state on render/mount/update, is a different failure, but not really a failure of React as much as a failure of the one using React.
But in general no, React doesn't really prevent you from having a component updating and re-rendering because some state changed, and because of that, changes the state again and thus re-renders again, forever.
If you take a look at the syllabus of the course, it's not about JavaScript, it's about the formalism and understanding the core concepts, without locking you to a particular technology. Whatever they are teaching in the course, you can apply to JavaScript/Swift as much as you can apply it to Desktop/Mobile.
Disclaimer: haven't actually taken the course, but planning to and I've read the description of it.
Until this process is made easy this will always just be a fever dream of fools in ivory towers
You get people to adopt technology or paradigms by explaining the benefits and drawbacks of adopting that set of technology/paradigm and then discussing it together with your team/s. Not sure why it would be different for Statecharts compared to any other paradigm?
What process is too hard for you now exactly, to describe states or something? You're already doing this implicitly when you write UI code with state. Statecharts only changes it from being implicit to being explicit. If you're having a hard time actually naming your states, you can use tools like https://sketch.systems/ to explore what you're building with a simple DSL, then implement it properly in the programming language of your choice.
> laying out a diagram
I think layout belongs to the views (you call them presentation rendering).
View models provide data to be visualized, but it’s rarely a good idea to compute pixel positions, or handle low-level input there. These things are coupled with views too much.
Similar to animations. Unless the app being developed is a video editor or similar where animation is the core functionality, animations don’t affect models nor view models, they merely prettify GUI state transitions or provide progress indication. No need to wire them all the way into view models. Some frameworks even run them on a separate thread called “compositor thread” so they play fine even if the main thread is blocked or starved for CPU time.
At first when I adopted this architectural pattern, it was a development of classic MVC so already used the terms “model” and “view”. I actually did adopt the term “view-model” for what I’m referring to here as the presentation data layer. After all, it was the connection between the view and model, so what else should it be called? :-)
However, while there certainly are similarities between the architecture I described above and MVVM — the VM can collect data from the Model and reformat it for the View, and the VM might manage some aspects of the view state — they also differ in some important ways.
Most significantly, MVVM has a linear V–VM–M relationship between the components. The V and M never communicate directly in either direction.
In the architecture I described, the rendering code might depend on both state layers, accessing both application state directly and derived or view-specific data from the presentation state. This avoids any need for boilerplate in the middle layer if nothing special needs to be done with the application data before it’s used for rendering.
Likewise, in response to events triggered from something in the rendered output, messages might be sent to the application and/or presentation state layers so they could update accordingly. The distinction is always clear: any event that can affect application data gets sent there, while anything that affects view state goes to the presentation data layer. In practice, it’s unusual for a single UI event to affect both, so usually only one message is needed. (There is an implicit assumption being made here that the starting point for handling events triggered by user interactions is defined as part of the rendering, which is not necessarily always the case, but let’s gloss over those details for now or this comment will get insanely long.)
In MVVM, there may also be direct two-way data-binding between the V and VM. The architecture I described never assumes that kind of 1:1 relationship between something in the rendered output and the underlying data.
I think layout belongs to the views (you call them presentation rendering). View models provide data to be visualized, but it’s rarely a good idea to compute pixel positions, or handle low-level input there. These things are coupled with views too much.
This is another place where I think the responsibilities in the architecture I described are allocated differently. I want any non-trivial calculations to be in the presentation data layer, and I want the presentation rendering layer to be as dumb as possible. This separates data processing from I/O, which is something I favour as a general principle for organising programs.
In this case, it has the specific advantage that you might have very different strategies for testing them. The presentation data layer is pure computation and can be easily unit tested. The I/O is all about rendering and, in some cases, simulating actions on the rendered output that trigger some sort of message to be sent, so for this you might want some sort of snapshot-based testing or simulated UI environment.
You might also want to incorporate totally different third party libraries or depend on different platform APIs to implement each step. For example, switching out one UI rendering library for another or updating to integrate with some new-style platform API probably shouldn’t affect any of the data (including presentation data like diagram layouts or animation logic) unless we’re specifically shifting responsibility for something like handling animations to a dependency rather than our own code or vice versa.
Some frameworks even run them on a separate thread called “compositor thread” so they play fine even if the main thread is blocked or starved for CPU time.
Sure. One of the nice things about the architecture I described is that it’s entirely neutral about these things. If your requirements so dictate, there is nothing to stop you implementing a more elaborate design at any level of the system. For example, that might include running time-consuming parts of the presentation data logic in separate threads, having changes in the presentation data triggered by timers or other system events, delegating some aspects to hardware acceleration where available, or using any sort of sophisticated caching or scheduling implementation for better performance.
Much of this would add unnecessary complexity in a simple CRUD application GUI. If you’re building a complex, real-time visualisation of an incoming data stream or you’re implementing a game engine that needs to draw part of a large game world with many moving parts, presumably you might have a different perspective on how much extra complexity in the design is acceptable if it gets you the performance you need.
There’s nothing wrong in dropping a layer there.
That intermediate VM layer makes sense when the data being presented is not precisely what’s stored in the model. Good place to implement filters or pagination. Good place to validate user input. However, in other cases one doesn’t need any of that, in which case there’s nothing wrong with data binding directly to models.
> the rendering code might depend on both state layers, accessing both application state directly and derived or view-specific data from the presentation state.
In .NET with MVVM that’s easily doable, expose a property on VM that returns the model, and this will work.
> In practice, it’s unusual for a single UI event to affect both
In the apps I work on this happens all the time. User clicks “do something” button, the model needs to start doing that, in the meantime the view needs to disable that button and show “doing something, please wait…” with a progress bar.
For this reason, I normally handle such higher-level events in VMs and call models from there.
> The presentation data layer is pure computation and can be easily unit tested.
I think strongly typed languages handle 90% of what people usually unit test for, with better developer’s UX. If you change something that broke 33 places of other code, a compiler will tell you precisely which 33 places you gonna need to fix.
> switching out one UI rendering library for another or updating to integrate with some new-style platform API probably shouldn’t affect any of the data
I think that only works when switching to something extremely similar. I did when porting iPhone code to iPad, or WPF XAML to UWP XAML, but these were very similar platforms. UI libraries are complicated; replacing them with something else is huge amount of work.
> including presentation data like diagram layouts or animation logic
I don’t believe these pieces are portable in practice. Both coupled with UI libraries too much. For the diagram, half of the 2D rendering libraries are immediate mode other half are retained mode. Animations are totally different as well, CSS animations in HTML don’t have much in common with visual state manager in XAML.
The user-side code basically describes what the UI should look like in the current frame, those "instructions" are recorded, and this recording is reasonably cheap.
The UI backend can then figure out how to "diff" the new instruction stream against the current internal state and render this with the least changes to the screen.
However some immediate mode UI systems came to the conclusion that it might actually be cheaper to just render most things from scratch instead of spending lots of processing resources to figure out what needs to be updated.
In conclusion: "Immediate Mode UI" doesn't say anything how the UI is actually rendered or generally how the internals are implemented, it only describes how the public API works.
If the public API requires you to give a new paint command on every frame (everytime the scrollbar is dragged), then regardless of whether the underlying rendering engine performs each of these paint commands, you still have to run through every item of the list (and so does the diff'ing code), making this a O(N) operation on every frame.
When you call ImGui::Button("Hello World") it has no way of telling if it’s the same button as on the previous frame, or a new one. You’re simply not providing that data to the framework.
It’s often good enough for rendering, they don’t really care if it’s an old or new button as long as the GPU renders identical pixels. When you need state however (animations certainly do), the approach is no longer useful.
A framework might use heuristics like match text of the button or relative order of things, but none of them is reliable enough. Buttons may change text, they may be dynamically shown or hidden, things may be reordered in runtime.
> Layout problems can be solved by running several passes over the UI description before rendering happens.
You’re correct that it’s technically solvable. It’s just becomes computationally expensive for complex GUIs. You can’t reliably attach state to visual elements, gonna have to re-measure them every frame.
> There need to be accessibility APIs which let user interface frameworks connect to screen readers
Windows has it for decades now. For Win32 everything is automatic and implemented by the OS. Good custom-painted GUI frameworks like WPF or UWP handle WM_GETOBJECT message and return IAccessible COM interface. That COM interface implements reflection of the complete GUI, exposing an objects hierarchy. I’m not saying it’s impossible to do with immediate rendering, just very hard without explicit visual tree that persists through the lifetime of the window.
ImGui does have stable widget ids, otherwise many features which require keeping state across frames wouldn't work. In case of your button example, the widget id is hashed from the label string "Hello World":
https://github.com/ocornut/imgui/blob/61b19489f1ba35934d9114...
This is just the default behaviour, there's also a convention to define the id separately from the visible label string behind a '##':
https://github.com/ocornut/imgui/blob/61b19489f1ba35934d9114...
Of course other immediate mode UIs might have a different way to define widget ids.
Can you solve this by adding stable IDs to the API, so you'd call `ImGui::Button("my-button-id", button_text)`?
Who and when should destroy backend visual nodes? If you won’t use an ID in some frame, does it indicate the backend should drop the related state? What if an animation is still playing on the element in question? What if the missing control re-appears later i.e. was only hidden temporarily? What should backend do if you reuse same ID for different things?
One can implement simple stuff using any approach, immediate GUI is often the simplest of them. It’s when use cases are complicated you need a full-blown retrained mode OO framework. Win32, HTML DOM, MS XAML, QT, UIKit are all implemented that way, that’s not a coincidence.
But otherwise, yes, just use a framework that separates State from UI and has bidirectional updates, like SwiftUI.
For example from a quick look at the code it looks like this project needs to draw in a bunch of unrelated to each other libraries just to get some basic GUI functionality you'd find even in toolkits from the 80s provide (e.g. drawing text or using file dialogs).
And check these sources [0] and [1], this is doing at least as much bookkeeping as in a "retained" GUI toolkit - except it also has to do things that such a toolkit would do automatically, like explicitly drawing the widgets [2]. People at the past were complaining how tools like Visual Basic were mixing presentation and logic, yet the worst you could do in these tools is to have the logic in event handlers (...which if you think a bit about it at least that does make a bit of sense), yet here is actual application logic being part of the drawing code in [3] (this is inside a method View::DrawPlayback, scroll a bit upwards to find it).
Now sure, someone might say that this isn't the best example of immediate GUIs... but this is the one that was brought up as a good example. And TBH all it convinced me is that if (for some reason) i had to use ImGui (the library), to spend some time building a retained GUI wrapper around it :-P.
[0] https://github.com/wolfpld/tracy/blob/master/server/TracySou...
[1] https://github.com/wolfpld/tracy/blob/master/server/TracyVie...
[2] https://github.com/wolfpld/tracy/blob/d8c1dae9e120c27801e762...
[3] https://github.com/wolfpld/tracy/blob/d8c1dae9e120c27801e762...
I sincerely hope no one looks at the screenshot on that page and thinks "this is something that works well". This UI screams "I've been made that way because that was the easiest way in the tool I'm built with" (which is bad - good tools should not dictate the form of the resulting product)
That's where you'd be wrong. Profilers traditionally have looked like this regardless of the tool/GUI lib/GUI paradigm they'd been made with.
It's the domain need (to show lots of information, graphs, timelines, stack trees, etc, to compare) that asks for this...
the profiler I use most (hotspot) definitely looks much cleaner: https://www.youtube.com/watch?v=6ogEkQ-vKt4
And why do you think the UI looks that way because of restrictions of the UI framework?
I'm quite sure if the UI would have been written with (for instance) Qt or React, it would look exactly the same, because the UI requirements would be the same (minus the different default style). The question is whether this could have been achieved with less, or "cleaner" code (which I doubt).
I don't think the chosen GUI library causes the other UI issues you perceive. The author probably hasn't put much time into learning visual design, and thus fell into whatever the UI library supported. But to produce a better one, the author would have had to do a lot more learning and thinking, which is independent of the library choice.
Not a distinction made because of immediate vs retained GUI.
1. writing code adds “debt” 2. Your solution is to now add state charts too which also adds “debt”
Where are these state charts tracked? Who maintains them? When product asks engineering to change code => you now also update state charts. Added technical debt.
If this is difficult for you to understand (the problem I’m describing is very common at large companies) I’m happy to expand more on it.
Thoughts?
Yes, writing code can add debt, but not all code is debt. "Debty" code is code that can be better, but was needed in order to take shortcuts temporarily, probably in order to trade moving faster now against moving faster in the future. If you're taking shortcuts you're gonna have to fix it in the future, or deal with having to move slower/more careful because of it.
And yes, the solution to code debt is to go back and fix it properly. When it comes to UI programming, I'd argue that doing implicit state machines with conditionals scattered all over the place (which is the de facto standard today everywhere I look), is code debt, which can be fixed by REPLACING it with explicit Statecharts. It can also be fixed in other ways obviously, but besides the point here.
The developers would obviously be responsible for the code they write and the Statecharts are all handled in the same source control system your developers already use (typically Git today), so nothing really changes here.
And yes, when you figure out that you have to change the code to do something different, you're gonna have to ask developer to change the code. The same goes for updating Statecharts (that also exist in the code). If you have to change the states/transitions, you're gonna have to update the code that handles the states/transitions. This is the same no matter if you use Statecharts or not.
In the end, Statecharts is not a programming language itself, it's just a way of doing programming. Basically like how functional programming is a way of programming, or object oriented programming is one way, Statecharts is a different way where you're upfront with what states exist in the application.
I say this as a FAN of Statecharts and functional programming. I myself love this idea. I am looking and prodding people like you for SOLUTIONS to make teams adopt this.
So far you’ve failed to convince me on how to sell this to teams or make it easy to integrate.
Developers, probably. But generally, whoever your team/company assigns this task to.
> When product asks engineering to change code => you now also update state charts. Added technical debt.
So is documentation. I would hope a team handles this somehow.
But if you approach it as a problem you actually want to solve, here's an idea: have the canonical statechart in code, and have the visual statechart be generated from that code. Benefits include: your visual/documentation charts are always synchronized with actual code; if somebody starts playing fast and loose with statecharts in code, the picture generator will break - if you treat this as a build error, you've just removed one common source of project rot.
In the apps I work on this happens all the time. User clicks “do something” button, the model needs to start doing that, in the meantime the view needs to disable that button and show “doing something, please wait…” with a progress bar. For this reason, I normally handle such higher-level events in VMs and call models from there.
I used too strong a word before. As you point out, situations where you’d want to update both the application and presentation data aren’t really unusual. You gave one good example. Another might be controlling a modal UI, like closing a dialog box or moving to the next step of a wizard, when maybe you also want to commit data user has entered in a temporary form to the permanent application state. In these cases, handling a UI event would indeed need to get the relevant information to both of the other layers one way or another. This still easily fits within the architecture pattern I was describing, though.
I think strongly typed languages handle 90% of what people usually unit test for, with better developer’s UX.
I am a big fan of using expressive type systems to prevent defects, particularly as an alternative to huge unit test suites full of boilerplate that still don’t do as good a job.
However, no mainstream type system will verify that for a given list in your application data and a given sorting option chosen in the UI, a function in your presentation data layer has generated the correct top 10 items in order. (No unit test can ever verify that property in general either, of course, but at least we can test some representative example cases.)
I think that only works when switching to something extremely similar. I did when porting iPhone code to iPad, or WPF XAML to UWP XAML, but these were very similar platforms. UI libraries are complicated; replacing them with something else is huge amount of work.
I suppose that depends on what you consider extremely similar. Around 10–15 years ago, when what we call “single page applications” today were starting to gain serious traction, if you wanted an interactive visual representation of your data, you probably used one of the proprietary embedded technologies like Flash, Java or ActiveX. But if you were drawing, say, an org chart, you were still ultimately just putting boxes and lines and text in certain places within the allocated area on the page. Today we have tools like SVG, canvas and WebGL available handle the drawing part, but everything we’d want to display in that drawing might still be the same.
I don’t believe these pieces are portable in practice. Both coupled with UI libraries too much. For the diagram, half of the 2D rendering libraries are immediate mode other half are retained mode. Animations are totally different as well, CSS animations in HTML don’t have much in common with visual state manager in XAML.
I think perhaps we have very different types of animation in mind. From the above, I’m guessing you’re thinking of things like pulsing a button when it’s pressed or a little progress spinner while downloading some data? I’m thinking of things like smoothly moving all of the boxes on that org chart to their new positions, including animating the shapes of the paths between them, when the user does something that moves the chart around. Another example might be animating weather data overlaid over a map graphic through the next few hours when the user presses a “play” button. That is, I’m talking about animations where the data to be shown at each step needs to be determined according to complex requirements, not just playing a simple, predetermined effect that a UI library or platform API might handle for you almost for free anyway.
Just because you can doesn't mean you should.
> a function in your presentation data layer has generated the correct top 10 items in order
There're too many things which can go wrong with visuals. Incorrect order has smaller probability than a bug in a style somewhere which makes data invisible. For the 2nd reason you want to test that thing somehow else anyway: unit tests don't catch rendering bugs. Some other automated tests can in theory, but in practice maintaining these tests wastes a lot of time i.e. expensive.
> Around 10–15 years ago, when what we call “single page applications” today were starting to gain serious traction
Traction is overrated. Around 20 years ago Outlook Web Access (a server-side component of Exchange server) already was a modern single page application, written in HTML and JavaScript. All the required functionality was already there in IE, and even documented nicely on MSDN.
> everything we’d want to display in that drawing might still be the same.
Totally different. SVG renders vector 2D graphics, WebGL is a 3D GPU API. Try to render something slightly more complicated than rectangle of solid color, and you'll see. One example is rectangle with rounded corners: trivially simple in SVG, relatively hard on GPU. A Mandelbrot set is opposite example.
> animations where the data to be shown at each step needs to be determined according to complex requirements
Doesn't matter if the data is static or not. Once you know the data, you do not want to run your code at 60 Hz (or 240Hz if you're really unlucky) computing functions of time and updating things. It's almost always a good idea to use the GUI framework for playing animations. They do better with rendering and power saving.
Sure, but that’s not an argument for not doing it either. As I’ve been trying to explain, I’ve had practical success building applications using the kind of software architecture I described.
I don’t like to get into personal backgrounds in these discussions, because as a rule I think arguments should stand on their own merit. However, just for some perspective, I wrote my first professional GUI around three decades ago. More to the immediate point, the longest I’ve maintained and extended a single GUI built using the architectural pattern I described was over a decade, and that one was probably more complicated in its requirements than a lot of web applications you’d see today. So although it’s just one anecdotal data point, I think it’s a good demonstration that this kind of architecture can work well over an extended period.
There're too many things which can go wrong with visuals. Incorrect order has smaller probability than a bug in a style somewhere which makes data invisible. For the 2nd reason you want to test that thing somehow else anyway: unit tests don't catch rendering bugs.
You seem to be making some implicit assumptions about the platform here with the reference to styling. In any case, this is why I mentioned that you might want a different testing strategy for the presentation rendering layer to the code that collects the presentation data.
Totally different. SVG renders vector 2D graphics, WebGL is a 3D GPU API. Try to render something slightly more complicated than rectangle of solid color, and you'll see.
Sorry, I’m not sure I understand the point you’re trying to make here. I was saying that web-based GUIs might have moved from rendering graphical content using one of the plugins a few years ago to rendering it using one of the native technologies today, without necessarily changing the content itself. Obviously which native technology you’d choose would depend on what kind of visuals you were creating and how you wanted any interactions with them to work.
Once you know the data, you do not want to run your code at 60 Hz (or 240Hz if you're really unlucky) computing functions of time and updating things. It's almost always a good idea to use the GUI framework for playing animations. They do better with rendering and power saving.
To the best of my knowledge, the GUI framework you are describing does not exist. You might be able to extract some basic movements that could be handed off to something running at a lower level with hardware acceleration, but ultimately you still need code to define the way the layout should animate according to whatever rules your system uses, for the same reason you need to define a static layout before or after the animation that way.
I’m not sure what you’re getting at with “doing better with rendering and power saving”. Can you clarify?
Yeah, because I'm not trying to convince you of anything. It's a tool in your toolbox, use it when you think it's advantageous, otherwise use your other tools. I couldn't care less of what you chose to do or how you "sell this to teams". I'm a developer who simply chooses the best tool for the job, sometimes that's Statecharts and sometimes it's something else. Also don't have any idea about what ideas are circling around in "Academia" as I'm far away from that ecosystem and only focus on shipping working products.
If you're looking for something "easy" in particular, then whatever you don't know is gonna be hard. Such is the life of a developer, where sometimes the simpler way is harder but worth it in the end. React was hard for people to grok in the beginning as well, but that doesn't mean it's bad, it just means people are not familiar with it. If you're just looking for easy solutions then yeah, feel free to stop improving and continue use the stuff you already know or is familiar to you.
So in the end, do what you will with the knowledge and experience I've shared with you, I have zero interest in selling you anything and I'm simply discussing stuff here on HN as I'm curious about things and want to have discussions with people who are also curious about things, but this discussion stopped being productive a long time ago.
But I guess different UI frameworks have different solution for this. Creating and updating a 1 million item list wouldn't be a cheap operation in a traditional UI system either.
Concrete examples of code that is not immediate mode api, vs. code that is immediate mode api, both implementing the same thing, can help discuss / reflect on that fear. IME immediate mode is great for mapping tree-ish data to tree-ish UI; when things have a lot of nontree linkages or reordering (eg. when you do drag and drop) it gets trickier. React's JSX / createElement API also feels somewhat immediate mode, tbh; the updates are just scheduled to fire on state changes.
IDK, works pretty well with Qt's item model system
So it does it just like an immediate mode GUI does it, and only renders what it must.
Not sure about QT specifically, but all other non-immediate mode GUIs I know use the same trick, and immediate mode GUIs like React also use it.
This sounds a lot like paintEvent() in traditional OO-style GUI systems; i.e. event-driven.
So my understanding now is that with immediate-mode callbacks happen within the scope of a function-call rather than from some event-loop. I probably have to look into it deeper to get a good understanding. It is still unclear where state is stored (e.g. for the position of a scroll-view), and if state is passed through the function call tree on every frame.
re: widget-local state -- React is one of the most popular models for that. Basically if widgets are identified over time by the sequence of ids (includes array indices like React keys) on the path to them from the root, state is coherent across that identity, and mount / unmount events exist at the boundary of the identity. Callstack / push-pop based APIs like ImGUI maintain this sequence either impicitly or explicitly in the stack. Then there is some API to read / write to the state store (like React hooks or ImGUI generic state storage) with optional update triggering in async APIs like React's.
I think for me the figure is about 8 years of development of 1 GUI software, with MVVM and .NET.
> implicit assumptions about the platform here with the reference to styling
It's 2021, pretty much all of them support styling now.
> I was saying that web-based GUIs might have moved from rendering graphical content using one of the plugins a few years ago to rendering it using one of the native technologies today, without necessarily changing the content itself.
None of them were able to reuse much code while switching just the renderer: ActiveX written in C++, Flash in ActionScript.
> the GUI framework you are describing does not exist
WPF and UWP definitely exist. I think modern HTML+CSS can do that too, but I ain't a web nor electron developer and not sure.
> ultimately you still need code to define the way the layout should animate according to whatever rules your system uses
It's the matter how you define them. When you offload playing animations to a framework, you tell the framework which properties to animate and how: define function of time (typically supported ones include step functions, polylines, and Bezier), pass metadata about timing and repeats, etc. That's for the hard case when animations depend on the data, when they do not they're made offline by art people.
> Can you clarify?
When you do them manually, i.e. computing functions of time and changing visuals, following happens.
1. Your code runs at refresh rate of the monitor, typically 60Hz, but for high-end gaming monitors can be 240Hz. Even if you aren't animating anything, this alone prevents certain power saving modes from engaging.
2. If you manually change positions of things not just color, your updates gonna invalidate layout.
3. There're multiple ways of measuring time with different tradeoffs about precision and when it's running versus paused.
A framework is in a better position because it drives the main loop. It knows how many nanoseconds passed since the last frame. It knows when to render at 240Hz and when to sleep on GetMessage() because there's nothing to update. If it knows which properties are animated, it can integrate animations with layout for reasonable performance cost.
Great. As I said before, there are many different architectures that can work well.
None of them were able to reuse much code while switching just the renderer: ActiveX written in C++, Flash in ActionScript.
But if you’re calling that renderer from JavaScript that has already done everything except the rendering, which would be the case with the kind of architecture I was describing if you were just using the plugin for the final rendering layer…
It's the matter how you define them. When you offload playing animations to a framework, you tell the framework which properties to animate and how: define function of time (typically supported ones include step functions, polylines, and Bezier), pass metadata about timing and repeats, etc. That's for the hard case when animations depend on the data, when they do not they're made offline by art people.
Right. But where do you get those data-driven paths from so you can use them as your key frames to drive your animation? That is what is generated in the presentation data layer in this example. No GUI framework can do this for you, because no GUI framework knows the rules of your system for how to lay out a particular diagram and how a transition between states should appear.
Actually moving lines or rotating text or whatever is then little more than an implementation detail, which can be done manually by the presentation layers or handed off to some platform-provided animation system, as the situation dictates.
Your code runs at refresh rate of the monitor, typically 60Hz, but for high-end gaming monitors can be 240Hz. Even if you aren't animating anything, this alone prevents certain power saving modes from engaging.
This seems like such an extreme case that it’s no longer very useful as an example. Who is using a 240Hz high-end gaming monitor, wanting our UI to produce animations that can keep up with that frequency of updates, yet concerned about some unspecified power saving mode not engaging if we do the maths manually when it would have engaged if the work had been delegated to some system service?
If you manually change positions of things not just color, your updates gonna invalidate layout.
We’re talking about immediate mode rendering. What “layout” is there to invalidate?
Writing such renderers is huge amount of work. I know because I once did something similar: https://github.com/Const-me/Vrmac When I can, I prefer someone else to do that. Unfortunately, they all do it differently. Not just in terms of pixels on output, but general things.
Like, compare SVG and PostScript – many features are common, yet you gonna need substantially different things on input to make these two formats with the same image.
It’s somewhat similar story about rendering libraries.
> No GUI framework can do this for you
Sometimes XAML can. I remember there was a checkbox in Blend for some target platforms called “fluid layout” or something, the framework automatically replaced some discrete changes with auto-generated animations. Not universally applicable, but sometimes it was good enough.
> Who is using a 240Hz high-end gaming monitor .. yet concerned about some unspecified power saving
On the first page of US Amazon bestsellers in the “Laptops” category there’re a few models with 144Hz displays. These people care, occasionally.
I can agree that was an extreme case, but to lesser extent that applies to laptop users in general. I think laptops have been outselling desktops for a decade now. People don’t like choppy animations; they cause otherwise good GUI to be perceived slow. They don’t like spinning/noisy fans and reduced battery life either.
Not so much, if you’re only asking them to do the last step of translating information you already have into whatever format/APIs are needed for a particular medium, which is essentially what I’m advocating with the architecture here.
Long-lived web apps have had to deal with this several times in recent years as the various plugin technologies have been deprecated and ultimately removed from browsers. It obviously takes a certain amount of time to reimplement that logic on what might effectively be a new platform, but it’s certainly achievable, and it’s easier than rewriting more of your application because you’ve been forced to change platforms.
Sometimes XAML can.
I think perhaps you’re misunderstanding my argument, because it is literally impossible for any GUI framework to do what I’ve been describing. It absolutely requires application code to specify what animation is to be constructed, based on the data available. A GUI framework can then handle the mechanics of actually running that animation, but it can’t know what the animation should be. That’s an application problem.
Take a simple example. Suppose we want to illustrate a sorting algorithm. We will do this by drawing 10 boxes in a row on the screen, labelled with different data values to be sorted. Without me telling you how I want to animate those boxes from one order to another after each step of the sorting algorithm, how will your XAML or whatever know what animations to draw? Are we fading out the items that will change positions and then fading them back into their new positions? Are we going to animate a moving item up out of the row, then slide the other items along one way as the moving item slides above them the other way, then animate the moving item back down to its final place? If we do that, are we going to do both horizontal slides at the same time, or one and then the other? What if we want a hybrid, where the moving item fades out, then we slide the others that need to shift, then the moving item fades back in at its new position?
There is no way any GUI framework can know what should happen in a situation like this. The required behaviour needs to be specified as part of your code. The GUI framework can handle the mechanics of fading things in and out or sliding shapes from A to B according to some timing curve, but only once you’ve told it what needs to happen.