CherryzZZ Logo
Back to all posts

Intent-Driven UI

A discussion on design thinking for user-facing software UI

UIDesignCode

Introduction

In this era when generative large language models (LLMs) are spreading all over the world, the UI (software interface) design industry is no exception — AI (LLMs) have contributed a remarkable boost in efficiency!

So under these circumstances, when people build software they have started to pay real attention to whether the UI looks good and is easy to operate, because building software with AI is simply too fast — what you need to compete on is innovation and aesthetics!

But modern software UI was not invented out of thin air. It grew out of the physical buttons of early industrial machines being turned into software, and gradually formed a set of design principles and habits that spread to more and more people as electronic devices became ubiquitous.

Because its way of operating is heavily procedural and designer-oriented, software can be made very convenient and beautiful, yet its users may not be programmers or designers at all — they may simply be an ordinary painter, writer, musician, psychologist, and so on.

When they use software, they may spend ages hunting for a button, which seriously degrades the experience and gradually breeds resistance to more advanced software.

So what this article wants to discuss is how software aimed at ordinary users should be designed — how to break away from the UI design philosophy that has persisted for nearly 40 years, and build interfaces that suit ordinary users better.

Professional software inherently requires a procedural way of working, so none of the concepts and design directions proposed in this article apply to professional software!

Where Software UI Came From

If we want to understand why software UI has become increasingly complex, perhaps we first need to return to a more fundamental question:

How did software UI come into existence in the first place?

The windows, icons, menus, buttons, and mice we take for granted today did not exist from the start. They took shape over decades of computing history, gradually forming the graphical user interface (GUI) we know today.

1. Before the GUI: Interacting with Computers Through Commands

Early computers were operated mainly through punched cards, paper tape, control panels, and later text terminals.

In this mode, the computer was more like a machine to be operated:

A person tells the computer which command to execute; the computer executes it and returns the result.

Users had to understand what instructions the computer could accept.

This mode of interaction was highly efficient for professionals, but for ordinary people the biggest barrier was:

You had to learn the computer's language first.

For that reason, early human-computer interaction research began looking for a more direct way to let people interact with the content on a computer through vision and motion.

2. 1963: Sketchpad

In 1963, Ivan Sutherland developed Sketchpad.

It is regarded as one of the important precursors of the graphical user interface.

Sketchpad used a light pen as the input device, allowing users to draw, select, and modify graphic objects directly on the screen.

Image source: Ivan Sutherland and Sketchpad (1962, MIT Lincoln Laboratory), from the Computer History Museum

This differed markedly from traditional command input:

The traditional way:

Tell the computer:
"draw a line"

↓
The computer executes it

Sketchpad was closer to:

See the object on the screen

↓
Manipulate the object directly

This meant a new relationship was beginning to emerge between humans and computers:

Users were no longer just sending commands to the computer — they could directly manipulate the objects on its screen.

Many of the ideas behind Sketchpad later became an important foundation for the development of graphical interaction.

Image sources:

3. 1968: The Mouse and "Direct Manipulation"

In the 1960s, Douglas Engelbart and his team at the Stanford Research Institute (SRI) continued researching more direct ways for humans to interact with computers.

They developed NLS (oN-Line System).

NLS used a mouse, an on-screen pointer, windows, hypertext, and other concepts.

On December 9, 1968, Engelbart gave this system a famous public demonstration.

The demo later became known as:

The Mother of All Demos

Image source: Wikimedia Commons — File:SRI Douglas Engelbart 1968.jpg

In that single demonstration, the mouse, hypertext, windows, and other technologies and ideas that would later become the foundation of modern computing all appeared together.

Modern GUI, therefore, was not invented overnight — during this period a whole series of important interaction ideas gradually took shape.

Image sources:

4. The 1970s: Xerox PARC

The one who truly consolidated these ideas was Xerox's Palo Alto Research Center (PARC).

In 1970, Xerox founded PARC and began a large volume of computer research.

There, researchers continued exploring:

  • Graphic displays
  • The mouse
  • Windows
  • Icons
  • Menus
  • Bitmap displays
  • WYSIWYG
  • The desktop metaphor

These technologies later together formed the foundation of the modern GUI.

5. 1973: Xerox Alto

In 1973, Xerox PARC built the Xerox Alto.

The Alto is a hugely important machine in the history of the graphical user interface.

Image source: Wikimedia Commons — File:Xerox Alto.jpg (photo: Martin Pittenauer, CC BY-SA 2.5)

It had:

  • A bitmap display
  • A mouse
  • A keyboard
  • Windows
  • A graphical interface
  • Visual representations of files and documents
  • The desktop metaphor

It could even make files look like sheets of paper and directories look like folders.

In other words, information inside a computer began, for the first time, to resemble real-world objects.

Users no longer needed to fully understand the computer's internal data structures; instead they could understand the machine through visual metaphors like:

File → Folder → Desktop → Window

Image sources:

6. WIMP: The Basic Shape of the Modern GUI Takes Form

Out of the research at Xerox PARC gradually grew a hugely important system of graphical interaction:

WIMP

That is:

  • Windows
  • Icons
  • Menus
  • Pointer

This paradigm became the foundation of desktop software UI for the decades that followed.

Users moved a pointer with the mouse:

Click an icon → open a window → choose a menu item → execute the action.

The computer's abstract commands were transformed into objects on the screen that could be seen, selected, and operated.

This is one of the most important historical contributions of the GUI:

It turned operating a computer from "memorizing commands" into "seeing and manipulating".

7. 1981: Xerox Star

In 1981, Xerox released the Xerox Star.

Image source: Wikimedia Commons — File:Desktop icons for Xerox Star 8010.jpg

The Star turned a large share of PARC's research into a commercial workstation.

It further reinforced the concepts of:

Desktop, documents, folders, icons, windows, menus

— concepts that remain deeply familiar today.

Although the Star was never a huge commercial success, it had an enormous influence on the later development of the personal-computer GUI.

Image sources:

8. 1983: Apple Lisa

In 1983, Apple released the Lisa.

Image source: Wikimedia Commons — File:Apple Computer, 1983 (Lisa).jpg (photo: Alan Light, CC BY 2.0)

The Lisa brought the graphical user interface, the mouse, and the desktop metaphor into the world of consumer personal computers.

It was never a commercial success and was extremely expensive, but it further proved one thing:

A graphical interface could become the primary way of interacting with a personal computer.

Image sources:

9. 1984: Macintosh

In 1984, Apple released the first Macintosh.

Image source: Wikimedia Commons — File:Computer macintosh 128k, 1984 (all about Apple onlus).jpg

Compared with the Lisa, the Macintosh was cheaper and far more oriented toward ordinary consumers.

It further popularized:

  • The mouse
  • Windows
  • Menus
  • Icons
  • The desktop
  • Folders
  • The trash can
  • Drag and drop

For many ordinary users, the computer became, for the first time, something truly:

understandable and operable through sight.

Image sources:

10. From Then On, the GUI Became the Foundation of Software

By the late 1980s, the graphical user interface had gradually become the dominant mode of interaction for personal-computer software.

For the following decades, software kept building on this framework:

Window
│
├── Menu
├── Toolbar
├── Button
├── Icon
├── Dialog
└── Settings

Software gained more and more capabilities, so the GUI kept adding more:

  • Pages
  • Buttons
  • Menus
  • Toolbars
  • Pop-ups
  • Settings items
  • Levels of hierarchy
  • Parameters

Most of the software UI we are familiar with today can actually be traced back to this system of graphical interaction that took shape between the 1960s and the 1980s.

From Sketchpad to Engelbart's NLS, and on through Xerox PARC, the Alto, the Star, the Lisa, and the Macintosh — modern software UI was not designed in one stroke.

It is the accumulated result of decades of human-computer interaction research.

And what we know today as:

Windows, icons, menus, buttons, settings pages

is precisely the language of interaction this history left behind.

Toward a New UI Design Philosophy

In the previous chapter we looked at where "complex" software UI came from. Software has now entered countless homes along with the spread of electronic devices, but as noted in the introduction, complex UI can be far too cumbersome for ordinary users. So we arrive at the core question of this article: how do you design a UI that loses none of the functionality, yet lets ordinary users get started quickly and work efficiently?

Before the New UI: People Have Always Been Trying to Lower the Cost of Learning Software

Looking back over decades of UI development, you will notice a very interesting pattern:

People have always been trying to make software easier to use.

It is just that most past solutions tried to hide the complexity inside the software, so that users would not need to understand how the computer itself worked.

1. From "Typing Commands" to "Direct Manipulation"

The biggest problem with early computers was that users had to understand the computer's "language".

To accomplish anything, users needed to know which command to type, what arguments it took, and how commands should be combined.

This mode of interaction is very powerful, but it requires users to memorize the software's rules.

Later, a hugely important idea emerged in human-computer interaction:

Direct Manipulation

It no longer asks the user to tell the computer:

"Execute this command."

Instead, it lets the user manipulate the objects on the screen directly.

For example:

  • Want to move a file → drag the file
  • Want to resize a window → drag its edge
  • Want to delete a file → drag it to the trash
  • Want to edit text → click the text and change it

In 1963, Ivan Sutherland's Sketchpad had already demonstrated grabbing, moving, and adjusting graphic objects directly with a light pen; in the 1970s, Xerox PARC developed these ideas further. In 1982, Ben Shneiderman formally coined the term "Direct Manipulation". (CMU School of Computer Science)

Image source: Wikimedia Commons — File:Xerox Star 8010 ViewPoint.jpg

This accomplished a truly significant shift:

From "telling the computer what to do" to "just doing it".

2. Using Real-World Metaphors to Lower the Cost of Understanding

Another hugely important technique is mapping abstract concepts inside the computer onto things people already know from the real world.

The most typical example is the:

Desktop Metaphor

Things like the "desktop", "folders", and the "trash can" do not exist inside a computer.

But designers discovered that if you map the objects in a computer onto real-world items, ordinary users do not need to learn an entirely unfamiliar set of concepts.

And so:

ComputerReal world
DesktopDesk
FolderFolder
FileDocument
TrashTrash can
DocumentPaper document

Xerox PARC developed this design through the 1970s, the Xerox Alto was an important early implementation, and the Xerox Star of 1981 carried the idea into a commercial product. (Wikipedia)

Image source: Wikimedia Commons — File:Desktop icons for Xerox Star 8010.jpg

This approach is very clever.

Because the user does not need to learn:

"There is a hierarchical structure in the file system, and you access objects through paths."

The user only needs to understand:

"This is my desktop, this is my folder, and I put my files in it."

The software's internal structure did not disappear.

It was simply hidden away.

3. From "Opening a Program" to "Opening a Thing"

This was another extremely important design decision of the Xerox Star.

The traditional logic of computing was usually:

Find the program → open the program → then open the file

The Star tried to reverse it:

Find the file → open the file → the system automatically picks the right program

Users do not need to know:

"Which program is supposed to handle this file?"

They only need to say:

"I want to open this document."

The Xerox Star's design explicitly emphasized that users should deal with documents, not programs. Opening a document automatically invoked the corresponding application. (ScienceDirect)

This hid yet another layer of the software's structure:

Hiding the "software" behind the "task".

What the user wants to do is:

Write an article.

Not:

Invoke a word-processing program.

4. WYSIWYG: What You See Is What You Get

Another important attempt was:

WYSIWYG (What You See Is What You Get)

In early computers, users often had to describe the final result through commands, parameters, or special formatting.

WYSIWYG tried to make it so that:

What you see while editing ≈ what you end up with

For example:

The text looks on screen exactly the way it will come out of the printer.

That way users do not need to understand how the computer describes a page internally — they only need to:

See it, then change it.

Systems like Bravo at Xerox PARC drove the development of WYSIWYG editing, and commercial systems such as the Xerox Star, Apple Lisa, and Macintosh later popularized the idea. (CMU School of Computer Science)

So the development of the GUI was never just:

"Turning the black command line into pretty windows."

It was a continuing attempt to:

Turn the computer's internal abstract rules into objects humans can directly observe and manipulate.

5. From "Recall" to "Recognition"

One of the greatest learning costs of the command line was:

You have to remember the commands.

The GUI gradually turned this pattern into:

You don't need to remember — you only need to see.

For example:

rm file.txt

became:

See the file → click the file → click Delete

and then:

See the file → drag it to the trash

This is a hugely important shift:

From Recall to Recognition.

Users do not need to remember what the delete command is; they only need to see a trash can to understand:

"Dropping things in here must delete them."

WIMP (Windows, Icons, Menus, Pointer) likewise became the foundation of the modern desktop GUI, entering mainstream computing through products like the Xerox Star, Apple Lisa, and Macintosh. (Wikipedia)

6. From the Mouse to Touch

Later, computing devices went through another shift:

The touchscreen.

The mouse had already shortened the distance between people and computers:

Hand → mouse → cursor → on-screen object

The touchscreen shortened it further:

Hand → on-screen object

Users could now directly:

  • Tap
  • Swipe
  • Drag
  • Pinch to zoom
  • Long-press

This further reinforced the idea of "direct manipulation".

The touchscreen did not change how complex the software itself was, but it further lowered the cost of learning the mode of operation itself.

7. But None of These Solved the Fundamental Problem

Put the UI simplification methods of the past several decades side by side, and a very clear pattern emerges:

EraMethodCost mainly reduced
Command lineCommandsDirect control of the computer
GUIWindows, menus, buttonsNo need to memorize commands
IconsVisual metaphorNo need to grasp abstract concepts
Desktop metaphorFolders, desktop, trashBorrowing real-world experience
Direct manipulationDrag, clickNo need to describe the steps
WYSIWYGWhat you see is what you getNo need to understand internal formats
TouchscreensTap, swipe, gesturesRemoving the abstraction of mouse and cursor

You can see that:

Every advance in UI has essentially hidden another piece of the software's internal structure.

But the software itself kept getting more complex.

So decades later, we ran into a new problem:

Settings ↓ General ↓ Appearance ↓ Theme ↓ Dark mode

All the user wanted was:

"Switch to dark."

Yet they had to understand the software's internal hierarchy.

So the problem itself began to change:

In the past, we kept finding ways to make it easier for users to operate the features the software exposed.

What is truly worth thinking about now is:

Why do users still need to hunt for features?

If users only had to tell the software:

"Here is what I want to do."

and the software decided on its own:

"Which feature to invoke."

then the next step for UI might no longer be optimizing menus, buttons, and settings pages.

It might be:

Moving from "operating an interface" to "expressing intent".

Intent-Driven UI

Under these circumstances I began to wonder: what kind of UI is actually better suited to ordinary users?

Quite a lot of existing UIs are, in a broad sense, already intent-driven — they just don't do it particularly well.

Let's take an example: the light/dark mode toggle.

In traditional software design, switching between light and dark might require you to go:

Settings -> General -> Appearance -> Light/dark toggle

Or something even more complex — a maze of settings items that gives people a headache!

That's why in many websites and software today, designers put the light/dark toggle right where users can see it at first glance:

Image source: SpinDeck documentation

This is the part that has already been achieved. But what if I add one more requirement — say, I want it to switch to dark at a certain time at night, and back to light at a certain time the next morning?

In this situation, nearly all the software I have seen tries to guide you into the settings (via navigation or other means). That is indeed a big step forward from making users hunt for the setting themselves — but it overlooks one problem: when users land inside that settings page and see a bewildering crowd of options packed together, they will still feel that it is complex and hard to use. And if the options are named in overly technical terms, it feels even more complicated.

So perhaps the essence of intent-driven UI is the intent itself. For example:

User: I want to switch my app's interface to dark mode, and right after switching I want to set it to stay dark from 6 PM tonight, and go back to light at 7 tomorrow morning when I wake up.

Reading that sentence, we can see that what the user expressed is natural language. So many people would say: why not just bring in an LLM (a large language model, AI) to do it for the user — wouldn't that be perfect?

But we would all be ignoring a very important cost problem here. Using an LLM means the user first has to organize their request into structured language and type it out as a sentence, and before that they may have to configure a provider's address and license key, and then tokens get consumed, the request might even get complicated by the LLM's interpretation of the requirement, and the LLM's inference takes time too.

So for a user who just needs to set up a scheduled theme switch, the cost of using an LLM may itself be far higher than the cost of learning how to configure it manually.

Therefore, UI that truly makes life easier for users does not mean "say one sentence and it's done", nor does it mean completely abandoning the design philosophy and framework of traditional UI. Since users are already familiar with the outermost layer of traditional UI, perhaps the best path forward is simply a combination of natural language and traditional UI.

From the very beginning, the user has already stated what they want to do, and their sentence contains expressions with an clearly sequential intent, such as:

  • Switch to dark mode
  • Then set the automatic-switch time
  • Set the time

So you can see very clearly here that what the user wants to do is a set of operations in a linear order. Which makes it quite simple — just let the user:

Finish one thing, then do the next

In other words, something that used to require running all over the place can be done right at the moment of switching!

We can simply make the scheduled-switch setting a tiny button placed right next to the toggle button. When users toggle the theme they just click the toggle; when they want to set a specific time they click the little timer button next to it, and set and save it right in the pop-up that appears!

As shown in this example:

A demonstration of the intent-driven UI concept

Closing Thoughts

For a very long time, people kept stuffing more and more features into software to make it solve more problems — but perhaps it doesn't take a "complex" piece of software to do complex things, and in recent years many teams have put serious effort into this.

Looking back over the whole article — from Sketchpad's light pen in 1963, to Engelbart's mouse and NLS, to Xerox PARC's WIMP, desktop metaphor, and WYSIWYG — we can see a single thread running through decades of history:

Every advance in UI has essentially hidden another piece of the software's internal complexity, bringing users one step closer to what they want to do.

The command line demanded that users first learn the computer's language; the GUI let them "see and manipulate"; the desktop metaphor let them borrow real-world experience; direct manipulation removed the trouble of describing every step; and the touchscreen let fingers touch objects directly. None of these changes ever reduced the amount of functionality — they reduced the cost of learning. The software's internal structure has been there all along; it has simply been hidden away, layer by layer.

Today, software has swelled to a whole new scale, and the old "layer upon layer of settings" approach is showing its age. What users have always wanted is not "to find a particular setting" but "to get something done". As the beginning of this article said, the people using the software may be nothing more than ordinary painters, writers, or musicians — they should not have to wander through the maze of settings just to switch to dark mode.

The intent-driven UI approach does not mean tearing down the design heritage accumulated over these decades, nor does it mean having users say one sentence to an AI and calling it a day — because LLMs carry real costs: organizing your words, configuring keys, consuming tokens, and waiting on inference. Paying all of that for a simple scheduled theme switch is clearly not worth it.

The real direction, perhaps, is finding the exact right meeting point between natural language and traditional UI:

When the user expresses their intent, the software puts the next needed function right at their fingertips.

Like that little timer button in the article — it doesn't make the user climb through the maze of settings pages, and it doesn't ask the user to compose a complete natural-language request. It just sits quietly where the user was already about to click, waiting for the moment it is needed.

Perhaps we should always remember: stuffing more features into software is also stuffing more convenience into it.

Just like the errands we once had to run to an office in person can today be done from the phone in our hand.

Good software shouldn't make users adapt to it — it should walk toward the user.

References

Of the references below, the Wikipedia and Wikimedia Commons content is licensed to this site under CC-BY-SA-4.0; the remaining sources are cited as references for viewpoints and historical facts only.

Historical Sources

Sketchpad

Douglas Engelbart / NLS

Xerox Alto / Xerox PARC

Apple Lisa

Macintosh

The Evolution of UI Simplification

Intent-Driven UI and Modern Interaction