A 3D GUI Operating System


This is a thought experiment. I’m sitting before a new, state of the art computer: large, wide LCD screen, a cool looking keyboard with some strange controls around the edges – these are optical sensors for your fingers that replace a mouse, trackball, touchpad, joystick, etc. You need many more controls than a mouse or joystick can provide, and they must be ergonomically placed in relation to the keys, so that hand movement is minimized.

The screen background is a virtual 3D display that can be panned by moving the pointer to an edge of the screen, so if you have many documents open you can easily move to them – no separate, numbered desktops. Of course, you can move your document to any position on the screen/desktop, enlarge or shrink it, rotate it, tilt it in any direction.

The screen is black, but the computer is up and running, there’s just nothing selected to display. I’m asking you to throw away all you know about the desktop metaphor, generic windows, background image and icons. You can have them if you want, but they aren’t necessary. I put my finger on one of the optical sensors, a pointer appears on the screen, I make another move and a cursor appears at the pointer, now I can type in a command, make another move on the sensor and the results will appear in any typeface, size and color I desire. I want to open a book I’ve been working on; I move my finger to another sensor, a dot of light appears in the dark background and quickly enlarges into a book. It looks like a real book, the pages will turn forward or back. It opens to where I left off writing. I can smoothly enlarge this book until a pixel of one character fills the entire screen or shrink it into the background until it disappears.

You will see no menus until you choose the appropriate sensor, then one pops up according to context. Tool bars work similarly.

There are no named applications as such. The OS only asks what you want to do: write a letter or a book, draw, database information, send email, browse the web, etc. Everything is functionally oriented.

I’m putting this idea for a new OS out on the net as a request for comments and to see what interest there is making it a reality. I think what I have envisioned is possible now. And what are the difficulties to be overcome? Is anyone out there interested in attempting it?

I am not a programmer, but an experienced and savvy computer user and creative thinker who has run Linux for 15 years and very much want an OS like this to fulfill my needs, my thinking and working styles.

The business, programmer and engineering installed user base legacies severely hamper any really user-friendly OS. These are outmoded ideas not relevant to many potential users in other fields, notably in the arts and humanities and many other fields of research. The interfaces are too square, functions are too separate and kludgy, the hierarchies of files/directories and approaches limits flexibility and usability. Current OS interfaces are not like the real world and life in general. The metaphor of a VR world that can mirror the real world is a good start, but the electronic medium can be a world to itself with its own peculiar and unique properties that might make user interface easier, more creative, etc. Therefore, the interface must begin as a graphical drawing/processing object-manipulation engine.

I would like to see an interface that is 3-D object oriented. Documents/objects should look like real world objects. For example, a book should appear as a book with turnable pages. To access an object the user could type in its title or could search for objects of preferred type. There need not even be icons. The operating system would know where everything is and what it is. All information about an object should be included in object headers. No more opening an application for specific kinds of data. When a work-object file is selected the OS would run the appropriate modules and display it the way the user wants. In a compound object, such as a book with pictures, tables, charts, etc., the objects would retain their identity but be linked to the master object. If saved for archiving, all the associated objects could be included in one compound file, like a UNIX TAR file. If an object is linked to another, the OS would not allow deletion or at least warn the user of the link. When an object is selected, the appropriate toolbox/palette opens and can be positioned anywhere on the screen and expanded or contracted in size. The GUI object can also be brought near and thus expanded or pushed away into the background until it disappears.

You might want a background image, but it could be an interactive desktop similar to an HTML image map, so that by clicking certain areas specific actions could be invoked.


On bootup, the kernel first establishes a 3-D VR GUI space. Therefore, graphic recognition is built in at system level. Writing this interface for Xwindows probably won’t work, but some legacy code might be used.

The OS is not exclusively graphically-based; a command line may appear anywhere the cursor is. Text is both character and graphic object, i.e., all text displayed has graphical properties.

The OS is an application/database/very high-level programming language in one, using plug-in data/function/process modules that are 100% compatible. These modules are not separate applications.

The OS GUI draws objects according to the user’s preferences. There need be no permanent look and feel, as in current applications. The corporate/office model for software branding should go away.

Everything is an object, each occupying its own memory space.

Objects can have any size, shape, colors and position in VR space.

Objects are divisible, groupable (compound objects), linkable, can communicate, have inheritance, user permissions and passwords.

The OS databases all objects and their properties/characteristics.

Multitasking and multithreading is inherent.

No software memory or storage limit. Only limited by the hardware, file system and OS.

The file system is an object-based tangled hierarchy, spanning all drives. No directories or folders need exist except for user convenience.

The file system/drives might be partitioned into five sections: system/functions, data object templates, workspace (users), archive (compressed), caching (swap). These need not seen by the user.

No drive letters or numbers need be displayed. Only the removable drives and external storage (backup) are user accessible and might have icons.

All system utilities automatically run in background: anti-virus, anti-spam-adware-malware, file checking, compression, drive check, etc.

Digital sound recognition is built in.

Handwriting recognition built in.

Voice recognition built in.

Plug and play. Most other hardware recognition, especially scanning.

A new driver layer that eliminates the need for separate hardware drivers. Drivers should be built into hardware, solving the problem of manufacturers having to release proprietary hardware code to third parties.


Fully user configurable. Can be set up as any metaphor: desktop, artist’s studio, laboratory, library, etc. Modules to create graphical user metaphors and widgets would be a major component. Just drag and drop a widget anywhere, link them, create a functional display for what you want to do.

No windows (unless the user wants them) or program icons. Get rid of the rectangular box metaphor. Work-object icons might be appropriate, depending on the user’s needs, but optional.

A 3-D pointing device with many more buttons or optical sensors would be needed to control the interface.

A graphics tablet could be another standard input device.

At first, a standard keyboard would be one input device, then others might be designed to work better with this new metaphor.

High resolution stereo monitor glasses could become a monitor replacement.

Toolboxes/palettes to select tools to operate on a work-object may be selected on screen.

Tools: pen, brush, open/close hand (mover/grabber), outliner-selector (used with grabber), scissors or knife, eraser. What else?

Displayed objects can be any recognizable object, such as a piece of paper or book.

Displayed objects are sized by a zoom function using perspective of the VR space, so that they can fill the screen or vanish in the distance.

There should be only one desktop, but of potentially infinite size, depending on memory, processor, monitor, etc., that can be panned by moving the pointer to an edge of the screen if many documents are being displayed.


networking/communications, word processor, table (spreadsheet, etc.), MIDI, digital audio, video (MPEG-1-4), Extended graphics editor, Extended drawing editor (vector, raster, CAD), charting/graphing, Equation processor, data acquisition, special databases for specific fields, statistics, dictionary (spell checker, thesaurus, definitions), OCR, etc.


The display and work model metaphor goes beyond document-centric to work-object-centric. And it is process-oriented.

No more major applications programs would have to be written, only small function modules, which should make writing and debugging easier and eliminate application bloat.

The program/module installer would be built into the OS. All installs would follow the same procedure.

At first, we night want to create a single-user system, but it must eventually be capable of multi-user/network server functions.

OS should recognize most file data formats, especially from DOS/Windows, Macintosh and UNIX.

A PDA version that interfaces with the desktop might be a good idea.

OS should run on faster Pentiums and x86 clones, Power PCs, MIPS, Ultra Sparc, DEC Alpha, etc.

Could Linux be adapted? Could code from X Windows be used or should the graphical interface be written from scratch?

I’d appreciate input. Is anyone willing to tackle it?



Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: