Computational Semiotics

Smiley Toolkit
Route Map Drawing
Picture Organizer
Holistic Control Panel
Universal Mood Enhancer
Computer-Aided Rhetoric


Over the summer of 2003, I became aware that most of the software ideas I've had over the years have involved a model of the user. This model is used to solve a problem usually involving communication. The name Computational Semiotics implies that the software described here works with meanings as opposed to raw data.


Within a decade, most of the population of the developed world will have access to the internet, and a good understanding of how to surf. Access will be ubiquitous and cheap, and there will be reliable ways to determine who a user really is.

At this stage of the internet's evolution, online communities of like-minded people are beginning to form. It seems natural that these nascent communities should begin to evolve binding decision making systems whereby the memnbership can make decisions.

One of the central problems of democracy is, ideas and plans come from individuals, yet need the assent (or lack of dissent) of the majority in order to become group policy. Because whole-group decision making is a hard communications problem, we generally make only one decision every few years: who shall the leader(s) be? Then the leaders make decisions until their mandate runs out. Realizing the fallibility of leaders, we generally elect a few members of opposing factions to point out the leader's mistakes.

Technology has now evolved to the point where making detailed decisions as a whole group is becoming cheaper.
- Objection: Groups don't have ideas, individuals have ideas.
- Objection: The common man is an idiot; or at least insufficiently educated to make far-reaching decisions.
The answer is a process by which ideas are inserted into the system by individuals, then take on a life like a chat-room topic, but with voting.

Smiley Toolkit

When I was at Hotline, I had an idea for a smiley generator. Hotline the program supported chat rooms, so I wanted to make a graphical smiley editor in which you'd have a bunch of slider controls with which you could make a smiley exactly expressing your mood. It would have a palette of preconfigured smileys, to which you could add your own with the editor. Then drag-and-drop might be used to insert the smileys into chat or newsgroup messages. The user could also have quick control-key access to smileys in the palette, (e.g. cmd-shift-S-4 might insert the fourth smiley into the text).

  Implementation details: Since there's a Hotline client mediating both sides of the communication, each smiley can be described and transmitted as a sequence of perhaps six numbers, although it appears to the user as a graphic of a human face with curves indicating eyebrows, eyes, nose, mouth, and perhaps forehead creases and dimples.

  The applet at right is a simple example of what I'm talking about. Each slider may affect many of the curves on the face but a single emotion of interest to the user. Several improvements immediately suggest themselves: more emotions, smarter interactions between the emotions (like ligatures in a typeface), dimples and nose - dimples might only show up if certain emotions passed a predefined threshold value. If anyone is interested, drop me a line and I'll send along the source. Feel free to link to or copy the applet for use elsewhere.

Back to Home Page

Route Map Drawing

When a human draws a map to give directions, cartographical accuracy is often sacrificed to aid understanding. Here are a few distortions that might make a map easier to understand:

  Erase off-route areas of the map.
  Draw complicated parts of the route larger and long turn-free stretches smaller.
  Draw the route bolder and blacker than other streets.
  Indicate landmarks and roadsigns.
  Partially straighten the curves in the route to make it fit into a linear space on the page, leaving more room for explanatory notes.
  Orient the route from top to bottom of the page (unless North is important), so that the expanatory notes read like a story.

All these distortions would be easy to automate given that software already exists to give directions from one address to another. Note that map databases would have to include information about what landmarks are visible on what roads.

Picture Organizer

Programs like Photoshop are capable of applying a wide variety of effects to an image, but artists are often interested in the relationships among the objects represented by the image. It would be nice if one could start with a photograph and first identify the various objects in the image, then act on them with verbs of interest to artists: emphasize/mute, is behind/in front of, happens before/after, is more desirable, ethical, virtuous than, etc.

  An artist can move the viewer's attention around in a picture through the use of hatching, and by arranging the picture elements in such a way that their interrelationship draws the viewer's eye in predictable directions. The graphical arrangement of the picture elements can also suggest a relationship between the elements. If these various visual effects could be embodied in software, one could start with pictures of objects and create a picture communicating a relationship between those objects automatically.

Holistic Control Panels.

Let's say we have a control panel for a complex system, like a nuclear plant or airplane. The data displayed gives the operator a summary view of the whole at any given moment. Let's also say that there's a panel controller deciding from moment to moment what the most important thing going on is. All the information is always displayed, but some is more important than others, so the control panel draws attention to the important stuff in various ways while keeping the rest of the information available.

  For example, in the aircraft cockpit an engine overheating would usually cause loud beeping and flashing lights, but if a wing is also falling off, the engine warnings would be muted, allowing the pilot to concentrate on higher priority items. If a wing actually fell off, the gauges related to the engines and landing gear on that wing should go dark or say Not Applicable, instead of relaying meaningless information.

  The interesting part of the panel controller is the part that decides what is the most important subsystem to the user. I suspect this could be expressed with a few simple formulas based on something like Maslov's hierarchy of needs, which is a simplified model of what the user finds important in the system being controlled.

  A related problem is how to draw attention to a subsystem without preventing the user from interrogating other subsystems as needed. At all times the user's muscle memory should remain a reliable guide to the desired control.

Computer Cartoon Animation

Gave this important topic its own page.

Universal Mood Enhancer

Ever since 1992 when I started woodworking as a hobby, I've wanted to build a kind of sinuously curved, self-illuminating wood sculpture which I call a Universal Mood Enhancer. While it's possible to imagine any number of really cheesy designs that fit that description, I'm hoping ongoing study of gothic architecture and ornamentation will eventually make my sculptures simple and expressive.

  The sculpture should evoke different moods when illuminated from different angles or with different colours of light. The illumination should be controllable via a set of slider controls (e.g. joyous<->serene, mellow<->intense, energetic<->wan, ghostly<->solid) that will allow the mood to be dialled in directly. (An early example of computational semiotics) An on-board processor will be required to control the lighting effects, as many of the moods will require that the lighting change over time, flashing, pulsing or rising and falling more gradually.

  One attractive aspect of the UME is that its design has both artistic and technical aspects, much in the way renaissance building projects did.

Back to Home Page