An OSC Address Subspace for Wacom Tablet Data

Wacom digitizing graphic tablets are attractive gestural interfaces for real-time computer music. They provide extremely accurate two-dimensional absolute position sensing of a stylus, along with measurements of pressure, two-dimensional tilt, and the state of the switches on the side of the stylus, with reasonably low latency. The styli (pens) are two-sided, with a "tip" and an "eraser." The tablets also support other devices, including a mouse-like "puck," and can be used with two devices simultaneously.

Unfortunately, this measurement data comes from the Wacom drivers in an inconvenient form. Each of the five continuous parameters is available independently, but another parameter, the "device type," indicates what kind of device is being used and, for the case of pens, whether the tip or eraser is being used. For a program to have different behavior based on which end of the pen is used, there must be a switching and gating mechanism to route the continuous parameters to the correct processing based on the "device type." Similarly, the tablet senses position and tilt even when the pen is not touching the tablet, so a program that behaves differently based on whether or not the pen is touching the tablet must examine another variable to properly dispatch the continuous parameters.

Instead of simply providing the raw data from the Wacom drivers, our Wacom-OSC object outputs OSC messages with different addresses for the different states. For example, if the eraser end of the pen is currently touching the tablet, Wacom-OSC continuously outputs messages whose address is /eraser/drawing and whose arguments are the current values of position, tilt, and pressure. At the moment the eraser end of the pen is released from the tablet, Wacom-OSC outputs the message /eraser/release. As long as the eraser is within range of the tablet surface, Wacom-OSC continuously outputs messages with the address /eraser/hovering and the same position, tilt, and pressure arguments.

With this scheme, all of the dispatching on the "device type" variables is done once and for all inside Wacom-OSC, and hidden from the interface designer. The interface designer simply uses the existing mechanisms for routing OSC messages to map the different pen states to different musical behaviors.

We use another level of OSC addressing to define distinct behaviors for different regions of the tablet. The interface designer creates a data structure giving the names and locations of any number of regions on the tablet surface. An object called Wacom-Regions takes the OSC messages from Wacom-OSC and prepends the appropriate region name for events that occur in the region.

For example, suppose the pen is drawing within a region named "foo." Wacom-OSC outputs the message /tip/drawing with arguments giving the current pen position, tilt, and pressure. Wacom-Regions looks up this current pen position in the data structure of all the regions and sees that the pen is currently in region "foo," so it outputs the message /foo/tip/drawing with the same arguments. Now the standard OSC message routing mechanisms can dispatch these messages to the part of the program that implements the behavior of the region "foo."

Once again tedious programming work is hidden from the interface designer, whose job is simply to take OSC messages describing tablet gestures and map them to musical control.

state machine for wacom data

the following describes the state machine I use in my own wacom parser. the primary intention is to accurately describe all the "singular" states that occur in transition between hovering, drawing, etc. below, the "=>" notation can be understood as "transitions to" followed by the list of possible subsquent states.

"enter" => "exit" | "hovering" | "touch"
enter state is broadcast when the pen enters the tablets sensing field. this is always the first state repored.

"hovering" => "hovering" | "touch" | "exit"
this state is broadcast along with position and tilt as long as the pen is in the tablet's sensing field but not touching the surface (i.e. pressure = 0)

"touch" => "drawing" | "release"
a singular state which occurs at the instant the pen makes contact with the surface

"drawing" => "drawing" | "release"
state while drawing

"release" => "touch" | "hovering" | "exit"
state when drawing ends

"exit" => "enter"
state when pen leaves the sensing region

the entire state machine is applied to either "/tip/*" or "/eraser/*" in the natural manner (and since they are exclusive, one tool must exit before the other can enter). for sake of generality one may want to extend this to "/tool[00-99]/tip|eraser/*" when multiple tool-ids are in use.