Exemplar
Exemplar is a programming by demonstration tool to explore and prototype interactions involving sensors. For interaction designers, connecting sensor input data to application logic remains an exercise in patience and trial and error testing because current tool support is geared towards experts and heavy analysis.
With exemplar, a designer first demonstrates a sensor-based interaction. He then reviews and edits a visual representation of the action and tests his author behavior by performing the action again.
Scenario:
Dan is an interaction designer who wants to prototype a smart bicycle helmet with automatic blinkers such that tilting the helmet left or right causes the associated blinkers to signal. He picks a 3-d accelerometer and connects it to a compatible hardware interface. Data from all connected inputs is shown simultaneously in the exemplar software. Previews light up when sensor data changes rapidly to attract attention. This overview allows him to discover pertinent signals. Dan brings one accelerometer output into focus. Then demonstrates examples of the action he wants to recognize. He reviews and marks up the sensor signal to define the regions he wants to recognize. Here exemplar generalizes from the examples by calculating thresholds within the highlighted regions. In addition to drawing regions, Dan can also use a foot pedal to define examples while he performs them. Dan can immediately test his design to see if it works. Exemplar shows a visualization of sensor signal regions that match the demonstration.
Exemplar will then generate events based on matching regions in two-ways. First, the software can send system mouse and keyboard events to remote control over applications or Exemplar can send events to the d-tools prototyping environment to generate hardware output.
Dan can refine the learn interaction model through direct manipulation. He can apply filters to calibrate ad transform the incoming sensor data.
Exemplar can generalize from the examples, by calculating thresholds within the highlighted region, or by applying a pattern matching algorithm to capture complex interaction. For threshold events he can drag minimum and maximum threshold values, and adjust hysteresis as well as minimum diration parameters graphically. In pattern matching mode, Exemplar displays and error graph that shows a distance metric between the current input and the learned example. Dan can drag a threshold line to determine the maximum acceptable error. When the error line falls below the threshold, the corresponding event is fired.
Experts can add their own filters written in java to extend the library of signal transformations. Interactions can also be authored for events that depend on multiple sensors simultaneously (Accelerometer and Bend sensor: Event is fired only if both sensors match). Finally Exemplar also allows designers to output continuous values corresponding to filtered input signals.
A first use study with twelve participants showed how exemplar can enable novices and experienced designers alike, to create interaction designs involving sensors. Participants brainstormed, proto-typed, tested and refined their own functional game controllers in less than thirty minutes.
Exemplar has been released to the community as open source.
lynseytoner 