Adding Audio Feedback and Keyboard Control to Swing Based Applications
For the present, this site is a repository of documents describing the audio look and feel, its current status, and how it has been implemented. Sometime in the future, the executable code will be made available, along with some form of installer, so that interested parties can run and evaluate the user interface for themselves. In the interim, a number of pages are provided that document the sound sequences presented when the user navigates or activates some aspect of the user interface. It is hoped that these give a flavour how the audio interface "looks".
There are three kinds of documentation to peruse. There are the user interface documents, the architecture document, and the bugs/complaints documents. All are in various stages of completion -- keep in mind that these are drafts that currently under revision.
There are two parts to the user interface documentation. The main document describes the design of the audio look and feel. The second aspect is an entire web site in itself documenting the translation matrix for each swing component.
This is the "why we did it the way we did it" document. It describes the rationale behind the audio feedback provided when users interact with the Swing components. For further edification, read "User Interface Design" (as of this writing, this document is under construction).
A companion piece to the user interface design, the translation matrix is a full specification of the sounds and speeches used for each Swing component. This is "manual" that the programmers used to write the code. Its main page is entitled, "Swing Set Components: Table of Contents".
The translation matrix documentation already includes descriptions of the keystrokes used to navigation and or manipulate Swing components, but it does so on a component-by-component basis. There are two documents describing keyboard control separate from the audio feedback. This may suit the reader as a more convenient way of perusing them.
In order to achieve a level of consistency, the same keystroke is used for different components for conceptually similar tasks. For example, there are a number of components that can be thought of as grouping a set of members, and it is desired to inspect and navigate among them. Examples include menus and lists -- menus group together a set of menu items, and lists possess list items. The keystrokes to navigate a menu and a list are the same. Which is actually navigated at any given time depends on which has keyboard focus.
There are two keystroke charts; one for non-text components, and the other for text based components:
There have been a number of meetings regarding the rationale for, and the choices of, non-verbal audio for the various Swing components. The minutes of these meetings can be found here.
This is the "how we did it the way we did it" document. It describes the implementation of the audio look and feel from a technical standpoint. Read "Audio Look and Feel: Architecture".
These documents describes issues the programmers encountered while writing the code. It is made up of bug reports, complaints, and "have you considered this?" musings to provide feedback to Sun. The technologies covered are Swing, the Java Speech API (JSAPI), and the Java Media Framework (JMF). Go to the "Problems" section.
Copyright (C) 1999 Adaptive Technology Resource Centre, University of Toronto.
Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.
Updated: 1999 Sep 06 JS
Web site maintained by Joseph Scheuhammer