Nokey

Project Motivation

There are many alternative input devices for people with disabilities that allow control of the mouse pointer, such as eye, head, or hand tracking systems. For many of these people, their computer is the only viable way to communicate with the outside world. Unfortunately, very few of these assistive devices have the capability to accurately generate mouse click events. A lot of software has been written to address this problem, the majority of which takes the form of an on-screen keyboard. For these applications, the user must dwell over a button for a certain period of time to select it, which presents several usability problems. Since the dwell time to select a button is constant, the user is usually being either impeded or rushed in their typing speed. This causes frustration because the application dictates the pace instead of the user. Secondly, an on-screen keyboard usually requires a lot of screen space to display all the keys, therefore they either leave little room for the active application, or obscure part of it. Finally, there are far too many separate buttons displayed on the screen at the same time with on-screen keyboard applications. Studies by the military on eye tracking systems have shown that a maximum of nine regions can be reliably selected from without fatigue over the long term. This project has been designed to solve or mitigate each of these three problems.

Synopsis

Nokey is an accessibility application designed to facilitate complete control of a computer using only the movement of the mouse pointer. The GUI of the application consists of several semi-transparent windows, each containing a simple interface for a different task. For example, one such window will contain an interface to allow the input of text using Morse code generated by moving the mouse pointer into different coloured regions. Another window will contain a different interface to switch between application windows on the desktop, and perform other application tasks. Each of these windows are initially hidden, and can be brought forward by moving the mouse cursor to a certain corner of the screen. Corners and sides of the screen that are not used to bring forth interface windows, can be occupied by quick shortcuts that perform a certain task, or generate a key event.

The implementation of this interface would provide the KDE project, and Linux in general, a unique new method to control a computer using only mouse movement. For people dependent on an eye or head tracking system to communicate, it may be a more efficient and less fatiguing way to use their computers.

Deliverables

I intend to complete the Nokey application by the end of the Google Summer of Code Contest. Requirements for completion consist of:

1. All Nokey functions (except configuration) accessible using only mouse pointer movement.
2. An interface for entering text via Morse code
3. An interface to facilitate application switching and menu access
4. A method of sending common keyboard commands to active application windows.

Project Details

As it is quite difficult to accurately portray the Nokey GUI in words, I have posted screenshots of an early prototype I have coded (running on Windows).

Screenshot 1: http://www.ece.uvic.ca/~lspalteh/mm_minimized.png

This screenshot shows Nokey immediately after starting up. Each corner of the screen contains a translucent tab (circled in red), representing an available action. In addition, each side of the screen has an elongated tab(boxed in green).

If the mouse is moved over the tab in the top left corner for a second, the text entry interface is displayed (Screenshot 2).

Similarly, moving the mouse over the tab in the top right corner displays the application control window. (Screenshot 3).

The bottom left tab exits Nokey.

The bottom right tab sends a key event to the active application corresponding to "Enter".

The blue tabs on each corner of the screen correspond to the arrow keys on the keyboard.

To activate any of these tabs, the user moves the mouse pointer over them and hovers there for a certain period of time. Hovering over the top left or top right tabs slides a translucent window over the screen with the selected interface.

Screenshot 2: http://www.ece.uvic.ca/~lspalteh/mm_text.png

This screenshot shows the text entry screen. Note that the interface is semi-transparent so the user can still see their work, while maximizing the size of the interface buttons (very important since many alternative input devices are not that accurate).

into different regions. The simplicity of Morse code allows English language text to be entered using only 4 separate regions on the screen. Each region is colour coded to facilitate easy memorization of sequences. The user moves the mouse pointer into the dot and dash regions to compose a letter. To accept this letter, they move into the purple text buffer region. To send the letters in the text buffer to the active application, the user moves from the purple text buffer into the green region, and back again.

This layout of the regions is only an early concept and may change to make more efficient use of screen space.

Screenshot 3: http://www.ece.uvic.ca/~lspalteh/mm_apps.png

This is another example of a transparent overlay screen, with some buttons to control applications. The middle region is to switch the active application. The other regions are labeled as to their function. In the final application, the button labels would be replaced by simple icons, since the current translucent text is visually distracting.

Bio

I am a 4th year computer engineering student at the University of Victoria taking the co-op program. At my previous two co-op work terms, I was employed at the University of Victoria Assistive Technology Team (http://web.uvic.ca/uvatt), where I was developing an eye tracking system. During this time I gained extensive experience in the design requirements and usage constraints of eye and head tracking systems. This has led to my interest in alternative high level applications to control a computer using only mouse movement.

Future Work (if my application is successful)

After the initial goals for the Google Summer of Code are complete, I intend to continue working on this project by expanding the feature set. Some ideas for future features are: Alternate text input interfaces (for those uncomfortable with Morse code), different templates for layout modes that can be switched between, and a Linux live CD that is preconfigured for use with an alternative input device.