We have been working for many
years on a real-time system that allows a computer to play pinball.
It came to our attention that the system could be used for not only
controlling a pinball machine, but also to control other devices.
With that in mind, we set out to design a modular infrastructure for the
system to interact with the real
world. With this modularity, we are able to have a programmer
create a wide range of learning tools by incorporating various components
of the system.
A typical student does not ordinarily
have the opportunity to engage in the challenges of real-time programming.
Throughout a student?s journey into the world of
programming, they may hear the term Real-time System or learn about
a particular system, but rarely do they get the chance to actually work
or design a real-time system. It is typically a burden on a school
or laboratory to house such a system. Both cost and size of such
systems deter many institutions from providing activities of this kind.
To allow students the means of
working with a real-time system first hand, we have developed our own inexpensive
real-time test bed. The Real-time Object Oriented
Tool-kit (ROOT) we developed allows a novice programmer to manipulate
real world devices attached to a PC. The goal of our project is to
develop a computer package (hardware and software) that can be put together
with commonly available computer parts requiring minimal technical skills.
Our project is designed to facilitate students in developing programs to
control physical devices that interact with the real world. Real-time
systems generally involve very expensive equipment. Our goal was
to develop a system that can easily be brought into the classroom or laboratory.
The system is written primarily in Java, except for the camera driver written in the C language. The camera is a QuickCam by Connectix (now Logitech) which connects to the parallel port of the computer. Connected to one of the serial ports on the computer is a digital I/O board, ADR2200 by Ontrak Control Systems Inc., which controls the external devices. The I/O board has a series of relays that control the switches on the devices. No driver is required to use the I/O board. It is a simple serial device that accepts ASCII commands.
The computer interacts with the
world by seeing the environment from the camera and controlling devices
connected to the digital I/O board. As mentioned at the beginning
of this paper, the original idea of the system was to have the computer
play pinball. It became clear that the basic mechanics of the system
could be applied to other devices besides that of a pinball machine.
In particular the image analysis section of the program can be applied
to tracking other things besides
a ball on a pinball playing field.
Since Java is object-oriented, it allows for various changes to go on while still preserving the original design. Our system is designed modularly and various parts are replaceable. Experiments in various tasks are outlined below:
Laser Pointer Mouse:
Using a common pen laser pointer, a user is able to control the mouse pointer.
The camera is fitted with a red filter over its lens. The user points
the laser pointer on a white wall. The lens blocks out all the other
light, except for the laser pointer. Since the camera does not see
color, the laser point appears as a white light. Movement does not
come into play with the analysis. The computer while analyzing the
image looks for pixels that are pure white in a
single frame. The mouse pointer on the computer is moved to reflect
the position of the laser pointer in the frame. Clicking the laser pointer
on and off was the original idea for clicking the mouse. However
the computer would miss seeing the light periodically and would create
extra mouse clicks. To prevent this, we hold a computer mouse and
used its mouse buttons instead.
Hand Interaction:
We modified the analysis part to track a person?s right hand. The
analysis finds the furthest change in the right hand part of the image.
With a static background, this would be the person?s right hand.
The display inverts the image, so that a person sees their mirror image.
With the system able to track a person?s right hand, someone could interact
with the display. A simple interaction was chosen: touch the squares.
There are two squares on the display
that when a user reached up and touches them, the squares change color.
Robot: We have connected an OWI Incorporated Movit Soccer Robot to the I/O board. The system sees the robot moving and can track its progressive movement. As a simple test, we set the system to detect if the robot is making any progress moving. We do not detect the possibility of a collision, only that a collision has occurred. The robot has a fence that prevents it from going of bounds. When the robot hits the fence, it is stopped from making any progress in its current direction. When the system detects this, its movement is reversed. The robot store carries them. Link to product.
Morse Code Conversion:
We
hooked up a light to the I/O board so the computer could send Morse code
light pulses. Using the camera we can detect when the light is being
transmitted. The computer then reassembled the dots and dashes as
characters and displays the message sent. The Morse Code program
uses the same analysis code that the laser pointer uses; it detects white
light without movement. With the room lights dimmed, the Morse code
light becomes pure
white and is the only object found in the frame.
Compile:
Go into the project which has the Makefile and run:
make all
After we short while, it will compile all the parts of the system. If some part has not been made, you may need to compile that directory by hand using javac.
Run:
Go into the root directory containing the pinball packages. Then do:
$ java pinball.player.PLAYER_PACKAGE.PLAYER
Replace PLAYER_PACKAGE with the package that contains the player class.
Replace PLAYER with the player class.
Copyright 1997-2001 Dayton Clark & Lawrence Goetz