Interaction

• Interactive computer graphics (Sutherland)
  • View image
  • Interact with image via input device
  • "Rinse and repeat"

• Provides a cross-platform interaction API
• Lowest common denominator (minimal functionality) of interactivity
  • Keyboard
  • Mouse
  • Menus
  • Timers

Input Devices

• More important than the physical aspects of a device are its logical properties.
  • Think of a mouse and a trackpad-- in some respects they represent the same logical device
  • or an infrared touchscreen bezel and a touch sensitive screen
• We are thus interested in logical devices
  • Functionality of the device from the persepctive of an application program
• Analogy with high-level language I/O
  • scanf, printf, <<, >> have no awareness of the physical nature of the device; all that counts is that bytes can be transferred to and from them
  • Similarly, mouse input simply cares about being able to query for (x, y) position; the exact nature of the device providing that data is unimportant

Physical Devices

• But the physical aspect is important as well
  • If mice and trackpads are basically the same logical device, why are trackpads mainly on laptops and not desktops?
• Two basic categories
  • pointing
    • secondary button selection in addition to primary positioning function
  • keyboard
    • Returns character codes
      • Handwriting recognition system; speech-to-text are alo examples

• Mouse/trackball
  
  • Relative positioning devices
    • Absolute position plays no role
    • Inappropriate as a drawing device
    • Can be implemented as a variable-sensitivity device
      • Transmitted info is velocity (faster move the mouse, the further it moves)
• Data tablet
  
  • Absolute positioning
• Lightpen
  
  • Primarily absolute positioning
  • Mainly used for selection of on-screen objects
• Joystick
  
  • Can also be implemented with variable sensitivity
  • Mechanical elements can provide physical feedback to user
    • Vibration
    • Relative resistance
• Spaceball
  
  • Three-dimensional version of the joystick
  • Uses pressure sensors rather than movement to provide six input
    • three directions (up/down, left/right, forward/back)
    • three of twisting

Logical Devices

• Logical behavior of a device is described by
  • specific information (measurements) returned by the device to the program
  • timestamp of the measurement

1. String Device that returns strings (sequences of characters) to the program
   • keyboard
2. Locator Provides a position
   • mouse
   • Often have to convert backwards from screen to world coordinates
3. Pick Returns the identifier of an object on the display
   • mouse
   • ActionEvent and MouseEvent in Java-- clicking a button generates its own (action) event, though it's accomplished via a mouse event
4. Choice Allows the user to select from a set of choices
   • widget - graphical (software) component): menu, scrollbar, on screen buttons (e.g., checkboxes)
5. Valuators Provides (analog) input to the program
   • widgets - slidebars, dials
6. Stroke Returns an array of locations
   • mouse press begins the location trannsfer; mouse release terminates

• measure - information returned to user program
  • one or more characters from a keyboard
  • position for a locator
• trigger - manner in which device user can signal the computer that input is available
  • Enter key
  • button on locator

Input modes are distinguished by the relationship between the measure and trigger:

• request mode - measure is not returned until device is triggered
  

  // Two keyboards, cin and cin2-- want to read from either
  cin >> value;
  cin2 >> value;				// Can't read until cin read returns
  				

• sample mode - input is immediate-- measure is returned as soon as function is called
  

  while (!done) {
    if (cin.hasInput()) {
    	 cin >> value;
    	 done = true;
  	}
    else if (cin2.hasInput()) {
    	 cin2 >> value;
    	 done = true;
  	}
  } 
  				

• event mode - measure is place on an event queue
  
  • Event queue processed by application

    while (true) {
    			if (!eventQ.isEmpty()) {
    				 event = eventQ.remove();
    				 // Handle the event
    				 if (event.source == cin)
    				 		process cin input
    				 else							
    				 		process cin2 input
    			}
    }
    

  • Event queue processed by system which invokes callback functions

    void display() {
    		 ...
    }
    
    glutDisplayFunc(display); // register event handler
    glutMainLoop();						// allow the system to take over and wait for events
    									

Client-Server Model

• servers perform tasks for clients
  • print and file servers
• Workstation (display/keyboard/mouse) is a graphics server
• OpenGL program is the client

Display Processors

• How it used to be:
  • Main CPU in charge of controlling display
  • This task consumed most cycles
  • Solved by introduction of a display processor
    • limited instruction set oriented towards graphics
    • User program compiled into set of graphic instruction
    • These instructions sent to display processor
    • Stored in display memory/file/list
      • Sequence of graphic instructions that can be stored and retrieved repeatedly
      • May be compiled for performance speedup
      • Allows client-server configuration
• Display processor evolved into a full-fledged graphics server, and application program into a client

Immediate Mode

• As soon as a graphics instruction is issued to the server, it's executed
• No memory of the instruction is retained
• Redisplay involves sending the insturctions to the server again
  • Requires substantial data transfer between client and server

Retained-Mode

• Object is defined and specification is placed in a display list stored on server
• Object is displayed via simple call from client
  • Reduced network traffic
  • Often much numerical computation need be done only once and stored as part of the display list instruction

Display Lists in OpenGL

• Must be able to create, add instructions, and invoke the list
• glNewList / glEndList functions

  glNewList(...);
  	 ...
  glEndList();
  				

• Display lists must be identified by a unique (integer) identifier
• Can also specify whether should also display as list is constructed (GL_COMPILE_AND_EXECUTE), or merely compiled (GL_COMPILE).

  const int MY_SQUARE = 1;
  ...
  glNewList(MY_SQUARE, GL_COMPILE);
      glBegin(GL_POLYGON);
          glColor3f(1, 0, 0);
          glVertex2f(-.5, -.5);
          glVertex2f(.5, -.5);
          glVertex2f(.5, .5);
          glVertex2f(-.5, .5);
      glEnd();
  glEndList();				
  				

• glCallList to invoke (and thus display) the list.

  glCallLIst(MY_SQUARE);
  			

• Once the list is specified and compiled, it can no longer be modified

  static void display(void) {
      glClear (GL_COLOR_BUFFER_BIT);
      glColor3f(1, 0, 0);
      double d = 1;
      glCallList(MY_SQUARE);
      glColor3f(0, 1, 0);
      d = .5;
      glCallList(MY_SQUARE);
      glFlush();
  }
  
  void init() {
      glClearColor(1.0, 1.0, 1.0, 1.0);
      double d = 1;
      glNewList(MY_SQUARE, GL_COMPILE);
          glBegin(GL_POLYGON);
              // Current value of d is compiled into the list, the d-modification logic above has no effect
              glVertex2f(-d, -d); 
              glVertex2f(d, -d);
              glVertex2f(d, d);
              glVertex2f(-d, d);
          glEnd();
      glEndList();
  }					 
  					 

• The current state of the system (including transformations) affects how the list is displayed.

  static void display(void) {
    glClear (GL_COLOR_BUFFER_BIT);
    glColor3f(1, 0, 0);
    glCallList(MY_SQUARE);
    glColor3f(0, 1, 0);
    glRotatef(45, 0, 0, 1);
    glCallList(MY_SQUARE);
    glFlush();
  }			
  			

• Furthermore, execution of the display list changes the state of the system

  static void display(void) {
      glClear (GL_COLOR_BUFFER_BIT);
      glColor3f(0, 0, 1);
      glBegin(GL_POLYGON);
          glVertex2f(-.5, -.5);
          glVertex2f(.5, -.5);
          glVertex2f(.5, .5);
          glVertex2f(-.5, .5);
      glEnd();
      glRotatef(22, 0, 0, 1);
      glCallList(MY_SQUARE);
      glRotatef(22, 0, 0, 1);
      glBegin(GL_POLYGON);	 	 			// Will display in red
          glVertex2f(-.5, -.5);
          glVertex2f(.5, -.5);
          glVertex2f(.5, .5);
          glVertex2f(-.5, .5);
      glEnd();
      glFlush();
  }
  
  void init() {
      glClearColor(1.0, 1.0, 1.0, 1.0);
      glNewList(MY_SQUARE, GL_COMPILE);
          glBegin(GL_POLYGON);
              glColor3f(1, 0, 0);
              glVertex2f(-.5, -.5);
              glVertex2f(.5, -.5);
              glVertex2f(.5, .5);
              glVertex2f(-.5, .5);
          glEnd();
      glEndList();
  }			
  			

The Matrix Stacks

• OpenGL maintains stacks of the matrices to prevent execution of a display list from making unwanted/unexpected changes to the system state
• glPushAttrib(), glPushMatrix(), glPopAttrib(), glPopMatrix()

static void display(void) {
    glClear (GL_COLOR_BUFFER_BIT);
    glColor3f(0, 0, 1);
    glBegin(GL_POLYGON);
        glVertex2f(-.5, -.5);
        glVertex2f(.5, -.5);
        glVertex2f(.5, .5);
        glVertex2f(-.5, .5);
    glEnd();
    glRotatef(22, 0, 0, 1);
    glCallList(MY_SQUARE);
    glRotatef(22, 0, 0, 1);
    glBegin(GL_POLYGON);        // Will display in red
        glVertex2f(-.5, -.5);
        glVertex2f(.5, -.5);
        glVertex2f(.5, .5);
        glVertex2f(-.5, .5);
    glEnd();
    glFlush();
}

void init() {
    glClearColor(1.0, 1.0, 1.0, 1.0);
    glNewList(MY_SQUARE, GL_COMPILE);
        glPushAttrib(GL_ALL_ATTRIB_BITS);
        glPushMatrix();
        glBegin(GL_POLYGON);
            glColor3f(1, 0, 0);
            glVertex2f(-.5, -.5);
            glVertex2f(.5, -.5);
            glVertex2f(.5, .5);
            glVertex2f(-.5, .5);
        glEnd();
        glPopMatrix();
        glPopAttrib();
    glEndList();
}

Display Lists and Modelling

• Can invoke other lists within list definitions
• Allows for hierarchical specification of models

static void display(void) {
    glClear (GL_COLOR_BUFFER_BIT);
    glCallList(CAR_BODY);
    glFlush();
}

void init() {
    GLfloat c = (M_PI/180);

    glClearColor(1.0, 1.0, 1.0, 1.0);
    glNewList(WHEEL, GL_COMPILE);
        glPushAttrib(GL_ALL_ATTRIB_BITS);
        glPushMatrix();
        glColor3f(0, 0, 0);
        for (GLint i = 0; i <= 360;  i += 10) {
            glBegin(GL_TRIANGLES);
                glVertex2i(0, 0);
                glVertex2f(.1 *cos(c * i), .1*sin(c * i));
                glVertex2f(.1*cos(c * (i+10)), .1*sin(c * (i+10)));
            glEnd();
        }
        glPopMatrix();
        glPopAttrib();
    glEndList();

    glNewList(CAR_BODY, GL_COMPILE);
        glPushAttrib(GL_ALL_ATTRIB_BITS);
        glPushMatrix();
        glBegin(GL_POLYGON);
            glColor3f(1, 0, 0);
            glVertex2f(-.5, -.5);
            glVertex2f(.5, -.5);
            glVertex2f(.5, .5);
            glVertex2f(-.5, .5);
        glEnd();
        glTranslatef(.5, -.5, 0);
        glCallList(WHEEL);
        glTranslatef(-1, 0, 0);
        glCallList(WHEEL);
        glPopMatrix();
        glPopAttrib();
    glEndList();
}

Programming with Event-Driven Input

• Using GLUT functions only
• Only a small subset of actual events that can be handled by platform-specific functions

Working with the Mouse

• Two types of events:
  • move event - mouse moved with one of the buttons pressed
    • passive move event - move without button pressed
  • mouse event - button pressed or released
• information (measure) returned:
  • which button pressed
  • the state of the button after the event (up or down)
  • coordinates of the mouse in window coordinates -- origin at upper left
• glutMouseFunc - registers callback for mouse events

  void mouse(int button, int state, int x, int y) {
   ...
  }
  ...
  main() {
   ...
   glutMouseFunc(mouse);
   ...
  }
  		 

• glutMotionFunc(void (*func)(int x, int y))
  • mouse drag (button pressed)
• glutPassiveMotionFunc(void (*func)(int x, int y))
  • mouse motion with no button pressed
• glutReshapeFunc(void (*func)(int width, int height))
  • width and height of resized window
• glutKeyboardFunc(void (*func)(unsigned char c, int x, int y))
  • x and y represent the mouse location when key was pressed
• glIdleFunc(void (*func)(void))
  • Called repeatedly when no other events are being handled
• To change callback, call with new function; to clear callback, set to NULL

Callback-Related Coordinates

• The coordinates sent to the callback are in terms of window coordinates
  • Origin is upper left
• OpenGL has world coordinate system with origin at lower left
• y coordinate must therefore be inverted
  • Given height of window, h, we get y = h - y

Window Management

• int glutCreateWindow(char *title)
  • Must define callbacks on a window-by-window basis
• void setWindow(int)
• int getWindow()

Menus

• One of the (software) widgets we referred to earlier
• Nothing platform-specific required so can be defined in GLUT
• Need to be able to:
  1. Specify menu structure
  2. Actions to be taken on menu item selection
• Basic functions
  • int glutCreateMenu(menu handler)
    • function pointer (void (*func)(int))
  • void glutAddMenuEntry(char *menuText, int selectionValue)
  • void glutAttachMenu(int mouseButton)
  • void glutAddSubMenu(char *menuText, int menuId)
• Single-level menu

  glutCreateMenu(shapeMenu);
  glutAddMenuEntry("rectangle", 1);
  glutAddMenuEntry("triangle", 2);
  glutAddMenuEntry("oval", 3);
  glutAddMenuEntry("polygon", 4);
  glutAttachMenu(GLUT_RIGHT_BUTTON);
  ...
  void shapeMenu(int selection) {
  		 switch (selection) {
  		 		case 1: drawRectangle(); break;
  		 		case 2: drawTriangle(); break;
  		 		case 3: drawOval(); break;
  		 		case 4: drawPolygon(); break;
  		 }
  		 glutPostRedisplay();			 // to remove image of menu from display
  }
  		

• Hierarchical menu

c_menu = glutCreateMenu(color_menu);
glutAddMenuEntry("Black",0);
glutAddMenuEntry("Red",1);
glutAddMenuEntry("Green",2);
glutAddMenuEntry("Blue",3);
glutAddMenuEntry("Cyan",4);
glutAddMenuEntry("Magenta",5);
glutAddMenuEntry("Yellow",6);
glutAddMenuEntry("White",7);

glutCreateMenu(main_menu);
glutAddMenuEntry("new polygon", 1);
glutAddMenuEntry("end polygon", 2);
glutAddMenuEntry("delete polygon", 3);
glutAddMenuEntry("move polygon", 4);
glutAddMenuEntry("quit",5);
glutAddSubMenu("Colors", c_menu);
glutAttachMenu(GLUT_MIDDLE_BUTTON);

...

void color_menu(int index) {...}
void main_menu(int index) {...}
	 	

The square Program of the Text

• Leaves a trail of Squares in its wake when mouse is dragged with left button
• exits on right button click
• Handles resize

Redisplaying the Squares on Redraw

• Use vector
• Iterate through vector on redisplay

Picking

• Not so bad when dealing with a purely two dimensional image
• Becomes difficult if not impossible in a 3D pipleined system
• Several ways of achieving picking in a modern graphics system
  1. Associate with each object of interest, its aligned, bounding rectangle (smallest rectangle that can enclose the object and is parallel to the axes).
     • Applications can then achieve approximately accurate picking through this association.
  2. Use the back buffer to maintain unique color versions of the objects of interest
     • The back buffer is the one in which an image is built during double-buffering (the front buffer is the one being displayed).
     • Placing objects into the back buffer has no effect on the display
     • We can then get the mouse's position, retrive the pixels at that position, determine the color, and thus the object.
     • Normal rendering can then follow (assuming we are performing double buffering.
  3. Selection
     • Narrow down the area of interest (near the cursor) and only look at those objects within that area.
     • Place these objects (of interest) on a hit list
     • Provide this list to the application program for subsequent examination

The Viewport

• Just briefly mentioned until now
• Maps to a portion of the window (often the whole window)
  
• (Complete) Viewing volume in turn maps to (complete) viewport
• Viewport effectively determines size of viewing volumne in window
• Important to maintain same aspect ratio as window, or get distortion

void processHits (GLint hits, GLuint buffer[]) {
   unsigned int i, j;
   GLuint names, *ptr;

   printf ("hits = %d\n", hits);
   ptr = (GLuint *) buffer;
   for (i = 0; i < hits; i++) { /*  for each hit  */
      names = *ptr;
      ptr+=3;
      for (j = 0; j < names; j++) { /*  for each name */
         if(*ptr==1) printf ("red rectangle\n");
         else printf ("blue rectangle\n");
         ptr++;
      }
      printf ("\n");
   }
}

The CAD Program

• Functionality
  • Manipulation of polygons: creating, moving, deleting
  • Color selection
• Our Implementation
  • Vertex, Color, BoundingBox, and Polygon classes
  • Vectors of vertices
  • Dynamic, state-driven menus
  • Uses bounding box picking
  • Drag-and-drop moving
  • Point-and-delete
  • Pseudo-rubberbanding

Working With Shape Objects

• Idea is related to display lists and instances (objects) of classes
• Want to be able to predefine graphic shapes
  • Makes it easy to retrieve such shapes and insert them into the image
• First step will be to define a function for each shape and use menu to allow the user to select and manipulate the shape.
• We also need to keep track of shapes inserted into the system, so we define a class hierarchy for shapes

  class Shape {
  
  };
  class Square : public Shape {
  public:
  };
  		

• We can introduce a vector to hold the shape objects

Animation

• Periodically modify the image to provide the illusion of motion
• Rotating square

  double theta = 0;
  while (true) {
  	glBegin(GL_POLYGON);
  		glVertex2f(0.5, 0.5);
  		glVertex2f(0.5, 0.-5);
  		glVertex2f(-0.5, -0.5);
  		glVertex2f(-0.5, 0.5);
  	glEnd();
  	theta += 10;
  }
  		

• Need to display each image modification

  double theta = 0;
  while (true) {
  	glBegin(GL_POLYGON);
  		glVertex2f(0.5, 0.5);
  		glVertex2f(0.5, 0.-5);
  		glVertex2f(-0.5, -0.5);
  		glVertex2f(-0.5, 0.5);
  	glEnd();
  	theta += 10;
  }
  		

Double Buffering

• User will see image update in form of flicker or...
• Need to perform redraw away from prying eyes
• Need two buffers
  • One is displayed, while....
  • ... second is being updated
  • When done... swap buffers and repeat
• OpenGL provides double buffering
  • glutInitializeDisplayMode
  • glutSwapBuffers()

Timers

• Need some way of slowing down the display
• OpenGL supplies a timer callback that can act as a delaying element

Logic Operations

• Given two bits, there are sixteen functions that can be applied

  	0 0	0 1	1 0	1 1
  0	0	0	0	0
  &	0	0	0	1
  !->	0	0	1	0
  b1	0	0	1	1
  !<-	0	1	0	0
  b2	0	1	0	1
  ^	0	1	1	0
  |	0	1	1	1
  !|	1	0	0	0
  <->	1	0	0	1
  !b2	1	0	1	0
  <-	1	0	1	1
  !b1	1	1	0	0
  ->	1	1	0	1
  !&	1	1	1	0
  1	1	1	1	1

• If we consider this in the context of the new pixel and current pixel value, this becomes an issue of blending the old and new pixel values.

  

• We are interested in two modes:
  • copy/replacement (b1 in the above, assuming b1 is the new pixel)
    `
    • Completely overwrites the original value
  • XOR (^)
    • Has the useful property (as we'll see) of b1 ^ b2 ^ b1 = b2

Rubberbanding using XOR Logic Operation

• Use functions
  • glEnable(GL_LCOLOR_LOGIC_OP)
  • glLogicOp(GL_XOR) (Reset using GL_COPY)
• First time through the color will xor with the current pixel color, producung the new image in a weird color (the blending)
• Redrawing the line erases it (a ^ b ^ b = a)
• Allows one to update just the relevant portion of the display (rather than a full redraw as we do in the CAD program).