Visitors to a recent art opening had a chance to avert a romantic breakup and warp history to their own designs, all while an alien presence looked on, displaying its reactions on a wall-sized monitor. The artificially intelligent pieces of art are the work of Michael Mateas, assistant professor of computer science at the University of California, Santa Cruz.
Mateas and five other programmer-artists are featured in an ongoing exhibition titled Grand Text Auto, named for their collective blog about video games and art. Their work is on display through December 15 at UC Irvine's Beall Center for Art and Technology.
The name is a twist on Grand Theft Auto--perhaps the archetypal blood-guns-and-crime video game. But the artwork on show is far removed from anything you might see on a Sony PlayStation. Mateas's work attempts to build expressiveness into artificial intelligence, or AI. He writes programs capable of inspiring empathy, then uses them to study how people regard machines. He also strives to make the projects look cool.
"Each piece is simultaneously a research project, where we're doing AI research and user experience research, and it's an art piece," Mateas said. "My goal is that every piece is worthy of being shown in a gallery."
Mateas is the co-creator of Façade, a video-game melodrama structured as a 20-minute, one-act play. The game, on display at the exhibition, thrusts the player into an awkward evening at the apartment of a bickering couple. The player's participation can determine the fate of the couple's relationship.
He or she can try to smooth things over, egg the couple on, or nervously change the subject. The game understands each comment and advances the story by choosing an appropriate snippet of dialog from five hours of pre-recorded possibilities. To make it work, Mateas and collaborator Andrew Stern spent five years inventing a new programming language and teaching the computer a rudimentary understanding of English.
Also on display is AR Façade, an "augmented reality" version of the game, which is played on an actual set of the couple's apartment. The player wears special goggles that superimpose 3-D versions of the couple on the scene, allowing all three characters to move around at will.
"In a sense the impetus for AR Façade was like, 'How close can we get to the Holodeck?'" Mateas said, referring to the holographic rec room that was a staple on Star Trek: The Next Generation. "And we already had the AI part of that in Façade, so let's see if we can actually extend this out into the physical world."
But Façade and AR Façade aren't just 20-minute amusements. They've also led Mateas to some unexpected research insights. He found that the more-realistic characters of AR Façade forced some players toward social norms, making them reluctant to try out off-the-wall behavior. And many players preferred typing their words in Façade to speaking them in AR Façade.
"Having the computer you can talk to is one of these holy grails, and it's usually just sort of an untested assumption that, of course, speaking is going to be better than typing," Mateas said. "Typing is just sort of this interim step--we have to put up with these keyboards before we finally have a computer that can listen to us. Well, actually, typing has some positive features. When you're speaking, as soon as it's out of your mouth it's out of your mouth."
Mateas's other projects in the exhibit include Terminal Time, a 30-minute documentary of the past millennium that edits itself in response to audience feedback. The result is a version of history tailored to the audience's particular biases. Also on display will be Tableau Machine, a program that turns patterns of human activity into abstract art. Mateas and his collaborators designed the piece to hang in the home, where it would provide an alien perspective on ordinary life.
"It almost becomes an alien member of the family," Mateas said. "It becomes sort of like this alien dog, basically, that communicates via creating these abstract compositions."
The Tableau Machine uses four cameras to monitor its surroundings. A program dissects the images and assesses activity in terms of energy, density, and flow. At first, the machine represents the activity with vaguely mechanical shapes like circles, bulls-eyes, and pulleys. As time goes by, the program learns to recognize characteristic patterns of activity and refines the elements into a distinct style of composition for each activity.
Mateas programmed the system to understand basic tenets of composition, but steered away from giving the machine too much artistic advice. "The idea is not to build AI systems that replicate what human beings do, but that have their own kind of idiosyncratic subjectivity," he said. "Then they reflect that alien understanding back to people."
Completing the circle, Mateas has a research project under way in which a family lives with Tableau Machine for six weeks. Through periodic interviews, the family members report their impressions of what the machine has been trying to say about them.
The exhibition also features works from Grand Text Auto's five other contributors. Among them, [giantJoystick] is a fully functional, 10-foot-tall Atari joystick, so massive that it takes two people to control. Exhibit attendees can take turns playing a variety of vintage Atari games, including Space Invaders and Yar's Revenge.
In a piece called Screen, text appears on three walls of a darkened room. As the player reads the text, words appear to peel off the wall and then abruptly fly toward the viewer, who must swat the words aside before the whole passage collapses onto the floor.
The Donald R. and Joan F. Beall Center for Art and Technology opened in 2000 with the intention of melding technology with the museum experience. Exhibits often focus on applying digital technology to the tasks of creating art and forging new relationships between art and the sciences.
In addition to Mateas, the five exhibitors are Noah Wardrip-Fruin of UC San Diego; Mary Flanagan of Hunter College, New York; Nick Montfort of M.I.T.; Scott Rettberg of the University of Bergen, Norway; and Andrew Stern of Procedural Arts, L.L.C.
In addition to Mateas and Stern, AR Façade is the work of Steven Dow, Blair McIntyre, and Manish Mehta. Tableau Machine was created by Mateas, Zach Pousman, Mario Romero, and Adam Smith. Terminal Time is the work of Steffi Domike, Mateas, and Paul Vanouse. [giantJoystick] was created by Flanagan. Screen is the work of Wardrip-Fruin, Josh Carroll, Robert Coover, Shawn Greenlee, Andrew McClain, and Ben Shine.
For additional information about the exhibited projects, see the Beall Center's web site, the Façade web site, and Grand Text Auto's own running commentary about the exhibit.
Note to reporters: You may contact Mateas at (831) 459-1789 or michaelm@soe.ucsc.edu.
