Painters used to look to nature for their colors, extracting blues and red from plants and clay. A new digital paintbrush takes the practice to an amazing new level.

Artists can now sample colors, textures and even movements directly from the environment for use in a composition. The eyes in one painting created with the device actually blink at the viewer. They’re digitzed human eyes, from a real human.

The eyes were captured using a unique the new device called the I/O brush. Color, patterns and animated pictures are transferred to a digital screen where they become part of a work of art that can’t be categorized as part of any current genre.

Except for an electric cord snaking out from one end, the device looks like a normal wooden paintbrush with bristles at one end. Its inventors envision adding the ability to paint with sounds and smells in the future.

Started as a toy

Embedded inside the I/O brush is a small video camera surrounded by white LEDs (light emitting diodes) and pressure sensors. The video camera captures any color, texture or movement placed in front of it and sends it to a computer, where the image is saved as part of a digital palette.

The device was developed by a team of researchers at the Media Lab at the Massachusetts Institute of Technology (MIT). The team included Hiroshi Ishii, a media arts and sciences professor, and his graduate students, Kimiko Ryokai and Stefan Marti.

When the group first began developing the brush in 2003, the goal was to use visual arts projects to encourage creative learning in children. The kindergarten students that served as test subjects loved the new toy. When left alone with the device, the children would spend up to a third of their time just looking around for items they could turn into ink.

Among the items chosen were beads, stuffed animals, toys, picture books, leaves and feathers. They even digitized each other, sampling hair, belly buttons, wiggling tongues and of course, blinking eyes.

History repeated

The I/O brush also has a history feature that makes it interactive in another way as well. The computer saves the order in which an artist saves inks. These can be recalled as pop-up movies on the digital screen simply by touching different parts of the painting.

“With this feature, the portrait can take both the artists and audience back through the journey and reveal the stories behind the special palette of colors,” the researchers wrote in a science article about their invention.

This history feature is also intended to remind artists and viewers about the context in which the work of art was created.

The portrait a child creates with the I/O brush in the classroom, for example, will differ from one they create at home because the palette of colors and textures available in those two environments will be different.

Synesthetic art

Future versions of the I/O brush might incorporate sounds and smells, the researchers say. This would allow artists to create synesthetic paintings, where one kind of sensory stimulation is combined with another, sometimes in unexpected ways.

“What would it mean to mix our favorite music with the pattern of our favorite shirt?” the researchers wonder.

Another variation would be to analyze things like the tempo and loudness of sounds and music and then use that information to create visual patterns. Aggressive, fast music, for example, could create dark lines with jagged patterns while slow new age music could result in pastel colors with smooth patterns and high transparency.

Smells could be incorporated too. Users could try to “pick up” the soft smell of a rose, translating its fragrance into a visual pattern that might be completely different from, say, an onion.

Digital Newspapers Almost HereRare but Real: People Who Feel, Taste and Hear ColorRenaissance Artists Added Glass to PaintDa Vinci Sketch Hidden Under Painting