PhD Digital Media


SciSketch is a tabletop tool for sketch-based problem-driven collaborative learning in the sciences. The process of sketching can help students better understand scientific concepts. The system tracks multiple pen inputs on a tabletop display surface and can transmit sketch data to a remote computer. The first prototype provides basic functionality of digital sketching. We study how such a tool could be incorporated into the classroom environment for undergraduate courses in biomedical engineering.

NE1 View

NE1 View is a full-body interactive environment that extends the view of a person from the physical space to a virtual space. People who enter the space are tracked by the system. Every few seconds, one individual would be randomly chosen, indicated by a spot light. The body movement of this person is then translated by the system to control the view of the virtual space.

Kinesthetics Pathways

Pathways assists system biology researchers in manipulating the parameters more easily by adopting TUIs for visualization, and also seeing the system-wide effects of their manipulation across the reaction network. In current practice in systems biology, researchers run simulation programs that model different experimental parameters. Pathway can provide effective support to the researchers to discover hidden patterns in the reaction network by plotting the output using graphs.

Physical/Digital Puppetry

Pictures at an Exhibition is a physical/digital puppetry piece that uses tangible interface puppets to modify a virtual scene projected at the back of the stage in real-time. The actions of physical puppets controlled by live puppeteers tell the story of a heist in an art gallery, while simultaneously leaving their traces on an artwork that exists within their space. The piece merges traditional puppeteering practices with tangible interaction technologies and virtual environments to create a novel performance for the live stage.


It's an interactive game that players can play with gestures. His super power is to control the nature and conduct a symphony. Four Wii remotes and motion plus attached to Hawl's arms and legs capture the movement of Hawl and send the data to a computer through Bluetooth.

Pseudo Transparent Display

A transparent display is a display that allows people to see through it from either the front or the back. We designed and engineered a one-way pseudo transparent display technique that allows users in front of a monitor to see through the screen.

Clean-Hands: Promotion of Hand Hygiene with Ubiquitous Sensors and Wearables

Clean-Hands is a project that explores solutions to promote the hand hygiene in order to reduce cross-infection through contact. Research has shown that existing solutions succeed in some aspects but fail in others. The proposed solution is a combination of current methods, improved alternatives and a wearable device that keeps track of the health care workers' behavior.

Kudu: Creativity Collaboration through Tabletop

I designed the application within two hours on the interactive tabletop display I built. The application has two modes, one is the free-sketching mode and the other one is the object-manipulation mode. The free-sketching mode resembles to Finch's original experiment. The other mode is a new creativity evaluation method, which utilizes the benefit of this digital medium. It allows users to rotate, scale and move digital shapes with their hands that traditional paper and pen interface cannot provide.

Tangible Optical Chess

Optical Chess is a strategic board game created using the theoretical idea of lasers and mirrors, as well as many of the concepts and terminology from the more standard game of Chess. The game's initial objectives were to be easy to learn, to be difficult to master, to be strictly strategic with no element of randomness. I created a tangible version of Optical Chess for users to play optical experiments on this interactive tabletop display without setting complicated environments.

Interactive Play Spaces

In this project-based course, students develop the conceptual, aesthetic and technical basics of physical/digital co-design in the creation of interactive playful experiences for a multi-generational audience. I taught basic electronics including designing fundamental circuits, using passive and active components, using sensors, and programming Arduino boards. The final project was an outdoor installation/playground. It uses batteries and solar cell panels to power the system. It uses photocells to shutdown in the evening and turn on again in the morning. When visitors approach it, it plays sounds based on the location of the visitors and their distance to the sensors.


TwinSpace is a flexible software infrastructure for combining interactive workspaces and collaborative virtual worlds. Its design is grounded in the need to support deep connectivity and flexible mappings between virtual and real spaces to effectively support collaboration. This is achieved through a robust connectivity layer linking heterogeneous collections of physical and virtual devices and services, and a centralized service to manage and control mappings between physical and virtual.

Tangible Tracking Table (TTT) II

To explore more functional possibilities, the first prototype (TTT I) adopted two popular technologies that are used in many multitouch displays today, the "Frustrated Total Internal Reflection" (FTIR) and the "Diffused Illumination" (DI) approaches. The second table (TTT II) uses DI and installs two mirrors to increase the display and interaction surface to 55 inches. I built the technology inside the table and students from the Department of Industrial Design designed the case. I improved the stability of the table by aligning the optical path of the infrared camera with the projector. With the help of Synlab members, I upgraded the tabletop surface to a non-reflective plastic in 2011.

Tangible Visualization

Tangible Visualization is defined as the formation of a tangible representation of an abstract concept. Interactive visualizations existing in our physical environment include interactive art installations that show abstract data, interactive tabletop displays that have more interactions than traditional vertical LCD/CRT displays and interactive ambient displays.

Responsive Space

Seeing multiple moons visible overhead is not our normal experience in the real world. Even though we can picture this kind of experience in our imaginations, we cannot properly share it with others. The intention of this project is to share this experience by creating a poetic installation space of virtual and physical, where multiple moons can follow us around. And, the audience can share their own imaginative and individual experience with others.


ROSS stands for Responsive Objects, Surfaces and Spaces. The ROSS API simplifies cross-device communications, such as tabletop computers, touch-screen mobile devices, reponsive walls and interactive 3-D spaces. The ROSS API allows applications to: exchange information about devices they are running on and receive real-time input data from other ROSS-enabled devices.


The KinoPuzzle Project discovered how to combine the expressive power of the collage surface or interface with the representational depth and flexibility of the digital database to present new documentary forms in the digital age.

Synlab Website

I volunteered to design the information architecture of the Synaesthetic Media Lab website and engineer better interaction flow within the site. The site was designed using PHP and the MySQL database. The site includes a password protected administer interface to enter and edit data. The administrators can easily upload new files and assign associated persons through this interface.

Tangible Tracking Table (TTT) I

To explore more functional possibilities, the first prototype (TTT I) adopted two popular technologies that are used in many multitouch displays today, the Frustrated Total Internal Reflection (FTIR) and the Diffused Illumination (DI) approaches. The second table (TTT II) uses DI and installs two mirrors to increase the display and interaction surface to 55 inches.

MS Human Computer Interaction

WikiTUI: leaving digital traces in physical books

WikiTUI is a tangible user interface system that allows readers to manipulate multimedia information tied to page-level locations in common books. WikiTUI can also facilitate the exchange of information with other readers using wiki technology. WikiTUI not only bridges the real and digital worlds, but facilitates multiple contributions to a reference base spanning across these worlds.

LCC 6313 - Principles of Interactive Design

This is the class project of LCC 6313 - Principles of Interactive Design. I designed two project, one was a google map mashup integrated with Marta information and the other one was a Firefox plugin that shows TV schedule.

Share Projectors

This project illustrates a solution to share a projector through wireless network and a feasible user evaluation plan to verify this solution. Users don't have to plug/unplug VGA cables using our sharing software. In the evaluation part, users will be asked to make a photo calendar with both the software and the traditional projector with VGA cable collaboratively. We believe this software will increase the efficiency of work involving projectors.

HCC Education Digital Library

This digital library and web portal contains material useful for undergraduate and graduate education in Human-Centered Computing, Human-Computer Interaction and related areas. I designed the website layout, information architecture, taxonomy, programmed the front-tier and back-end part of the website.

Real-time Fingertip Tracking

I implemented an application for real-time fingertip tracking. This program was used later on my WikiTUI project.


We built a multitouch display and SonicTouch, a music application on it.

CS 7450 - Information Visualization project: Delta Flow

Our visualization aims to display migration patterns according to demographic data from the census data set. We use triangles to enhance the effectiveness of our visualization. Migration trends are shown in different sizes and colors depending upon the particular demographic. There are 4 members in our team: Ellen Matthews Jason Pedone Eric Yang and me.

CS 6452 - Prototyping Interactive System

I built an instant messenger in the class.


PowWow is a mobile friend locator which expands the capability of traditional social network. This is a project of CS 6750 Human Computer Interaction, instructed by Prof. Gregory Abowd. There were 4 members in our team, Nils Beck, Jeremy Johnson, Sunil Parihar and I. We learned the history of HCI and evaluation methods in the class.

The digital camera of tomorrow

This is a project of PSYC 7101 Engineering Psychology, instructed by Prof. Gregory Corsor. We redesigned and improved regular consumer cameras. During this study, we found that the problem is not only the design of the camera but also the design of photo sharing mechanisms. Therefore, we redesign the camera hardware and the sharing software. The new design simplified the sharing proceduresThere are 5 members in our team, Cara Bailey, Matt McKeon, Vanessa Rood, Dan Shaw and I.

Software Development

RealTime Remote Replication (RTRR)

The idea is quite simple - copy the data once they're written. However, there's no open source solution on Linux that can be adopted by our NAS. I had to do it by myself. The concept is to intercept the system calls that change the monitored files and copy the changed files to another place. Hence, the project is divided into three parts: a kernel space module that intercepts the system calls, a user space dispatching daemon that schedules the job sequence and a copy mechanism to duplicate the files. The company accepted my idea and applied for patents later. This program has been running on IEI's NAS products since 2005.

NAS-5100 NAS gateway project

A Network Attached Storage (NAS) is a dedicated server does nothing more than file sharing. A NAS gateway is a NAS controller, which is able to manage Storage Area Network (SAN) storage. I am the project manager of NAS-5100 NAS gateway, which is a high performance 1U NAS Controller, adopting advanced 64 bit BUS technology with 19" industrial standard chassis. It offers the extreme flexibility for external storage expansion such as SCSI hard disk drives, RAID subsystems, and tape drives. The most important feature of this product is the Real-time Remote Replication (RTRR) function, which makes real time remote backup possible.

IEI P-square

This was a 24-hour programming workshop held by IEI in Sep. 2003. The purpose of this workshop was to test our rookie software engineers. We were invited to a luxurious SPA hotel near a beautiful beach and tortured by our boss. The goal was to build a peer-to-peer program able to search and exchange data while enjoying the facility in the hotel. We found it difficult to write any codes without references. Teamwork was the key to achieve the project. My colleagues acclaimed my presentation, which emphasized on the commercial value and the technical advantages of our product. We won the trophy!

User space NFS 3 project

One of the most significant works I did in IEI in my first year there is the user space NFS project. I learned a lot about software development from this project. Software development concerned not only planning and working the project out. This NFS project contains more than 3 platform-independent network protocols. In this case, compatibility is more important than efficiency. After the product was issued, we found that there were some compatibility problems with a few UNIX systems. Before leaving IEI, I redesigned another NFS authentication method with another kernel NFS, which should be running in IEI's high-level products now.

MS Electrical Engineering

How web image formats affect a customer's buying impulse (ii)

This project didn't end after that semester. I applied the precious data to "Database Application Technologies" in which we studied data mining technologies. I used the data obtained in former semester to predict the buying impulse of some other people who are not in the original subject set. The demonstration was good and the prediction is highly acceptable using IBM DB2 i-miner as the tool. I used Neural Network as the data mining method.

Investigation of optimal designs for liquid crystal correlation filters

This is a project supported by the National Science Council, Taiwan. Prof. Chen, my advisor, led this project and I'm a research assistant. In this project, we tried to use the transparent liquid crystal display to replace the traditional spatial light modulator (SLM). A traditional SLM is made of holographic film. The procedure of holography making is a time-consuming work. By using an LCD panel connected to a computer, we were able to change the image immediately and hence make real-time optical pattern recognition possible.

How web image formats affect a customer's buying impulse (i)

This is the first user evaluation I conducted. I designed a web site, which recorded the users behaviors during their visits to an online shopping site. On this site, several types of image formats were shown to stimulate the visitors' buying impulse. At the end of visiting the site, users had to answer a series of online questions. The study result showed that a 360-degree view of the product (a PDA) affected the visitors the most. It even surpassed the effect of an online PDA simulator.

Color pattern recognition with Non-zero Order Joint Transform Correlator (NOJTC)

Joint Transform Correlator (JTC) plays an important role in optical pattern recognition. However, its massive zero-order term confines its application. I've applied the zero-order part removal procedures to multi-channel applications and made optical color pattern recognition with NOJTC possible. I used animals as my test objects, lady beetles, frogs, butterflies and birds. I started my research in NOJTC in Sep. 2001. I presented 2 conference papers and produced 4 journal papers in optical color pattern recognition.