From CoIN

Contents 1 Portrait Perfecto 1.1 Members 1.2 Introduction 1.3 Documents 1.4 Project Files 1.5 Technology & Equipment 1.6 Weekly Goals 1.6.1 Wren 1.6.2 Adrian 1.6.3 Mike 1.6.4 Nisheeth 1.6.5 Wuming 1.7 Completed

Portrait Perfecto

Members

• Wren Chen
• Adrian Chow
• Michael Oh
• Nisheeth Sharma
• Wuming Xie

Introduction

• The goal of this project is to use existing technologies to create an object, which simulates the behavior of a “living” portrait as depicted in the Harry Potter film. In brief, the portrait is expected to interact with the environment through sound and visual stimuli. Other aims include engaging in a pseudo conversation, and idling when no stimulus is present. The portrait would be represented by different video clips, and through algorithms that respond to sensors, it would make smooth transitions that make the portrait look like its continuous.

Documents

• Tasks division [1]
• Proposal [2]
• Conversation Scenarios [3] and Appendix [4] - scenario updated on 4/17/09 (version 8)
• Motions List [5] - updated on 4/6/09 (version 2)
• Dialogue Tree [6] (image: [7])
• System Architecture [8]
• Face Detection Code (C#) [9] - only need to modify Program.cs (main)
• Final Project Documentation [10]
• Project Files [11]

Project Files

Technology & Equipment

• Laptop
• LCD Monitor
• Video Camera (Web Cam)
• Speakers
• Microphones
• Speech API
• Motion Sensor software (Motion Detector by Andrew Kirillov)
• Face Tracking software (FaceOnIt)
• Video Editing/Animation software

Weekly Goals

Wren

• By week 6 (2/18): Research open source or available animation software for rendering portrait
• By week 7 (2/25): Look through open source or available motion detection software
• By week 8 (3/4): Attempt a demo of motion detection software
• By week 10 (3/25): Attempt to create function to call face detection/tracking software
• By week 11 (4/1): Attempt to integrate with the rest of the program

Adrian

• By week 6 (2/18): Researched about SAPI and contacted Geo/course staff for help.
• By week 7 (2/25): Got lap-top with Vista and have voice synthesis examples ready (hopefully)
• By week 9 (3/18): Small voice recognition examples ready.
• By week 10 (3/25): Modifying I/O for both recog. and synthesis to integrate with .NET framework

Mike

• By week 6 (2/18): worked mostly on building scenario
• By week 7 (2/25): get hands on face recognition equipment and learn / produce some samples, while constantly updating scenario
• By week 8 (3/4): update scenario, play with face recognition code and work with Wren (motion detection)
• By week 10 (3/25): finish compiling face detection code
• By week 11 (4/1): integration
• By week 12 (4/8): story expansion/correction (near completion) / character building
• By week 13 (4/15): wrapping up

Nisheeth

• By week 7 (2/25) figured out either a means of integrating Processing and the .Net framework or a means of importing and modifying 3D models using Direct X. Failing that there is always C and OpenGL, though it is unlikely that we will have to resort to that. Hope to have a sample user feedback loop with the animation by then.
  • Sample User feedback loop completed. Settled on using Panda 3d.

• By week 8, have a sample humanoid animation with the character created and animated in Maya 2008 imported into the Panda3D framework and have a sample animation of said character ready.
  • Sample character is done and imported. Working now on importing skeleton animations for this character.

Wuming

• Week 7 (2/25): Work with the .NET frame work. Hopefully will start integrating SAPI and processing with Adrian and Nisheeth.
• Week 8 (3/01): Continue working on the the control.

• Week 11(4/06): Help build grammar for Speech Recognition. Look at OpenCV as an alternative option to Face Recognition.
• Week 15(5/04): Install new voice for project. Got David from www.cepstral.com. Finishing project documentation.

Completed

• Basic animation
• Basic voice synthesis
• Face Detection
• Scenario