Links
Overview
Eric Chen and I built Pet Maker for the Fetching Reality challenge. The experience combines generated characters with a backyard environment reconstructed from a photograph.
Process
TypeScript and Three.js handle rendering and interaction. Next.js supports image-to-3D generation through fal.ai and rigging through Tripo. MediaPipe tracks both hands; smoothed landmarks and estimated reach connect gestures to virtual contact. A World Labs environment mesh provides the shared space for the character and ball.
Reflection
Hand tracking, camera perspective, contact detection, and character rigs all had to agree for petting and fetch to feel consistent. Different bone names and proportions also made reusable animation difficult.
Gallery
