Overview

I built JARVIS to explore how a wearable assistant could describe what is in front of a user and answer spoken questions. The first prototype mounted a Raspberry Pi, camera, microphone, earphones, button, and battery on a hat; later work focused on a smaller standalone form factor.

Role and constraints

I developed the assistant and worked on moving wake-word detection and speech-to-text onto a Seeed Studio XIAO ESP32S3 Sense with C++ and Edge Impulse.

Process

The first demonstrated version combined Gemini vision, Groq text generation, Azure speech services, and web retrieval. Button and wake-word activation started the interaction. I iterated toward the microcontroller prototype while gathering feedback at Vancouver AI meetups.

Result

Secured a $1,000 Ingenious+ Regional Award grant, presented at four Vancouver AI Community Meetups, and demonstrated JARVIS at the TKS Showcase. The resume records spoken output beginning in under two seconds.

Reflection

Live demos exposed microphone transcription problems, search errors, and fragile hardware mounting. I fixed the search behavior, adapted how I used the microphone, and sought connections to organizations serving visually impaired people for further feedback.

AI Meetup Demo

Ingenious+ Regional Award CAD$1K

2nd Prototype

2nd Prototype Continued