Research

Exploring the intersection of robotics, perception, and human interaction. Current focus on multimodal AI for autonomous characters.

Efficient Multimodal Tokenization for Real-Time Social Perception in Autonomous Robotic Characters

Investigating how autonomous robots can perceive and respond to human social cues in real time. The core challenge is building a tokenization scheme that fuses vision, audio, and proprioceptive data streams efficiently enough to run on edge hardware like the Jetson Orin, while maintaining the latency requirements needed for natural human-robot interaction.

Multimodal TransformersEdge AISocial RoboticsReal-time SystemsHRI

Build Progress

Documenting the mechanical design, control systems, and behavior layer development of a self-balancing inverted pendulum robot with character-driven social interaction.

Mechanical DesignComplete
PID Control LoopComplete
ROS2 IntegrationIn Progress
Behavior LayerPlanned
Social PerceptionResearch

Multimodal tokenization for real-time social perception

In Progress2026

Edge-optimized transformer architectures for robotic perception

Planned2026

Character behavior modeling in autonomous social robots

Planned2025
EOF

Joey Schnepel — Phoenix, AZ — 2026