Augmented Rock Climbing

• Implemented real-time pose estimation model to identify the locations of children’s hands and feet.
• Utilized a real-time multi-object tracker (MOT) to track children across camera frames.
• Optimized pose estimation model to reduce model size and inference time to be deployed on general-purpose computer systems.
• Integrated AI into Unity

Description

The Augmented Rock Climbing Wall Game is an interactive game, implemented in Robokids, that combines traditional rock climbing with augmented reality (AR) technology. Let’s see how it works:

1. Projection: The climbing wall is equipped with a projector that display various interactive elements such as targets, obstacles, and game graphics directly onto the wall surface.
2. Real-time Motion Tracking: A webcam track the climber’s movements in real-time. This allows the system to respond dynamically to the climber’s actions.
3. Game Mechanics: Climbers can participate in different types of games, such as hitting targets, following specific routes, or avoiding obstacles.

Interactive Painting

• designed and implemented a real-time detection and segmentation of fictional characters painted by children using image processing and augmented reality (AR) techniques

Features

• Webcam Support
• Real-time Painting Detection and Segmentation
• Windows and Linux Support
• Fisheye Correction
• Not sensitive to distance and angle between camera and paper
• Integrated with Unity

Product Description

1. Child color various drawings and place their painitng into a designated box.
2. The system uses advanced image processing to recognize and extract different elements from the painitng.
3. Once the painitng is processed, it is animated and displayed on a large screen.

For example, if a child colors a sun, the sun is brought to life on the screen, creating an effect where it appears to move and transition into daylight.