Phiji

My Role: Lead AI Engineer

• Responsible for AI in an educational hybrid physical-digital gaming system for children
• Directed training, evaluation, and deployment of CV models for object detection and face recognition.
• Managed Unity, Android, IOS, and software teams to integrate CV/NLP models into production.
• Developed a context-aware Q&A system using RAG to enhance children’s learning in interactive storytelling.
• Led data pipeline for collecting and annotating 3M+ high-quality real-world images.
• Optimized CV models for real-time offline deployment on Android and IOS devices (10-20 FPS).
• Improved efficiency of and deployed real-time offline speech recognition model (Kaldi).
• Used: TensorFlow, TensorFlow-Lite, Python, SSD architecture, Image Processing, YOLOv8, FaceNet, Multi-Object Tracking, OpenCV

\

What (Who) is Phiji?

Phiji is an AI-powered smart toy for education through entertainment and interaction.

Education:

Phiji teaches letters, words, pronunciation, numbers, mathematics, and enhance a significant range of cognitive abilities through diverse games.

Interaction:

Phiji is an AI-powered robot character designed to interact with children using both visual and auditory perception.

• Visual Perception:
• Phiji observes its surroundings through its “eyes,” which are the front and rear cameras of a smartphone.
• Phiji can identify children, recognize their emotions, and interpret their facial expressions.

• Phiji detects objects in the environment and can identify elements moved by the child in games, all in real-time.

• Auditory Perception:
• Phiji listens and analyzes a child’s speech using its “ears” (microphones) and responds through its “voice” (speakers) to communicate effectively with the child.

Entertainment:

• Phiji offers a wide range of diverse games for children.

Phiji Components:

Application:

Phiji’s application can run on all smartphones and tablets supporting Android and iOS operating systems.

Physical Tokens:

Physical tokens play a key role when a child interacts with Phiji. These tokens are placed under the stand and used for both play and learning activities with Phiji. The tokens include Persian and English alphabets, Persian and English numbers, cartoon characters, and a spaceship.

Games

Alphabet Game

The child learns to write letters, form various words, and pronounce them. In this game, the child assembles physical alphabet tokens to create predetermined words. Phiji uses AI to recognize the letters and offer feedback to the child.

As the game progresses, Phiji prompts the child to pronounce each word. It listens to the child’s pronunciation and provides feedback, indicating whether the pronunciation is correct or incorrect.

Number Game

Phiji helps the child learn numbers and mathematical operations, such as addition and subtraction. The learning process begins with simple tasks and gradually moves to more complex ones. The child arranges physical number tokens to perform these operations, and Phiji uses AI to detect the numbers and provide feedback on the accuracy of the results.

Memory Game

This game is designed to enhance the child’s memory skills. Various shapes are briefly displayed for the child to memorize before they disappear. The child is then asked to place physical tokens with those shapes in front of Phiji, which uses AI to recognize the shapes and provide feedback on their accuracy.

Galaxy Game

The game takes place in a galactic environment where the child pilots a spaceship and uses lasers to target enemy creatures. The game progresses from easy to difficult levels, requiring the child to quickly maneuver the spaceship token to advance. Phiji, using AI, recognizes the spaceship’s movements in real-time, allowing the child to see its position on the game screen.