Phiji

Role

AI & Computer Vision Lead

• Integrated AI into Phiji, a physical/digital toy for educating children through interaction and entertainment
• Developed and implemented computer vision algorithms and models for various applications such as object detection, tracking, and face recognition
• Collaborated with Unity, Android, IOS, and software teams to integrate CV models into production
• Conducted experiments, analyze results, and iterate on algorithms to improve accuracy and speed
• Implemented and optimized real-time offline speech-to-text models
• Performed data collection and automatic annotation to prepare large datasets (+3M images of game tokens) for training and evaluation
• Optimized models for real-time deployment on Android and IOS mobile phones
• Developed data generation by placing objects on backgrounds in real-world scenarios, used as a train dataset for annotator models
• Designed and implemented multi-stage algorithm to correct annotation errors
• Used: TensorFlow, TensorFlow-Lite, Python, SSD architecture, Image Processing, YOLOv8, FaceNet, Multi-Object Tracking, OpenCV

What (Who) is Phiji?

Phiji is an AI-powered smart toy for education through entertainment and interaction.

Education:

Phiji teaches letters, words, pronunciation, numbers, mathematics, and enhance a significant range of cognitive abilities through diverse games.

Interaction:

Phiji is an AI-powered robot character designed to interact with children using both visual and auditory perception.

• Visual Perception:
• Phiji observes its surroundings through its “eyes,” which are the front and rear cameras of a smartphone.
• Phiji can identify children, recognize their emotions, and interpret their facial expressions.

• Phiji detects objects in the environment and can identify elements moved by the child in games, all in real-time.

• Auditory Perception:
• Phiji listens and analyzes a child’s speech using its “ears” (microphones) and responds through its “voice” (speakers) to communicate effectively with the child.

Entertainment:

• Phiji offers a wide range of diverse games for children.

Phiji Components:

Application:

Phiji’s application can run on all smartphones and tablets supporting Android and iOS operating systems.

Stand:

Phiji includes a specially designed stand that serves as a mobile phone holder. This stand is uniquely invented, designed, and manufactured to position the phone’s rear camera in front of a mirror. The mirror reflects the image below the stand to the phone’s rear camera, while the phone’s front camera simultaneously captures the user’s face.

Physical Tokens:

Physical tokens play a key role when a child interacts with Phiji. These tokens are placed under the stand and used for both play and learning activities with Phiji. The tokens include Persian and English alphabets, Persian and English numbers, cartoon characters, and a spaceship.

Games

Alphabet Game

The child learns to write letters, form various words, and pronounce them. In this game, the child assembles physical alphabet tokens to create predetermined words. Phiji uses AI to recognize the letters and offer feedback to the child.

As the game progresses, Phiji prompts the child to pronounce each word. It listens to the child’s pronunciation and provides feedback, indicating whether the pronunciation is correct or incorrect.

Number Game

Phiji helps the child learn numbers and mathematical operations, such as addition and subtraction. The learning process begins with simple tasks and gradually moves to more complex ones. The child arranges physical number tokens to perform these operations, and Phiji uses AI to detect the numbers and provide feedback on the accuracy of the results.

Memory Game

This game is designed to enhance the child’s memory skills. Various shapes are briefly displayed for the child to memorize before they disappear. The child is then asked to place physical tokens with those shapes in front of Phiji, which uses AI to recognize the shapes and provide feedback on their accuracy.

Galaxy Game

The game takes place in a galactic environment where the child pilots a spaceship and uses lasers to target enemy creatures. The game progresses from easy to difficult levels, requiring the child to quickly maneuver the spaceship token to advance. Phiji, using AI, recognizes the spaceship’s movements in real-time, allowing the child to see its position on the game screen.