SeePal is an intelligent indoor navigation system designed for blind and low-vision (BLV) users, helping them orient, localize, and safely navigate unfamiliar spaces. By combining computer vision, spatial mapping, haptic feedback, and conversational AI, SeePal provides real-time guidance that mimics how a sighted companion describes and interprets the environment. The project is grounded in accessibility research and inclusive design, exploring how AI can augment spatial awareness and independence for BLV users.
Indoor environments often lack coherent navigation cues for BLV individuals—GPS fails indoors, signage is inaccessible, and audio descriptions are inconsistent. SeePal addresses this gap by translating visual information into multimodal feedback through touch, audio, and natural-language interaction. The system reconstructs spatial context by detecting objects, identifying navigational landmarks, and offering step-by-step guidance that adapts to the user’s orientation and distance from target locations.
SeePal integrates a 4K camera, microcontroller, and haptic motor with AI-driven interpretation. Python-based vision modules analyze indoor scenes, while ChatGPT powers high-level instruction, conversational correction, and context-aware guidance. Haptic feedback reinforces directional cues, enabling users to form a mental map without visual input. The workflow—see → interpret → guide → confirm—ensures that information is delivered clearly and redundantly across multiple senses.
The design process involved rapid prototyping in Blender and interface development in Figma to explore how spatial instructions should be structured for non-visual users. Motion paths, object markers, and route descriptions were iteratively tested to achieve clarity and semantic consistency. The final prototype demonstrates SeePal’s ability to combine AI reasoning with embodied cues, offering an accessible and intuitive navigation experience.
How can AI reconstruct visual environments into navigable, multimodal experiences for blind and low-vision users?
