A Framework for Bearing-Only Sparse Semantic Self-Localization for Visually Impaired People

Self-localization in indoor environments is a critical issue for visually impaired people. Most localization approaches use low-level features and metric information as input. This can result in insufficient output for visually impaired people since humans understand their surroundings from high-level semantic cues. They need to be provided their location with respect to the objects in their surroundings. Thus, in this work, we develop a novel framework that uses semantic information directly for localization, which can also be used to inform the user about his surroundings. The developed framework directly uses sparse semantic information such as the existence of doors, windows, tables, etc. directly within the sensor model and localizes the user within a 2D semantic map. It does not make use of any distance information to each semantic landmark, which is usually quite difficult to obtain. Nor does it require any kind of data association- the objects need not be uniquely identified. Hence, it can be implemented with simple sensors like a camera, with object detection software. For our framework, one of the most popular game engines, Unity was chosen to create a realistic office environment, consisting of necessary office items and an agent with a wide-angle camera representing the user. Experimentally, we show that this semantic localization method is an efficient way to make use of sparse semantic information for locating a person.