Introduction

Home robots are increasingly becoming an integral part of modern households, assisting with
various tasks from cleaning to security. However, these robots face numerous vision
challenges in the dynamic and often cluttered indoor environments. Newsight Technology,
with its advanced image sensor chips, offers innovative solutions to these challenges. This
white paper explores how Newsight’s technology effectively addresses the key vision
challenges in home robotics.

Vision Challenges in Home Robotics

1. Object Recognition

Challenge: Home robots must accurately identify and differentiate between various
household objects.
Solution: Newsight chips offer high-resolution depth imaging (up to 1024×480). Their
programming scheme allows for seamless integration with 3D color HD cameras, facilitating
robust object recognition. This high-resolution depth data, whether combined with 3D color
imaging or not, significantly enhances the robot’s ability to accurately recognize and interact
with household objects.

2. Low Light Conditions

Challenge: Operating effectively in dimly lit or varying light conditions is a significant
challenge for vision systems.
Solution: Newsight’s eTOF (enhanced Time of Flight) technology uses an active lidar approach,
enabling vision in all light conditions, with optimal performance even in complete darkness. By
measuring and deducting ambient light when calculating distance, this active illumination
method ensures that the robot can navigate and perform tasks reliably, regardless of ambient
lighting conditions.

3. Dynamic Environment

Challenge: Home environments are dynamic, with moving objects like pets and people
requiring real-time adaptation.
Solution: The Newsight NSI9000 chip can trade resolution for frame rate, achieving
thousands of frames per second (FPS). This high frame rate, coupled with the efficient
computation of cloud maps, allows for rapid and dynamic responses to changes, ensuring the
robot can navigate and adapt to moving objects effectively.

4. Surface Visibility

Challenge: Recognizing different floor types and surfaces, including reflective objects, is
complex.
Solution: The NSI9000 and eTOF technology are well-suited for identifying various reflective
objects. Newsight’s HDR capabilities allow for effective performance with both low and high
reflectance surfaces, aiding in navigation and task execution across diverse household
environments.

5. Reflective and Transparent Surfaces

Challenge: Reflective and transparent surfaces like mirrors and glass can confuse vision
systems.
Solution: Newsight’s gated imaging feature allows selective range imaging, filtering out
noise from floors or transparent surfaces. This selective imaging capability enhances the
robot’s ability to accurately interpret its environment, avoiding confusion caused by
reflections and transparency.

6. Occlusion

Challenge: Objects partially covered or blocked from view present a recognition challenge.
Solution: Newsight chips enable the use of 3D models for object recognition, which helps in
identifying objects even when they are partially covered or blocked from view. This capability is
further enhanced when fused with a color camera, providing more accurate and reliable
recognition in challenging scenarios.

7. Scale and Perspective

Challenge: Accurately judging distances and sizes of objects is crucial for effective
interaction.
Solution: In laboratory tests, Newsight demonstrated an accuracy with an error rate of less
than 0.1% of the measured distance. This high level of accuracy ensures that home robots can
make precise judgments about object scale and perspective, facilitating better interaction and
task execution.

8. Color and Pattern Variability

Challenge: Differentiating objects with similar colors or complex patterns can be difficult.
Solution: The NSI1000 and NSI9000 chips generate 3D images that are not affected by color or
pattern complexity, making it easier to differentiate objects with similar colors or complex
patterns. This capability significantly enhances the accuracy and reliability of object
recognition.

Conclusion

Newsight Technology’s advanced image sensor chips offer comprehensive solutions to the
key vision challenges faced by home robots. Through high-resolution depth imaging, active
lidar for low light conditions, high frame rate for dynamic environments, and selective range
imaging for reflective and transparent surfaces, Newsight enables home robots to navigate
and interact with their surroundings effectively. These innovations pave the way for more
reliable, efficient, and intelligent home robotic systems.