Chinese team develops vision chip with complementary pathways for open-world sensing

(ECNS) -- A Chinese research team has developed a vision chip with complementary pathways for open-world sensing. Their achievements were featured as the cover article in the international academic journal Nature on Thursday.

Based on the vision chip, Tianmouc, the team has developed high-performance software and algorithms, and validated their performance on an open-environment in-vehicle platform. In various extreme scenarios, this system has achieved low-latency, high-performance real-time perception and inference, demonstrating its application potential in the field of intelligent unmanned systems.

Professor Shi Luping, the corresponding author of the paper and a professor from Tsinghua University, explained that in an open world, intelligent systems must handle not only large amounts of data but also extreme events such as sudden dangers in driving scenarios, drastic light changes at tunnel entrances, and strong flash interference at night.

Traditional visual perception chips often experience distortion, failure, or high latency in these scenarios, which limits the stability and safety of the system.

To better address these issues, the team has focused on brain-inspired visual perception chip technology and proposed a new visual perception paradigm based on visual primitives and a complementary dual-pathway approach.

This paradigm draws on the fundamental principles of the human visual system, breaking down the visual information of the open world into representations based on visual primitives, said Shi. "By organically combining these primitives, it mimics the characteristics of the human visual system, forming two complementary and information-complete visual perception pathways.

The corresponding author of the paper, Professor Zhao Rong from Tsinghua University, stated that the Tianmouc Chip paves the way for important applications such as autonomous driving and embodied intelligence. Combined with the team's technical accumulation in brain-inspired computing chips (Tianjic Chip), brain-inspired software toolchains, and brain-inspired robotics, the Chip will further enhance the brain-inspired intelligence ecosystem and strongly promote the development of artificial general intelligence.