LightMem-Ego: 你的日常AI记忆系统

Jul 16, 2026

LightMem-Ego is a lightweight streaming multimodal memory system designed for everyday-life assistance on smartphones and AI glasses. It continuously captures egocentric visual and audio streams, aligns them on a shared timeline, and organizes experiences into a hierarchical memory of current, short-term, and long-term levels. For a user query, a dynamic memory router retrieves evidence from the appropriate temporal scope and generates grounded answers. The system supports object finding, conversation recall, life summarization, routine discovery, and personalized assistance.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages9

Reading Time23 min

Introduction

Challenges of Everyday Egocentric Memory

LightMem-Ego System Overview

Related Work

Conversational Memory

LightMem-Ego: Your AI Memory for Everyday Life Yijun Chen 1 * , Boyi Xiao2 * , Yixian Zhao 1 * , Haoting Xia3 * , Buqiang Xu 1 , Jizhan Fang 1 , Yanya Li 1 , Yaqi Zheng 1 , Xuehai Wang 1 , Zirui Xue 1 , Liuxin Zhang4 , Hui Li4 , Ningyu Zhang 1† 1Zhejiang University 2 South China University of Technology 3 Central China Normal University 4Lenovo Group Limited [ie_yijunchen@outlook.com](ie_yijunchen@outlook.com) , [zhangningyu@zju.edu.cn](zhangningyu@zju.edu.cn) arXiv :2607 . 1 1487v 1 [ cs .CL] 13 Jul 2026 Abstract Personal AI assistants on mobile and wearable devices continuously perceive users’ daily lives through visual and audio streams. However, answering queries about past experiences requires lightweight multimodal memory that can continuously accumulate, organize, and retrieve long-term experiences, which remains challenging. To address this challenge, we present LightMem-Ego, a lightweight streaming multimodal memory system for everyday-life assistance. The system continuously captures egocentric visual and audio streams, aligns them on a shared timeline, and organizes them into a hierarchical memory consisting of current, short-term, and long-term memory. Given a user query, LightMem-Ego dynamically routes retrieval to the appropriate memory level and generates answers grounded in multimodal evidence. The demonstration can be deployed on smartphones and AI glasses, supporting object finding, conversation recall, life summarization, routine discovery, and personalized assistance 1. 1 Introduction Personal memory is a fundamental part of everyday intelligence. People constantly rely on memory to recall where they placed an object, what someone just said, what happened during a meeting, or how their routines change over time. As smartphones and AI glasses become natural interfaces for capturing egocentric visual and audio streams, they create a new opportunity for AI assistants: instead of only answering isolated questions, an assistant can become a memory companion for everyday life (Huang et al., 2025b ; Yang et al., 2025, 2026) . Such a system could help users find misplaced items, recall recent conversations, summarize daily activities, and reason over long-term habits and routines * † 1 Equal contribution. Corresponding author. [https://github.com/zjunlp/LightMem-Ego](https://github.com/zjunlp/LightMem-Ego). (Yang et al., 2025 ; Tang et al., 2026 ; Gao et al., 2026 ; Deng et al., 2026) . Recent multimodal large language models have made human-computer interaction more natural by combining vision, speech, and language (Long et al., 2025 ; Yeo et al., 2025 ; OpenAI, 2024 ; Gemini Team, 2025) . Meanwhile, wearable devices and smartphones provide practical channels for continuous, hands-free perception in daily environments (Huang et al., 2025b ; Yang et al., 2025, 2026 ; Jiang et al., 2026 ; Deng et al., 2026) . However, turning such interfaces into everyday memory assistants requires addressing three fundamental challenges: First, egocentric experience arrives as continuous visual-audio streams without explicit event boundaries, requiring assistants to transform raw observations into coherent event-level experiences. Second, continuously accumulated experiences must be incrementally organized into current, short-term, episodic, and semantic memory while remaining efficient for long-term deployment. Third, user queries naturally span multiple temporal horizons, requiring dynamic routing across different memory levels instead of relying on a single context window or flat retrieval store. Together, these challenges call for an experience-centric memory system that can continuously capture, organize, retrieve, and reason over everyday life (Tang et al., 2026 ; Xiao et al., 2026 ; Wang et al., 2026 ; Alam et al., 2026 ; Gao et al., 2026) . To address these challenges, we present LightMem-Ego, a deployable streaming multimodal memory system for everyday-life assistance. LightMem-Ego conne

FAQs

LightMem-Ego如何将日常视觉与音频流转化为可检索的记忆？

系统持续捕获自视角的视觉与音频流，并将它们对齐到共享时间轴上。随后把经验组织成当前记忆、短期记忆与长期记忆的分层结构。

当用户提出问题时，LightMem-Ego如何选择检索范围？

它使用记忆路由器，根据问题的时间范围与意图动态选择对应的记忆层级证据。这样可以在同一接口下回答现在、近期过去以及长期习惯相关的问题。

LightMem-Ego主要支持哪些日常任务或应用场景？

系统展示了面向对象寻找、对话回忆、生活总结与日常规律发现的能力。通过在不同记忆层之间路由查询，它为个性化日常助理提供支持。

Jul 16, 2026

Upload by Theodora