CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street View Semantics and Temporal Context Training

Jul 16, 2026

Accurately estimating urban carbon emissions is essential for sustainable urban planning, but current satellite-based approaches are hard to apply across cities due to data heterogeneity and limited fine-grained semantic-temporal context. CarbonCLIP introduces a task-oriented multimodal distillation framework that transfers contextual knowledge into a unified satellite representation using dual-branch contrastive learning. Spatial priors come from street-view text descriptions generated by large multimodal models, and temporal priors are encoded by a month encoder. Pretraining uses multimodal data, while inference uses only satellite imagery for scalable deployment; experiments on Beijing and Singapore validate improved prediction performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

Introduction

Challenges in Satellite-Based Carbon Estimation

Limits of Static Visual Features

Motivation for Human-Centric and Temporal Context

Framework Overview: CarbonCLIP

arXiv :2607 .07292v 1 [ cs .CV] 8 Jul 2026 CARBONCLIP: ENHANCE CARBON PREDICTION FROM SATELLITE IMAGERY VIA INTEGRATED STREET-VIEW SEMANTICS AND TEMPORAL CONTEXT TRAINING Zeru Yang 1 ,2 , Fang-Ying Gong3 , Steve H.L. Yim4 ,5 , Chau Yuen2 ,5 1Energy Research Institute at NTU, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore 2 School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 3 School of Public Administration and Policy, Renmin University of China, Beijing, China 4Asian School of the Environment, Nanyang Technological University, Singapore 5 Center for Climate Change and Environmental Health, Nanyang Technological University, Singapore [zeru001@e.ntu.edu.sg](zeru001@e.ntu.edu.sg) , [fangying.gong@ruc.edu.cn](fangying.gong@ruc.edu.cn) , [steve.yim@ntu.edu.sg](steve.yim@ntu.edu.sg) , [chau.yuen@ntu.edu.sg](chau.yuen@ntu.edu.sg) ABSTRACT Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source hetero geneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based car bon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling. 1 Introduction With the rapid urbanization occurring worldwide, cities have become the dominant sources of anthropogenic carbon emissions, accounting for more than one-third of global totals [1, 2] . Accurately quantifying and estimating urban carbon emissions is essential for developing sustainable and climate-resilient cities [3] . Urban sustainability requires not only reducing carbon footprints but also integrating intelligent monitoring tools that can inform policy-making and promote equitable environmental outcomes. A core challenge in achieving this goal lies in how we perceive and analyze cities at scale. Carbon emissions in urban areas are deeply intertwined with spatial patterns such as land use and infrastructure, as well as human-scale factors like greenery, density, and activity levels. In recent years, visual data have become a cornerstone of urban analytics and environmental monitoring, providing unprecedented means of observing cities at multiple scales. Satellite imagery offers a comprehensive overview of urban environments, capturing spatial structure and physical layout [4] . As a scalable and consistent data source, it has been widely applied in air pollution mapping [5], agricultural monitoring [6], and carbon stock estimation [7] . Advancesin remote sensing, including high-resolution Planet imagery at 3 m resolution [8], enable detailed detection of urban expansion and environmental change [9

FAQs

What problem does CarbonCLIP address in urban carbon emission prediction?

It tackles the difficulty of applying existing satellite-based methods across cities caused by heterogeneous data sources and the lack of fine-grained semantic-temporal context in remote sensing inputs.

How does CarbonCLIP integrate street-view information into satellite representation learning?

It uses a spatial branch where street-view images are converted into fine-grained textual descriptions by large multimodal models, providing semantic priors tied to building functions, infrastructure, and activities.

When is street-view or other multimodal data required for CarbonCLIP?

Multimodal data are used only during pretraining; during inference, the method relies solely on satellite imagery, enabling scalable deployment when ground-level data are unavailable.

Jul 16, 2026

Upload by Seraphina