LeOCLR - Leveraging Original Images for Contrastive Learning of Visual Representations - A framework for improved instance discrimination

Aug 4, 2026

Upload by Violet

Contrastive instance discrimination methods outperform supervised learning on downstream computer vision tasks, yet their performance depends strongly on data augmentation choices during representation learning. Random cropping and resizing can produce paired views with mismatched semantic content, degrading learned representations. LeOCLR (Leveraging Original Images for Contrastive Learning of Visual Representations) introduces a new instance-discrimination strategy and an adapted loss to avoid semantic-feature loss. Experiments show consistent gains across datasets, including improvements over MoCo-v2 on ImageNet-1K.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages16

Reading Time40 min

AccessFree to download

Abstract

1 Introduction

LeOCLR: Leveraging Original Images for Contrastive Learning of Visual Representations [Mohammad Alkhalefi](Mohammad Alkhalefi m.alkhalefi1.21@abdn. ac.uk)[ m.alkhalefi1.21@abdn. ac.uk](Mohammad Alkhalefi m.alkhalefi1.21@abdn. ac.uk) Department of Computing Science University of Aberdeen Georgios Leontidis [georgios.leontidis@abdn. ac.uk](georgios.leontidis@abdn. ac.uk) [Department of Computing Science & Interdisciplinary Institute](Department of Computing Science & Interdisciplinary Institute)[ ](Department of Computing Science & Interdisciplinary Institute)University of Aberdeen Mingjun Zhong [mingjun.zhong@abdn. ac.uk](mingjun.zhong@abdn. ac.uk) Department of Computing Science University of Aberdeen Reviewed on OpenReview: [https: // openreview. net/ forum? id= y8qGOvUn1r](https: // openreview. net/ forum? id= y8qGOvUn1r) Abstract Contrastive instance discrimination methods outperform supervised learning in downstream tasks such as image classification and object detection. However, these methods rely heavily on data augmentation during representation learning, which can lead to suboptimal results if not implemented carefully. A common augmentation technique in contrastive learning is random cropping followed by resizing. This can degrade the quality of representation learning when the two random crops contain distinct semantic content. To tackle this issue, we introduce LeOCLR (Leveraging Original Images for Contrastive Learning of Visual Representations), a framework that employs a novel instance discrimination approach and an adapted loss function. This method prevents the loss of important semantic features caused by mapping different object parts during representation learning. Our experiments demonstrate that LeOCLR consistently improves representation learning across various datasets, outperforming baseline models. For instance, LeOCLR surpasses MoCo-v2 by 5 . 1% on ImageNet-1K in linear evaluation and outperforms several other methods on transfer learning and object detection tasks. 1 Introduction Self-supervised learning (SSL) approaches based on instance discrimination (Chen et al. , 2020b; Chen & He, 2021; Chen et al., 2020a; Misra & Maaten, 2020; Grill et al., 2020) heavily rely on data augmentations, such as random cropping, rotation, and colour Jitter, to build invariant representation for all the instances in the dataset. To do so, the two augmented views (positive pairs) for the same instance are attracted in the latent space while avoiding collapse to the trivial solution (representation collapse) . These approaches have proven efficient in learning useful representations by using different downstream tasks, e.g., image classification and object detection, as proxy evaluations for representation learning. However, these strategies ignore the important fact that the augmented views may have different semantic content due to random cropping, which can lead to a degradation in visual representation learning (Song et al., 2023a; Zhang et al., 2022; Liu et al., 2020; Mishra et al., 2021) . On the one hand, creating positive pairs through random cropping and encouraging the model to make them similar based on the shared information in the two views makes the SSL task harder, ultimately improving representation quality (Mishra et al., 2021; Chen et al., 2020a) . In addition, random cropping followed by resizing leads model representation to capture information for the object from varying aspect ratios and induce occlusion invariance (Purushwalkam & Gupta, 2020) . On the other hand, minimizing the feature distance in the latent space (i.e., maximizing similarity) between views containing distinct semantic concepts tends to result in the loss of valuable image information (Purushwalkam & Gupta, 2020; Zhang et al., 2022; Song et al., 2023a) . Figure 1: Examples of positive pairs that might be created by random cropping and resizing. Figure 1 (a and b) show examples of incorrect semantic pos

FAQs

Why can random cropping harm contrastive instance discrimination learning?

Random cropping followed by resizing may generate positive pairs whose semantic content differs, so aligning them can discard important image features and degrade representation quality.

What is LeOCLR’s key idea?

LeOCLR leverages original images in an instance-discrimination framework with an adapted loss function to prevent losing important semantic features when paired views focus on different object parts.

How does LeOCLR perform compared with baseline methods?

Experiments show LeOCLR consistently improves representation learning across multiple datasets, including outperforming MoCo-v2 by 5.1% on ImageNet-1K in linear evaluation, and delivering strong transfer learning and object detection results.

Aug 4, 2026

Upload by Violet