Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer

Jul 16, 2026

Upload by Levi

Sentiment analysis built on frozen pretrained language model backbones is widespread, yet the real value of explicit domain adaptation is uncertain, especially when backbones already encode differing amounts of target-domain knowledge. A controlled study evaluates frozen embedding backbones (Qwen3-Embedding 0.6B/4B/8B), with RoBERTa-base and FinBERT, using lightweight MLP adapters trained via DANN, MMD, and supervised contrastive learning. Transfer is tested from consumer reviews to SST-2 and Financial PhraseBank. Results show negligible gains on SST-2, but substantial recovery on the restricted financial subset; adversarial alignment can degrade specialized backbones.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages9

Reading Time23 min

Introduction

Problem motivation: contamination vs generalization

Research question and controlled setup

Experimental findings and transfer regimes

Contributions

Is Domain Adaptation Always Helpful? A Frozen-Backbone Study of Cross-Domain Sentiment Transfer Phat Tran Artin Lahni Pranav Kulkarni Yaolun Zhang Oregon State University {tranphat, lahnia, kulkarnp, [zhanyaol}@oregonstate.edu](zhanyaol}@oregonstate.edu) arXiv :2607 .05937v 1 [ cs .CL] 7 Jul 2026 Abstract—Sentiment analysis with frozen pretrained language model (PLM) backbones has become a common paradigm, yet the practical benefit of explicit domain adaptation remains unclear, particularly when backbones encode varying degrees of target-domain knowledge. We present a preliminary case study evaluating a controlled family of frozen embedding backbones (Qwen3-Embedding 0.6B, 4B, 8B), alongside RoBERTa-base and FinBERT. We train a lightweight MLP adapter on consumer reviews using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL), and evaluate transfer to movie reviews (SST-2) and a heavily restricted subset of financial news (Financial PhraseBank) . Within this constrained sample, we observe two distinct transfer patterns. On SST-2, domain adaptation provides negligible gain regardless of scale. On the financial subset, explicit domain adaptation appears to recover substantial performance for small general-purpose backbones. Notably, we find that adversarial alignment (DANN) is associated with degraded performance for domain-specialized backbones like FinBERT, consistent with erosion of pre-existing domain-specific structure, whereas supervised contrastive loss appears to preserve it. These preliminary findings suggest that the efficacy of explicit domain adaptation is highly contingent on whether the frozen backbone already possesses target-domain coverage. Index Terms—Sentiment analysis, domain adaptation, pretrained language models, frozen backbones I. Introduction Sentiment analysis, the task of classifying the subjective polarity of text, is a cornerstone application of natural language processing (NLP), powering everything from product recommendation systems to financial market monitoring. Modern approaches increasingly rely on large pre-trained language mod- els (PLMs) used as frozen feature extractors, with lightweight task heads trained on top of their embeddings [1], [2] . As these backbone models have scaled from hundreds of millions to billions of parameters, their reported performance on standard benchmarks has grown correspondingly impressive [3] . However, this progress raises a critical measurement problem: at scale, PLMs may be exposed to benchmark evaluation examples during pretraining, making it difficult to distinguish genuine generalization from memorization. When a frozen backbone has already seen SST-2 [4] or Financial PhraseBank [5] during pretraining, high zero-shot accuracy reflects data contamination rather than genuine generalization. This makes it fundamentally unclear whether explicit domain adaptation techniques, methods that align a model’s representations across source and target domains, are truly helpful, or whether they merely compensate for contamination artifacts introduced by smaller, less-contaminated backbones. We address this question directly: “When does domain adaptation help for sentiment transfer?” To reduce reliance on potentially contaminated benchmark memorization, we use the Qwen3-Embedding series (0.6B, 4B, and 8B parameters) [6], an embedding family that provides a controlled scale progression for studying frozen-backbone transfer. This yields a setting in which the effects of backbone scale and domain specialization can be compared more directly. On top of these frozen embeddings, we train a lightweight MLP adapter using three complementary domain adaptation objectives: DomainAdversarial Neural Networks (DANN) [7], Maximum Mean Discrepancy (MMD) [8], and Supervised Contrastive Learning (SCL) [9] . We train on consumer product and restaurant reviews (Yelp Reviews [10] and Amazon Polarity [

FAQs

How do the authors test whether domain adaptation helps for cross-domain sentiment transfer?

They train lightweight MLP adapters on consumer review sources using DANN, MMD, and supervised contrastive learning, then evaluate zero-shot transfer to SST-2 and Financial PhraseBank without using target labels during training.

What transfer behavior is observed on SST-2 compared with Financial PhraseBank?

On SST-2, domain adaptation yields negligible additional benefit across backbone scales. On Financial PhraseBank, explicit domain adaptation substantially improves performance, especially for smaller general-purpose backbones.

Why can adversarial alignment (DANN) hurt domain-specialized backbones like FinBERT?

The study finds that adversarial alignment erodes pre-existing domain-specific structure encoded by specialized backbones, degrading their performance, while supervised contrastive loss helps preserve it.

Jul 16, 2026

Upload by Levi