Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation

TL;DR

To solve the bias and instability of model selection in unsupervised domain adaptation, we propose a new validation method DEV that leverages adapted features.

Problem

In deep unsupervised domain adaptation (Deep UDA), model selection is difficult due to the lack of labels in the target domain. Existing methods are biased, restricted, or even require target labels, hindering algorithm comparison and progress in the field.

Approach

The proposed DEV (Deep Embedded Validation) incorporates adapted feature representations into the validation procedure to obtain an unbiased estimator of the target risk. The control variate technique is used to further reduce variance. Theoretical unbiasedness and variance reduction are proven, and the method is designed to be applicable to various UDA methods.

Results & Contribution

Experiments on multiple UDA benchmarks demonstrate that DEV predicts target performance more accurately and stably than existing model selection methods. This method enables reliable model selection even without labeled target data, contributing as a standard evaluation protocol for UDA research.