Skip to main navigation Skip to search Skip to main content

Deep Nonparametric Quantile Regression under Covariate Shift

  • Xingdong Feng
  • , Xin He*
  • , Yuling Jiao
  • , Lican Kang*
  • , Caixing Wang
  • *Corresponding author for this work
  • Shanghai Univ Finance & Econ
  • Stat Shanghai Univ Finance & Econ Shanghai

Research output: Contribution to journal › Article › peer-review

8 Scopus citations

Abstract

This work focuses on addressing the challenges posed by covariate shift in nonparametric quantile regression using deep neural networks. We propose a two-stage pre-training reweighted method that leverages importance weighting to mitigate the effects of distribution shift. In the first stage, density ratios are estimated with a neural network by minimizing least squares. In the second stage, a deep neural network estimator is obtained using pre-training weights. Theoretical analysis is provided, offering non-asymptotic error bounds for the unweighted, reweighted, and pre-training reweighted estimators. We consider scenarios with both bounded and unbounded density ratios. Notably, we employ a novel proof technique to bound the generalization error, characterized by the size and weights bound of ReLU neural networks. This enables us to establish fast rates of convergence under the adaptive self-calibration condition, distinguishing our approach from those relying on local Rademacher complexity techniques. Additionally, we derive the approximation error with weight bounds for ReLU neural networks approximating the Holder class. Our theoretical findings provide valuable insights for the pre-training process and highlight the efficacy of reweighted techniques. Numerical experiments are conducted to further validate the theoretical findings and demonstrate the effectiveness of our proposed method.
Original languageEnglish
Article number385
Pages (from-to)1-50
Number of pages50
JournalJournal of Machine Learning Research
Volume25
Issue number1
StatePublished - Dec 2024

Funding

We would like to express our sincere gratitude to the editor, the action editor, and two anonymous reviewers for their constructive comments, which have greatly contributed to the significant improvement of the manuscript. This research was partly supported by the National Natural Science Foundation of China grant (12371441, 12371270, 12001356, U24A2002) , Shanghai Science and Technology Development Funds (23JC1402100) , Natural Science Foundation of Shanghai (24ZR1421400) , Shanghai Research Center for Data Science and Decision Technology, and the Fundamental Research Funds for the Central Universities.

Keywords

  • Deep neural networks
  • distribution shift
  • non-asymptotic error bounds
  • robust estimation

Indexed by

  • SCIE
  • EI

Fingerprint

Dive into the research topics of 'Deep Nonparametric Quantile Regression under Covariate Shift'. Together they form a unique fingerprint.

Cite this