A transformer-based explainable machine learning framework for heterogeneous ESG risk and return prediction using text intelligence

Abstract

Corporate ESG disclosure has shifted toward narrative form, yet vendor ESG ratings that most empirical work depends on show substantial disagreement across providers, and encoders dominant in financial text analysis inherit a 512 token ceiling that discards most of a typical 10-K. HET-ESGFormer combines hierarchical long-document encoding, heterogeneity-aware sparse expert routing conditioned on industry sector and firm size, and a dual-channel explanation layer subject to faithfulness verification. The framework is evaluated on 3,182 U.S.-listed firms over 2015 to 2024 using 10-K filings, sustainability reports, earnings call transcripts, and ESG-tagged news. Against nine baselines spanning econometric, dictionary-based, contextual encoder, sparse-attention, and time-series approaches, HET-ESGFormer reduces return RMSE from 0.0417 to 0.0389 and QLIKE from 0.148 to 0.128, with DieboldMariano tests rejecting equal predictive accuracy against every baseline after correction for multiple comparisons. The text channel's marginal contribution to downside risk prediction is materially larger than its contribution to return prediction, widening at longer forecast horizons, indicating that ESG narratives carry more information about risk than about expected return. Ablations reveal a decoupling between predictive skill and explanation faithfulness: removing the consistency regularizer barely affects accuracy but sharply degrades explanation quality. The dual-channel explanation improves ERASER comprehensiveness by 50% over SHAP applied to XGBoost, and human raters significantly prefer the framework's rationales over SHAP-based rationales on trustworthiness. Decile-sorted long-short portfolios deliver a post-cost Sharpe ratio of 1.31 against 0.74 and 0.62 for portfolios built on Refinitiv and MSCI ratings, and pass VaR backtests at 95% coverage. The results support textbased ESG modeling as an alternative to vendor-rating-based approaches, particularly where downside risk management and traceable explanation are jointly required.