2024-10-13-国际清算银行-CB-LM_中央银行的语言模型（英）-33份_33页_804kb

CB-LMs: Language Models for Central Banking

Authors: Leonardo Gambacorta, Byeungchun Kwon, Taejin Park, Pietro Patelli, Sonya Zhu; Bank for International Settlements (BIS), *CEPR
Publication: BIS Working Paper No 1215, October 2024
JEL Classification: E58, C55, C63, G17
Keywords: Large Language Models (LLMs), gen AI, central banks, monetary policy analysis

Introduction

Central banks are increasingly using language models to analyze monetary policy communication. CB-LMs are specialized encoder-only language models retrained on a comprehensive corpus of central bank speeches, policy documents, and research papers. This research introduces CB-LMs to improve domain-specific NLP analysis in monetary economics and central banking, addressing limitations of general-purpose models like BERT and GPT.

Methodology

CB-LMs are developed through domain adaptation and fine-tuning using large-scale central banking corpora, including 37,037 research papers and 18,345 speeches. Foundational models (BERT and RoBERTa) were adapted to this domain-specific data. Performance is evaluated using masked word prediction and monetary policy sentiment classification tasks.

Key Findings

• CB-LMs outperform foundational models and generative LLMs in specific central banking tasks like masked word prediction and monetary policy stance classification in FOMC statements.
• RoBERTa-based CB-LMs show superior performance in domain-specific analyses, with accuracy exceeding 80% in sentiment classification.
• Generative LLMs (e.g., ChatGPT-4, Llama-3 70B) underperform smaller encoder-only models in sentence-level tasks but excel in more complex scenarios like longer texts with limited data.
• CB-LMs provide significant improvements for NLP analysis in monetary policy, potentially enhancing accuracy and insights for central banks.

Considerations for Central Banks

The deployment of generative LLMs poses challenges regarding confidentiality, privacy, replicability, and cost-efficiency. Central banks must weigh these against the benefits of advanced models. CB-LMs offer a cost-efficient alternative, while proprietary generative models require robust infrastructure and address transparency concerns.

Conclusion

CB-LMs represent a significant advancement in domain-specific language models, enabling more accurate and nuanced analysis of central bank communication. However, central banks need strategic approaches to model selection, considering factors like data availability, technical infrastructure, and operational risks to leverage AI effectively in monetary policy contexts.