大语言模型 / Large Language Models

本页结构

核心概念

注意力机制、Transformer 模块与序列建模 Attention, transformer blocks and sequence modeling
预训练、微调与指令调优 Pretraining, fine-tuning and instruction tuning
情绪抽取、因子生成与 Agent 工作流 Sentiment extraction, factor generation and agentic workflows

学习顺序

把 LLM 当成概率化文本处理器，而不是确定性真值源。 Use LLMs as probabilistic text processors, not deterministic oracles.
区分模型能力、数据管线和评估设计。 Separate model capability, data pipeline and evaluation design.
明确幻觉、延迟、隐私和市场状态漂移风险。 State risks around hallucination, latency, privacy and regime drift.

Large Language Models (LLMs) are transforming quantitative finance by providing powerful tools for processing unstructured data, generating predictive signals, and enabling autonomous decision-making. Their ability to understand context and reason over vast textual corpora makes them essential for extracting alpha from non-traditional data sources.

大型语言模型（LLM）通过提供强大的工具来处理非结构化数据、生成预测信号和实现自主决策，正在改变定量金融。他们能够理解大量文本语料库的上下文和推理，这使得他们对于从非传统数据源中提取阿尔法至关重要。

The Transformer Architecture

变压器架构

LLMs are built upon the Transformer architecture, which introduced the self-attention mechanism to efficiently process sequential data (text).

LLM 建立在 Transformer 架构之上，该架构引入了 自注意力机制 来高效处理顺序数据（文本）。

Self-Attention: Allows the model to weigh the importance of different words in the input sequence when processing a specific word. This mechanism is key to capturing long-range dependencies and context, which is crucial for understanding complex financial narratives.
Encoder-Decoder vs. Decoder-Only: Encoder-Decoder (e.g., BERT): Used for tasks like classification and sequence-to-sequence translation (e.g., summarizing a report). Decoder-Only (e.g., GPT-series): Used for generative tasks, predicting the next token in a sequence, which forms the basis of conversational AI and content generation.

自注意力：允许模型在处理特定单词时权衡输入序列中不同单词的重要性。这种机制是捕获长期依赖性和背景的关键，这对于理解复杂的金融叙述至关重要。
编码器-解码器与仅解码器：编码器-解码器（例如 BERT）：用于分类和序列到序列翻译等任务（例如，总结报告）。 仅解码器（例如，GPT 系列）：用于生成任务，预测序列中的下一个标记，这构成了对话式 AI 和内容生成的基础。

Training Paradigms

训练范式

LLMs are typically trained in a multi-stage process:

法学硕士通常经过多阶段过程的培训：

Stage	Description	Financial Relevance
Pre-training	Unsupervised training on massive, general-purpose text corpora (e.g., web data, books) to learn language structure and world knowledge.	Establishes foundational linguistic and general reasoning capabilities.
Domain-Specific Pre-training	Continued pre-training on domain-specific corpora (e.g., financial news, earnings call transcripts, SEC filings).	Creates Financial LLMs (e.g., BloombergGPT, FinGPT) that understand financial jargon, context, and entities.
Fine-tuning (Supervised)	Training on smaller, labeled datasets for specific tasks (e.g., sentiment classification, question answering).	Adapts the model for specific quant tasks like classifying news sentiment as bullish/bearish.
Reinforcement Learning from Human Feedback (RLHF)	Training to align the model's output with human preferences and instructions (e.g., making the model's financial advice safer or more relevant).	Crucial for building reliable Quant Agents that follow complex instructions and avoid generating misleading information.

阶段	描述	金融相关性
预训练	对大量通用文本语料库（例如网络数据、书籍）进行无监督训练，以学习语言结构和世界知识。	建立基础语言和一般推理能力。
特定领域的预训练	继续对特定领域的语料库（例如财经新闻、财报电话会议记录、SEC 文件）进行预培训。	创建了解金融术语、背景和实体的金融法学硕士（例如 BloombergGPT、FinGPT）。
微调（监督）	针对特定任务（例如情感分类、问答）对较小的标记数据集进行训练。	使模型适应特定的量化任务，例如将新闻情绪分类为看涨/看跌。
基于人类反馈的强化学习 (RLHF)	进行培训，使模型的输出与人类偏好和指令保持一致（例如，使模型的金融建议更安全或更相关）。	对于构建遵循复杂指令并避免生成误导性信息的可靠的定量代理至关重要。

The primary role of LLMs in alpha generation is to transform qualitative, unstructured data into quantitative, predictive signals.

法学硕士在阿尔法生成中的主要作用是将定性、非结构化数据转化为定量、预测信号。

1. Sentiment Extraction

1. 情感提取

LLMs excel at extracting nuanced sentiment from text, moving beyond simple keyword counting.

法学硕士擅长从文本中提取细致入微的情感，而不仅仅是简单的关键词计数。

Embedding-Based Classifiers: Using pre-trained LLMs (like FinBERT) to generate dense vector representations (embeddings) of financial text, which are then fed into traditional classifiers.
Prompt-Based Classification: Directly prompting a generative LLM (like GPT-4) to classify the sentiment of a news headline or earnings report, leveraging its advanced reasoning capabilities. This has shown predictive power even after accounting for traditional factors.

基于嵌入的分类器：使用预先训练的 LLM（如 FinBERT）生成金融文本的密集向量表示（嵌入），然后将其输入到传统分类器中。
基于提示的分类：直接提示生成式 LLM（如 GPT-4）利用其先进的推理功能对新闻标题或收益报告的情绪进行分类。即使在考虑了传统因素之后，这也显示出了预测能力。

2. Factor Generation

2. 因子生成

LLMs can act as a "factor agent" to generate novel alpha factors.

法学硕士可以充当“因子代理”来产生新的阿尔法因子。

Conceptual Factor Discovery: LLMs can be prompted to conceptualize new trading factors based on financial theory and market intuition, and even generate the Python code required to compute them from raw data. This automates the initial, creative phase of factor research.
Relational Representation: LLMs can extract complex relationships between companies, sectors, or events from text, which can be used to build dynamic Knowledge Graphs for more sophisticated network-based predictions.

概念因素发现：可以提示法学硕士根据金融理论和市场直觉概念化新的交易因素，甚至生成根据原始数据计算它们所需的 Python 代码。这使因素研究的初始创造性阶段自动化。
关系表示：法学硕士可以从文本中提取公司、部门或事件之间的复杂关系，可用于构建动态知识图，以进行更复杂的基于网络的预测。

The most advanced application involves integrating LLMs into multi-agent systems that can autonomously execute complex financial workflows.

最先进的应用程序涉及将法学硕士集成到可以自主执行复杂金融工作流程的多代理系统中。

Architecture: LLM-based quant agents typically combine a central LLM (for reasoning and planning) with external Tools (APIs for data retrieval, numerical computation, and order execution).
Multi-Agent Systems: These frameworks simulate a trading desk, with specialized LLM agents (e.g., a Fundamental Analyst, a Technical Analyst, a Portfolio Manager) collaborating to make decisions. This approach enhances robustness and provides a degree of Explainability through the agents' natural language reasoning chains.
Financial Decision-Making: Agents can handle the entire alpha pipeline: Data Processing: Analyze news, reports, and social media. Prediction: Generate trading signals. Portfolio Optimization: Use external solvers to determine optimal asset allocation. Execution: Interact with trading APIs to place orders.

架构：基于LLM的量化代理通常将中央LLM（用于推理和规划）与外部工具（用于数据检索、数值计算和订单执行的API）相结合。
多代理系统：这些框架模拟交易台，与专门的 LLM 代理（例如基本分析师、技术分析师、投资组合经理）协作做出决策。这种方法增强了鲁棒性，并通过代理的自然语言推理链提供一定程度的可解释性。
金融决策：代理可以处理整个 alpha 管道： 数据处理：分析新闻、报告和社交媒体。预测：生成交易信号。 投资组合优化：使用外部求解器来确定最佳资产配置。执行：与交易API交互以下订单。

Despite their power, LLMs face unique challenges in the financial domain:

尽管法学硕士实力雄厚，但他们在金融领域面临着独特的挑战：

Challenge	Description	Mitigation Strategy
Hallucination	Generating factually incorrect or nonsensical information, which is catastrophic in finance.	Retrieval-Augmented Generation (RAG): Grounding LLM responses in verified, real-time financial documents and data.
Non-Stationarity	Financial data distributions change over time (regime shifts).	Continual Pre-training and frequent fine-tuning on the most recent market data; use of time-aware architectures.
Latency	Large models can be slow, making them unsuitable for high-frequency trading.	Model Compression (quantization, pruning) and focusing on lower-latency tasks like end-of-day or low-frequency alpha generation.
Data Leakage	LLMs trained on public data may have seen sensitive financial information, leading to false confidence in predictions.	Use of Private/Domain-Specific LLMs (e.g., BloombergGPT) trained exclusively on proprietary or carefully curated financial data.

挑战	描述	缓解策略
幻觉	生成事实上不正确或无意义的信息，这在金融领域是灾难性的。	检索增强生成（RAG）：将法学硕士的回答基于经过验证的实时金融文档和数据。
非平稳性	金融数据分布随着时间的推移而变化（制度转变）。	持续的预训练并根据最新的市场数据进行频繁的微调；使用时间感知架构。
延迟	大型模型可能会很慢，因此不适合高频交易。	模型压缩（量化、修剪）并专注于低延迟任务，例如日终或低频 alpha 生成。
数据泄露	受过公共数据培训的法学硕士可能看到了敏感的金融信息，导致对预测产生错误的信心。	使用私人/特定领域的法学硕士（例如 BloombergGPT）专门针对专有或精心策划的金融数据进行培训。

补充讲解

把 LLM 当成概率组件

Treat LLMs as probabilistic components

LLM 适合文本抽取、摘要和流程自动化，但输出需要结构校验、检索 grounding 和评估体系约束。

LLMs are useful for text extraction, summarization, and workflow automation, but their outputs need schema checks, retrieval grounding, and evaluation.

区分指令质量和系统质量

Separate instructions from system quality

清晰指令不能替代生产系统中的来源选择、缓存、审计链路、权限边界和失败兜底。

Good instructions cannot replace source selection, caching, audit trails, permission boundaries, and fallback behavior in production systems.

量化应用要控制时效性

Quant use cases need freshness control

当 LLM 处理新闻或公告时，时间戳、去重和延迟数据处理与模型本身同样重要。

When LLMs process news or filings, timestamping, deduplication, and delayed-data handling are as important as the language model itself.

大语言模型