Course Syllabus

Course Information：

Year-Semester Course Number Course Name Phase Credits Hours Required/Elective Instructor Class Student NO. Withdraw Note

Spring Semester
Year 2026 360023 Foundations of Reinforcement Learning with Applications in Finance 1 3.0 3 ★ Ting-Yu Chang GIFM 7 1 新聘專任教師

Year-Semester	Course Number	Course Name	Phase	Credits	Hours	Required/Elective	Instructor	Class	Student NO.	Withdraw	Note
Spring Semester Year 2026	360023	Foundations of Reinforcement Learning with Applications in Finance	1	3.0	3	★	Ting-Yu Chang	GIFM	7	1	新聘專任教師

Course Plan：

Instructor Ting-Yu Chang　Office Hours

Email kay.c@ntut.edu.tw

Last Updated 2026-02-07 02:03:28

Course Objective 本課程旨在介紹強化學習（Reinforcement Learning, RL）的核心概念、數學基礎與主要演算法，並結合財務與投資管理的應用實例。內容涵蓋馬可夫決策過程（MDP）、動態規劃（DP）、蒙地卡羅（MC）方法、時序差分（TD）學習、策略梯度（Policy Gradient）、深度強化學習等主題，同時透過金融市場的真實案例，引導學生理解 RL 在投資組合最佳化、資產配置、避險策略、交易策略自動化等領域中的應用方式。課程採用理論與實作並重的方式，搭配 Python 程式實作與金融資料分析，協助學生逐步累積建構 RL 模型的能力。學生將學習如何定義財務問題中的狀態空間、動作空間與報酬函數，並訓練智能代理人於市場模擬環境中進行策略學習、決策與改善。本課程亦將討論強化學習在金融應用上的挑戰，例如市場隨機性、風險控管、環境非平穩性、評估困難等問題，並介紹最新的研究方向與業界實務。修讀本課程後，學生能具備基礎的強化學習能力，了解 RL 模型在 FinTech、量化投資與數位財務決策上的角色，並能在課堂專題中打造具體的 RL 金融應用原型。 This course provides a comprehensive introduction to Reinforcement Learning (RL) and its applications in finance. Topics include Markov Decision Processes (MDP), Dynamic Programming, Monte Carlo methods, Temporal-Difference learning, Policy Gradient approaches, and Deep Reinforcement Learning. Through lectures and coding exercises, students will learn how to formulate financial problems within the RL framework—defining states, actions, and reward functions—and how to train agents to make sequential investment decisions. The course integrates theory with practical applications such as portfolio optimization, asset allocation, hedging strategies, and algorithmic trading. Students will gain hands-on experience implementing RL algorithms in Python and applying them to real-world financial datasets. Challenges unique to financial markets, including stochasticity, risk constraints, non-stationarity, and evaluation difficulties, will also be discussed.

Course Schedule Course Schedule and Topics Part I: Foundations and Programming Paradigms (Weeks 1–2) Week 1: Course Introduction and Overview of Reinforcement Learning (Ch. 1) Week 2: Programming Design and Abstraction Patterns (Ch. 2) Part II: Markov Models and Planning Algorithms (Weeks 3–7) Week 3: Markov Processes and Stock Price Modeling (Ch. 3.1–3.4) Week 4: Markov Reward Processes, Inventory Models, and Stationary Distributions (Ch. 3.5–3.13) Week 5: Core Concepts of Markov Decision Processes (MDP) (Ch. 4) Week 6: Dynamic Programming Algorithms (Ch. 5) Week 7: Function Approximation and Approximate Dynamic Programming (Ch. 6) Part III: Midterm Progress Evaluation (Week 8) Week 8: Midterm Project Presentation Presentation of the proposed final project, including motivation, problem formulation, and preliminary design Part IV: Prediction and Control in Reinforcement Learning (Weeks 9–13) Week 9: Monte Carlo (MC) and Temporal-Difference (TD) Prediction (Ch. 11.1–11.5) Week 10: TD(λ) and Eligibility Traces (Ch. 11.6) Week 11: Reinforcement Learning Control Algorithms (Ch. 12) Week 12: Batch Learning, Experience Replay, and Deep Q-Networks (Ch. 13.1–13.4) Week 13: Least-Squares Reinforcement Learning Algorithms (Ch. 13.5–13.9) Part V: Advanced Financial Modeling and Policy Gradient Methods (Weeks 14–17) Week 14: Dynamic Asset Allocation Modeling (Ch. 8) Week 15: Policy Gradient Algorithms (Ch. 14.1–14.4) Week 16: Actor–Critic Architectures and Advanced Policy Gradient Methods (Ch. 14.6–14.10) Week 17: Real-World Considerations and Course Summary (Ch. 17) Part VI: Final Presentation (Week 18) Week 18: Final Project Presentation -Final project demonstration: students are required to implement a complete financial reinforcement learning agent (e.g., dynamic hedging, algorithmic trading, or asset allocation systems) and analyze its performance

Evaluation and grading policy Class Participation and Performance (including attendance) 30% Midterm Report (Introduction and proposal of the selected Final Project) 30% Final Project (implementation, analysis, and presentation) 40% Notes: -Class participation includes attendance, in-class discussion, and engagement in hands-on exercises. -The midterm report focuses on the motivation, problem formulation, and preliminary design of the selected final project. -The final project requires students to apply reinforcement learning methods to a finance- or investment-related decision-making problem, including model design, experimentation, and result analysis.

Materials All right reserved. Do not reproduce without permission.

Foreign-language textbooks:Yes
Foundations of Reinforcement Learning with Applications in Finance (Chapman & Hall/CRC Mathematics and Artificial Intelligence Series) PDF version of textbook available as free download https://stanford.edu/~ashlearn/RLForFinanceBook/book.pdf

The access to curricular consultation Office Hours: As announced by the instructor
Email Consultation: Students may contact the instructor via email for questions regarding course materials, assignments, or projects

Expanding teaching and resources

The course corresponds to the SDGs SDG9：產業創新與基礎設施（Industry, Innovation and Infrastructure）

Does the course incorporate AI ● 使用生成式AI作為教學過程的輔助工具（Use generative AI tools as an aid to the teaching process）
● 簡單介紹AI的基本概念（Briefly introduce the basic concept of AI）
● 深入探討AI的原理與應用（Explore the principles and applications of AI in depth）

Note
　

Instructor	Ting-Yu Chang　Office Hours
Email	kay.c@ntut.edu.tw
Last Updated	2026-02-07 02:03:28
Course Objective	本課程旨在介紹強化學習（Reinforcement Learning, RL）的核心概念、數學基礎與主要演算法，並結合財務與投資管理的應用實例。內容涵蓋馬可夫決策過程（MDP）、動態規劃（DP）、蒙地卡羅（MC）方法、時序差分（TD）學習、策略梯度（Policy Gradient）、深度強化學習等主題，同時透過金融市場的真實案例，引導學生理解 RL 在投資組合最佳化、資產配置、避險策略、交易策略自動化等領域中的應用方式。課程採用理論與實作並重的方式，搭配 Python 程式實作與金融資料分析，協助學生逐步累積建構 RL 模型的能力。學生將學習如何定義財務問題中的狀態空間、動作空間與報酬函數，並訓練智能代理人於市場模擬環境中進行策略學習、決策與改善。本課程亦將討論強化學習在金融應用上的挑戰，例如市場隨機性、風險控管、環境非平穩性、評估困難等問題，並介紹最新的研究方向與業界實務。修讀本課程後，學生能具備基礎的強化學習能力，了解 RL 模型在 FinTech、量化投資與數位財務決策上的角色，並能在課堂專題中打造具體的 RL 金融應用原型。 This course provides a comprehensive introduction to Reinforcement Learning (RL) and its applications in finance. Topics include Markov Decision Processes (MDP), Dynamic Programming, Monte Carlo methods, Temporal-Difference learning, Policy Gradient approaches, and Deep Reinforcement Learning. Through lectures and coding exercises, students will learn how to formulate financial problems within the RL framework—defining states, actions, and reward functions—and how to train agents to make sequential investment decisions. The course integrates theory with practical applications such as portfolio optimization, asset allocation, hedging strategies, and algorithmic trading. Students will gain hands-on experience implementing RL algorithms in Python and applying them to real-world financial datasets. Challenges unique to financial markets, including stochasticity, risk constraints, non-stationarity, and evaluation difficulties, will also be discussed.
Course Schedule	Course Schedule and Topics Part I: Foundations and Programming Paradigms (Weeks 1–2) Week 1: Course Introduction and Overview of Reinforcement Learning (Ch. 1) Week 2: Programming Design and Abstraction Patterns (Ch. 2) Part II: Markov Models and Planning Algorithms (Weeks 3–7) Week 3: Markov Processes and Stock Price Modeling (Ch. 3.1–3.4) Week 4: Markov Reward Processes, Inventory Models, and Stationary Distributions (Ch. 3.5–3.13) Week 5: Core Concepts of Markov Decision Processes (MDP) (Ch. 4) Week 6: Dynamic Programming Algorithms (Ch. 5) Week 7: Function Approximation and Approximate Dynamic Programming (Ch. 6) Part III: Midterm Progress Evaluation (Week 8) Week 8: Midterm Project Presentation Presentation of the proposed final project, including motivation, problem formulation, and preliminary design Part IV: Prediction and Control in Reinforcement Learning (Weeks 9–13) Week 9: Monte Carlo (MC) and Temporal-Difference (TD) Prediction (Ch. 11.1–11.5) Week 10: TD(λ) and Eligibility Traces (Ch. 11.6) Week 11: Reinforcement Learning Control Algorithms (Ch. 12) Week 12: Batch Learning, Experience Replay, and Deep Q-Networks (Ch. 13.1–13.4) Week 13: Least-Squares Reinforcement Learning Algorithms (Ch. 13.5–13.9) Part V: Advanced Financial Modeling and Policy Gradient Methods (Weeks 14–17) Week 14: Dynamic Asset Allocation Modeling (Ch. 8) Week 15: Policy Gradient Algorithms (Ch. 14.1–14.4) Week 16: Actor–Critic Architectures and Advanced Policy Gradient Methods (Ch. 14.6–14.10) Week 17: Real-World Considerations and Course Summary (Ch. 17) Part VI: Final Presentation (Week 18) Week 18: Final Project Presentation -Final project demonstration: students are required to implement a complete financial reinforcement learning agent (e.g., dynamic hedging, algorithmic trading, or asset allocation systems) and analyze its performance
Evaluation and grading policy	Class Participation and Performance (including attendance) 30% Midterm Report (Introduction and proposal of the selected Final Project) 30% Final Project (implementation, analysis, and presentation) 40% Notes: -Class participation includes attendance, in-class discussion, and engagement in hands-on exercises. -The midterm report focuses on the motivation, problem formulation, and preliminary design of the selected final project. -The final project requires students to apply reinforcement learning methods to a finance- or investment-related decision-making problem, including model design, experimentation, and result analysis.
Materials	All right reserved. Do not reproduce without permission. Foreign-language textbooks:Yes Foundations of Reinforcement Learning with Applications in Finance (Chapman & Hall/CRC Mathematics and Artificial Intelligence Series) PDF version of textbook available as free download https://stanford.edu/~ashlearn/RLForFinanceBook/book.pdf
The access to curricular consultation	Office Hours: As announced by the instructor Email Consultation: Students may contact the instructor via email for questions regarding course materials, assignments, or projects
Expanding teaching and resources
The course corresponds to the SDGs	SDG9：產業創新與基礎設施（Industry, Innovation and Infrastructure）
Does the course incorporate AI	● 使用生成式AI作為教學過程的輔助工具（Use generative AI tools as an aid to the teaching process） ● 簡單介紹AI的基本概念（Briefly introduce the basic concept of AI） ● 深入探討AI的原理與應用（Explore the principles and applications of AI in depth）
Note