Skip to main navigation Skip to main content
  • E-Submission

JKSPE : Journal of the Korean Society for Precision Engineering

OPEN ACCESS
ABOUT
BROWSE ARTICLES
EDITORIAL POLICIES
FOR CONTRIBUTORS
REGULAR

강화학습을 이용한 능동 현가장치 제어

Active Suspension Control Using Reinforcement Learning

Journal of the Korean Society for Precision Engineering 2024;41(3):223-230.
Published online: March 1, 2024

1 국립금오공과대학교 대학원 항공기계전자융합전공

2 국립금오공과대학교 기계시스템공학부

1 Department of Aeronautics, Mechanical and Electronic Convergence Engineering, Graduate School, Kumoh National Institute of Technology

2 School of Mechanical System Engineering, Kumoh National Institute of Technology

#E-mail: jwsohn@kumoh.ac.kr, TEL: +82-54-478-7378
• Received: November 20, 2023   • Revised: January 18, 2024   • Accepted: January 28, 2024

Copyright © The Korean Society for Precision Engineering

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

  • 7 Views
  • 0 Download
  • 2 Crossref
  • 1 Scopus
prev next

Citations

Citations to this article as recorded by  Crossref logo
  • Control Characteristics of Active Suspension in Vehicles using Adaptive Control Algorithm
    Jeong Seo Jang, Jung Woo Sohn
    Transactions of the Korean Society for Noise and Vibration Engineering.2024; 34(5): 568.     CrossRef
  • Suspension Mechanism Design of a Low-platform Target Robot for Evaluating Autonomous Vehicle Active Safety
    Jae Sang Yoo, Do Hyeon Kim, Jayil Jeong
    Journal of the Korean Society for Precision Engineering.2024; 41(5): 375.     CrossRef

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

Include:

Active Suspension Control Using Reinforcement Learning
J. Korean Soc. Precis. Eng.. 2024;41(3):223-230.   Published online March 1, 2024
Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:
Include:
Active Suspension Control Using Reinforcement Learning
J. Korean Soc. Precis. Eng.. 2024;41(3):223-230.   Published online March 1, 2024
Close

Figure

  • 0
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
Active Suspension Control Using Reinforcement Learning
Image Image Image Image Image Image Image Image Image Image
Fig. 1 Basic structure of reinforcement learning
Fig. 2 DDPG architecture
Fig. 3 Critic and actor network
Fig. 4 Quarter vehicle model
Fig. 5 Road profile model
Fig. 6 Simulation model
Fig. 7 Results of random road
Fig. 8 Results of bump road
Fig. 9 Results of sinewave road
Fig. 10 Average rewards per episode
Active Suspension Control Using Reinforcement Learning

Hyperparameters of DDPG

Hyperparameters Value
Critic Learn rate 1e-3
Gradient threshold 1
Actor Learn rate 1e-4
Gradient threshold 1
Agent Sample time 0.05
Experience buffer length 1e6
Discount factor 0.99
Minibatch size 64
Noise variance 0.6
Decay rate of noise variance 1e-5
Training process Max episodes 5,000
Max steps 100

Quarter vehicle model parameters

Parameters Symbol (Unit) Value
Sprung mass mb [kg] 365
Unsprung mass mw [kg] 43
Spring stiffness ks [N/m] 24,000
Damper coefficient bs [N·s/m] 2,126
Tire stiffness kt [N/m] 243,000

The value of the random road parameters

Symbol (Unit) Value Symbol (Unit) Value
v [m/s] 20 σ2 [mm2] 300
n0[m–1] 0.1 ρ [m-1] 0.45
G(n0) [m3] 256×10–6

RMS reduction percentage of sprung mass under random road excitation

RMS Reduction percentage [%]
Controlled (PID) Controlled (DDPG-Bump) Controlled (DDPG-Rand)
Displacement 19.1781 -16.4384 50.6849
Acceleration 27.4627 16.3753 93.1485

RMS reduction percentage of sprung mass displacement under bump road excitation

RMS Reduction percentage [%]
Velocity [km/h] Controlled (PID) Controlled (DDPG-Bump) Controlled (DDPG-Rand)
15 10.1046 34.8083 34.1317
30 19.1816 23.4568 44.1296
60 19.7987 -48.9933 40.2685

RMS reduction percentage of sprung mass acceleration under bump road excitation

RMS Reduction percentage [%]
Velocity [km/h] Controlled (PID) Controlled (DDPG-Bump) Controlled (DDPG-Rand)
15 20.7057 64.5131 83.6572
30 31.5066 27.4802 85.1189
60 28.5436 9.6335 78.9815

RMS reduction percentage of sprung mass under sinewave road excitation

RMS Reduction percentage [%]
Controlled (PID) Controlled (DDPG-Bump) Controlled (DDPG-Rand)
Displacement 17.1518 5.4054 69.2308
Acceleration 5.2941 2.3529 86.0131
Table 1 Hyperparameters of DDPG
Table 2 Quarter vehicle model parameters
Table 3 The value of the random road parameters
Table 4 RMS reduction percentage of sprung mass under random road excitation
Table 5 RMS reduction percentage of sprung mass displacement under bump road excitation
Table 6 RMS reduction percentage of sprung mass acceleration under bump road excitation
Table 7 RMS reduction percentage of sprung mass under sinewave road excitation