Deep reinforcement learning (RL) has attracted research interest in the manufacturing area in recent years, but real implemented applications are rarely found. This is because agents have to explore the given environments many times until they learn how to maximize the rewards for actions, which they provide to the environments. While training, random actions or exploration from agents may be disastrous in many real-world applications, and thus, people usually use computer generated simulation environments to train agents. In this paper, we present a RL experiment applied to temperature control of a chamber for ultra-precision machines. The RL agent was built in Python and PyTorch framework using a Deep Q-Network (DQN) algorithm and its action commands were sent to National Instruments (NI) hardware, which ran C codes with a sampling rate of 1 Hz. For communication between the agent and the NI data acquisition unit, a data pipeline was constructed from the subprocess module and Popen class. The agent was forced to learn temperature control while reducing the energy consumption through a reward function, which considers both temperature bounds and energy savings. Effectiveness of the RL approach to a multi-objective temperature control problem was demonstrated in this research.

One of common challenges in designing modern production machines is realizing high speed motion without sacrificing accuracy. To address this challenge it is necessary to maximize the stiffness of the mechanical structure and the control system with consideration on the main disturbance input, cutting forces. This paper presents analysis technologies for realizing high stiffness in production machines. First, CAE analysis techniques to evaluate the dynamic stiffness of a machine structure and a new method to construct the physical machine model for servo controller simulations are demonstrated. Second, cutting forces generated in milling processes are analyzed to evaluate their effects on the mechatronics system. In the effort to investigate the interaction among the structure, controller, and process, a flexible multi-body dynamics simulation method is implemented on a magnetic bearing stage as an example. The presented technologies can provide better understandings on the mechatronics system and help realizing high stiffness production machines.

Servo control loops are a core part in the control architecture of machine tools. Servo control loops manage acceleration, velocity and position of all the axes in a machine tool based on commands. The performance of servo control loops sets the basis for quality of production parts and cycle time reduction. First, this paper presents a general control architecture of machine tools and several control schemes in literature, which can be applicable to machine tools control; including Zero Phase Error Tracking Control (ZPETC) and Cross Coupling Control (CCC). After that, modern control strategies to mitigate the problem of high speed machining are reviewed. In high speed machining, high accelerations excite the machine structure up to high frequencies, thereby exciting the structure’s modes of vibration. These structural vibrations need to be damped if accurate positioning or trajectory following is required. Input shaping is an attractive option in dealing with structural vibrations. The advantages and drawbacks of using input shaping technique for machine tools are discussed in detail.

Control systems in machinery equipment provide correction signals to motion units in order to reduce or cancel out the mismatches between sensor feedback signals and command or desired values. In this paper, we introduce a simulator for control characteristics of machinery equipment. The purpose of the simulator development is to provide mechanical system designers with the ability to estimate how much dynamic performance can be achieved from their design parameters and selected devices at the designing phase. The simulator has a database for commercial parts, so that the designers can choose appropriate components for servo controllers, motors, motor drives, and guide ways, etc. and then tune governing parameters such as controller gains and friction coefficients. The simulator simulates the closed-loop control system which is built and parameter-tuned by the designer and shows dynamic responses of the control system. The simulator treats the moving table as a 6 degrees-of-freedom rigid body and considers the motion guide blocks stiffness, damping and their locations as well as sensor locations. The simulator has been under development for one and a half years and has a few years to go before the public release. The primary achievements and features will be presented in this paper.