The key components of smart manufacturing, a central concept in the era of the 4th Industrial Revolution, consist of digital twin technology, AI, and computer vision technology. In this study, these technologies were utilized to govern the Poppy robot, a humanoid robot designed for educational and research purposes. The digital twin creates a virtual environment capable of real-time simulation, analysis, and control of the robot’s motions. The digital twin of the robot was constructed using Unity, a 3D development program. Motion data was captured while simulating the physical structure and movements of the virtual robot. This data was then fed into a Tensorflow-based deep neural network to generate a regression modelthat predicts motor rotation based on the position of the robot’s hand. By integrating this model with a Python-based robot control program, the robot’s movements could be effectively managed. Additionally, the robot was controlled using Openpose, a computer vision algorithm that predicts characteristic points on a human body. Position data for human joint points was collected from 2D images, and the motor angle was calculated based on this data. By implementing this approach on an actual robot, it became possible to enable the robot to replicate human movements.

Knee contact forces and knee stiffness are biomechanical factors worth considering for walking in knee osteoarthritis patients. However, it is challenging to acquire these factors in real time; thus, making it difficult to use them in robotic rehabilitation and assistive systems. This study investigated whether trained deep neural networks (DNNs) can capture the biomechanical factors only using kinematics during gait, which is possible to measure via sensors in real time. A public dataset of walking on the ground was analyzed through biomechanical analysis to train and test DNNs. Using the training dataset, several DNN topologies were explored via Bayesian optimization to tune the hyperparameters. After optimization, DNNs were trained to estimate the biomechanical factors in a supervised manner. The trained DNNs were then evaluated using two new datasets, which were not used in the training process. The trained DNNs estimated the biomechanical factors with a high level of accuracy in both types of test datasets. Results confirmed that DNNs can estimate the biomechanical factors based on only kinematics during gait.