Inertial measurement unit (IMU)-based 3D joint angle estimation have a wide range of important applications, among them, in gait analysis and exoskeleton robot control. Conventionally, the joint angle was determined via the estimation of 3D orientation of each body segment using 9-axis IMUs including 3-axis magnetometers. However, a magnetometer is limited by magnetic disturbance in the vicinity of the sensor, which highly affects the accuracy of the joint angle. Accordingly, this study aims to estimate the joint angle using the 6-axis IMU signals composed of a 3-axis accelerometer and a 3-axis gyroscope without a magnetometer. This paper proposes a recurrent neural network (RNN) model, which indirectly utilizes the joint kinematic constraint and thus estimates joint angles based on 6-axis IMUs without using a magnetometer signal. The performance of the proposed model was validated for a mechanical joint and human elbow joint, under magnetically disturbed environments. Experimental results showed that the proposed RNN approach outperformed the conventional approach based on a Kalman filter (KF), i.e., RNN 3.48° vs. KF 10.01° for the mechanical joint and RNN 7.39° vs. KF 21.27° for the elbow joint.