Shape memory alloy (SMA) has been widely used for many engineering and scientific applications, because it is largely deformable with high power density, and can be actuated easily by resistive heating. It is possible to reduce the size of the actuators by integrating or embedding SMA into the structures. While many applications have been reported regarding linear or bending actuators using the SMA wire, the development of a rotary actuator remains important and challenging for the engineering applications. Here, a new type of millimeter-scale rotary actuator is proposed based on the twisted SMA wires. SMA wires are twisted, folded, and integrated into the rotary actuator, and simple change of the twist direction enables the rotary actuator to rotate in the opposite direction. By integrating the oppositely twisted SMA wires into one rotary actuator unit, bidirectional rotary motions are possible. The actuation mechanism, design, and fabrication processes of the proposed rotary actuator are presented and demonstrated with its actuation performance. The fabricated actuators had average rotary working ranges from -38.68±4.92 deg to +45.37±8.79 deg in counterclockwise (CCW) and clockwise (CW) directions. This study will leverage the practical advances in the relevant engineering and scientific applications.