Yakushiji was constructed by Emperor Temmu in the late 7th century for the recovery of the emperor's sick wife. One of Japan's oldest temples, Yakushiji has a strictly symmetric, Chinese style layout, with the main and lecture halls standing on a central axis, flanked by two pagodas.

The main hall was rebuilt in the 1970s after being destroyed by fire, and houses the Yakushi trinity, a masterpiece of Japanese Buddhist art. The east pagoda is the temple's only structure to have survived the many fires and dates from 730. It appears to have six stories, but is in fact three storied, like the west pagoda.