To take the back track

to retrace one's steps; to retreat.

See also: Back