If you say you are doing something to be on the safe side, you mean that you are doing it in case something undesirable happens, even though this may be unnecessary.
You might still want to go for an X-ray, however, just to be on the safe side.
Synonyms: be cautious, be careful, be prudent, be alert