To deny one's self

to decline the gratification of appetites or desires; to practice self-denial.

See also: Deny