If you say that someone has to take the rough with the smooth, you mean that they have to accept the unpleasant or difficult things that happen as well as the pleasant things.
See full dictionary entry for smooth
take the rough with the smooth
to be willing to accept both the unpleasant and the pleasant aspects of something
You have to take the rough with the smooth. I never promised there would be no risk.