You use all that in statements with negative meaning when you want to weaken the force of what you are saying. [spoken, vagueness]
He wasn't all that much older than we were.
He said it would not be all that difficult to reach a peaceful conclusion to the conflict.