You use as a matter of fact to introduce a statement that gives more details about what has just been said, or an explanation of it, or something that contrasts with it.
It's not that difficult. As a matter of fact, it's quite easy.
'I guess you haven't eaten yet.'—'As a matter of fact, I have,' said Hunter.
Synonyms: actually, in fact, in reality, in truth