disreputable or worthless people; rabble
Don't you deceive yourself by supposing for a moment that I am a man of family. I am a bit of dirty riffraff — Dickens
[Middle English riffe raffe, from rif and raf every single one, from early French rif et raf completely, from rifler to plunder + raffe act of snatching]