Both pence and pennies have existed as plural forms of penny since at least the 16th century. The two forms now tend to be used for different purposes: pence refers to sums of money (five pounds and sixty-nine pence) while pennies refers to the coins themselves (I left two pennies on the table). The use of pence rather than penny as a singular (the chancellor will put one pence on income tax) is not regarded as correct in standard English.