If you impose a late payment charge or interest onto a customer or client who has not paid then you do not pay VAT on this amount, as this is not a VAT-able supply.  You’ll need to work out the late payment charge on the gross amount that is outstanding also.

If you have agreed extended payment terms with your customer and additional charges in association with this additional credit, then these charges would be VAT-able since they would be considered as a price extension.  Of course, if subsequent to the agreed period your customer still defaults then any additional charges or interest will no be VAT-able.