Learn Italian: Mi scusi per il ritardo. - Excuse me for the delay.

Because scusi is the formal imperative (the formal command form). With informal commands, like telling a friend to excuse you, the pronoun attaches to the end of the verb to make Scusami. But with formal commands, addressing someone as Lei, object pronouns like mi must go before the verb.

Why does scusi end in -i? Doesn't that usually mean it's the tu (informal) form?

You are right that in the regular present tense, tu scusi means you excuse. However, this sentence uses the imperative (command) mood. For -are verbs like scusare, the formal command (the Lei form) takes the -i ending, while the informal command takes the -a ending (scusa).

Could I say Scusami per il ritardo instead?

Yes, but only to a friend, a family member, or a child. The phrase Mi scusi is used strictly for formal situations, like speaking to a boss, a customer, or a stranger you address with Lei.

Why do we use the noun il ritardo instead of a verb phrase like being late?

In Italian, it is much more natural to use the noun il ritardo, meaning the lateness or the delay, after the preposition per in this context. While English speakers typically apologize for being late, Italians apologize for the delay.