Learn Russian: Не обижайся, это была шутка.

Not exactly. Like other emotion verbs in this lesson (such as радоваться or удивляться), обижаться uses the reflexive ending to show an internal emotional state. Instead of doing an action to someone else, it simply means 'to get offended' or 'to take offense'.

If I wanted to add 'at me' to this sentence, would I use the Dative case like with радоваться?

Good guess, but no. While verbs like радоваться and удивляться take the Dative case directly, обижаться behaves differently. It uses the preposition на followed by the Accusative case. You would say не обижайся на меня.

Why is the imperfective aspect used here (не обижайся) instead of the perfective?

In Russian, negative commands (telling someone not to do something) almost always use the imperfective aspect. Even if you are talking about this one specific joke, the negative imperative rule requires the imperfective не обижайся.

The word это is neuter. Why does the past tense verb была have a feminine ending?

When это acts as a placeholder meaning 'this is' or 'it is', the past tense of the verb 'to be' agrees with the noun that follows it. Because шутка (joke) is feminine, the verb must be the feminine была instead of the neuter было.