Learn Icelandic: Þó að vináttan sé gömul gleðja þau hvert annað. - Even though their friendship is old, they make each other happy.

This is the "split case" feature of reciprocal pronouns. The first part (hvert) matches the case of the subject (þau, which is nominative). The second part (annað) takes the case governed by the main verb. Since gleðja (to make happy) requires an accusative object, we use the accusative form annað.

Since þau is plural, why are hvert and annað in their singular forms?

Even though the subject is plural, the reciprocal pronoun is almost always grammatically singular—think of it as literally saying "each [one] gladdens another". Because þau is a neuter pronoun (used here for a mixed-gender group), both halves take neuter singular forms.

Could I use hvort annað instead of hvert annað here?

Only if you were talking about exactly two people! The pronoun hvor annar is strictly for pairs. Because this sentence uses hvert annað (from hver annar), it implies that this friendship group consists of three or more people.

Why is the verb sé used instead of er?