Learn Swedish: Gästen borde alltid tacka värden. - The guest should always thank the host.

Because borde is a modal verb, which means it works directly with the bare infinitive of the next verb. Just like in English we say "should thank" instead of "should to thank", in Swedish you say borde tacka instead of borde att tacka.

I noticed borde ends in -de. Is it a past tense verb?

Historically, yes! Borde is technically the past tense form of the verb böra (ought to). However, in modern Swedish, borde is almost always used to give advice or say what "should" happen right now or in the general present, just like in this sentence.

Why does alltid come between borde and tacka?

In a standard Swedish main clause, sentence adverbs like alltid (always), inte (not), and ofta (often) are placed right after the first conjugated verb. Here, the first verb is borde, so alltid slips in right behind it, before the second verb (tacka).

Why do both gästen and värden end in -en here?

They are in the definite form (the guest, the host). Swedish often uses the definite form to make general statements about a typical role. Even though it translates to "the guest" and "the host", it really means any guest and any host in general.