Learn Hardcore Icelandic: Ég býð þér kaffi.

Because the verb bjóða “to offer/invite” takes the recipient in the dative case: the pattern is bjóða e-m e-ð (“offer someone something”). The 2nd person singular pronoun declines like this:

nominative: þú (subject)
accusative: þig (direct object)
dative: þér (indirect object = “to/for you”)
genitive: þín In this sentence the recipient is “you,” so we use the dative: þér.

Do I need a preposition for “to you,” like “til þín” or “fyrir þig”?

No. Icelandic typically expresses “to/for someone” with the dative case alone, without a preposition, when the verb requires it. With bjóða, the recipient is dative: bjóða þér.

til þín means “to your place/toward you” (movement/goal).
fyrir þig means “for you” in the sense “on your behalf/for your benefit,” not the recipient of an offer. So here, just use þér.

Why is there no word for “a” before kaffi?

Icelandic has no indefinite article (“a/an”). Bare nouns can be indefinite by default. If you need a definite meaning, Icelandic uses a suffixed article:

kaffi = “coffee” (indefinite/mass)
kaffið = “the coffee” (definite, specific coffee already known in context)

What case is kaffi in here?