Learn German: Ich bringe mir die neue Sprache selbst bei.

In this sentence, mir is a reflexive pronoun showing who you are teaching (you are teaching it to yourself). selbst is an intensifier that emphasizes how you are doing it (independently, without a teacher). Together, mir ... selbst essentially means "to myself, all by myself."

Could I use selber instead of selbst here?

Yes! selber has the exact same meaning as selbst. The only difference is that selber is mostly used in casual, spoken German, while selbst is standard for both writing and speaking.

Why is it mir (dative) instead of mich (accusative)?

The verb beibringen (to teach) requires the dative case for the person receiving the instruction, and the accusative case for the subject being taught. You are teaching "the new language" (accusative: die neue Sprache) "to yourself" (dative: mir).

Why is bei at the very end of the sentence?

This is because beibringen is a separable verb. In a standard main clause, the conjugated part of the verb (bringe) takes the second position, and the separable prefix (bei) gets sent all the way to the end.