Learn Russian: Мой брат работает в институте.

Because it is telling us where the action happens. In Russian, when you talk about a location using в (in) or на (on), the noun changes to the Prepositional case. For most masculine words ending in a consonant like институт, you just add an -е to the end.

I thought в meant "in", but the English translation says "at". Why?

Prepositions do not always translate perfectly one-to-one! In Russian, в usually means being inside an enclosed space or institution. While English speakers naturally say they work "at" an institute or hospital, Russian uses в to show someone is working inside that organization.

How do we get работает from the dictionary word работать?

Работать is the infinitive (the base dictionary form). Because the subject is Мой брат (my brother, which is a "he"), we change the verb's ending to match. For many verbs ending in -ать, you drop the -ть and add -ет for "he", "she", or "it".

Why is the first word Мой and not Моя?