Learn Russian: Мы редко ездим в отпуск.

The word редко (rarely) tells us this is about a habit or a repeated action in general, not a single trip happening right now. For repeated or habitual trips, you must use the multidirectional verb ездить. ездим is the "we" form of that verb.

Could I use ходим instead of ездим to say "we go"?

Not usually! The verb ходить implies going on foot or making a short local round trip. Because going on vacation (отпуск) generally implies traveling out of town by a vehicle (plane, train, or car), Russian requires the "by vehicle" verb ездить.

Is отпуск in the accusative case here? It looks like the dictionary form.

Yes, it is in the accusative case because it is the destination of your motion after the preposition в (to/into). Because отпуск (vacation) is a masculine inanimate noun (not a living person or animal), its accusative form looks exactly the same as its dictionary form.

Why does Russian use в for vacation instead of a word for "on"?

Prepositions rarely translate perfectly between languages. While English speakers go "on" vacation, Russian treats a vacation as a state or period of time that you enter, so you go "into" it (в отпуск).