Learn Swedish: Han är mycket petig och räknar vartenda öre. - He is very picky and counts every single cent.

Because the noun it modifies, öre (cent/penny), is an ett-word (ett öre). If the noun were an en-word, like krona, you would use varenda instead (varenda krona).

Why is öre in the indefinite form here?

Just like the word varje (every), the intensifiers varenda and vartenda are always followed by a singular noun in the indefinite form. Therefore, it is vartenda öre, not the definite vartenda öret.

How exactly is petig used? Does it mean picky about food or meticulous about details?

Both! It translates to picky, fussy, or meticulous. It comes from the verb att peta (to poke or pick at). You can be petig with your food, or as in this sentence, highly meticulous with your finances.

Sweden doesn't use physical öre coins anymore, so is this just an idiom?

The physical coins are gone, but an öre is still used in digital pricing, as 100 öre make up 1 krona. Furthermore, räknar vartenda öre is a very common fixed expression equivalent to the English phrase counting every single penny.