Learn Dutch: Wat houden we over van het budget? - What do we have left over from the budget?

Because it is a separable verb used in a main clause. In a question, the conjugated part (houden) goes right after the question word, and the separable prefix (over) moves to the end of the core sentence.

Why does over appear before van het budget instead of at the very end of the sentence?

In Dutch, prepositional phrases like van het budget can optionally be placed after the verbs at the end of the sentence. You could also say Wat houden we van het budget over? Both are correct and sound completely natural.

Could I use the verb overblijven (to remain) instead?

Yes, but the grammar would change. Overhouden requires a subject who has something left (like we). Overblijven just means something is left over on its own. To use it, you would have to drop the we and ask: Wat blijft er over van het budget? (What is left over from the budget?).

How would I answer this question by saying 'We have a quarter left'?

You would use the fraction een kwart and say We houden een kwart over. The separable verb splits and wraps around the fraction, placing over at the end of the sentence.