Learn French: C'est le bal au milieu duquel il est parti. - It is the ball in the middle of which he left.

The phrase for "in the middle of" is au milieu de. The relative pronoun refers back to le bal, which is masculine singular, so we need lequel. When the preposition de combines with lequel, they must contract to form duquel.

Since dont also means "of which", could I say "au milieu dont"?

No, you cannot put dont after a compound preposition (a preposition made of multiple words ending in de) like au milieu de or à côté de. In these cases, you must use the de + lequel forms, such as duquel.

What if the noun was feminine, like la fête (the party)? How would the sentence change?

If the noun were feminine, you would use laquelle. Because de and laquelle do not contract together into a single word, the sentence would become: C'est la fête au milieu de laquelle il est parti.

Just to review, why is it est parti and not a parti?