Learn Finnish: Sitä paitsi tämä ravintola on kallis. - Besides, this restaurant is expensive.

While paitsi on its own usually means 'except', combining it with a pronoun creates a fixed transitional phrase. You can think of sitä paitsi literally as 'except for that' or 'in addition to that', which is why it translates smoothly to 'besides' or 'furthermore' when you are adding a new point to an argument.

Why is the base pronoun se in the partitive case (sitä) here?

The word paitsi requires the noun or pronoun it governs to be in the partitive case. Even though sitä paitsi functions together as a fixed adverbial phrase, it still obeys this grammatical rule, which is why the base pronoun se (it/that) becomes the partitive sitä.

Since sitä paitsi is at the start of the sentence, why doesn't the verb come before the subject (on tämä ravintola)?

Unlike many Germanic languages that use strict V2 (verb-second) word order, Finnish generally maintains its standard Subject-Verb order even when a sentence begins with an adverb or transitional phrase. Inverting it to Sitä paitsi on tämä ravintola kallis would sound unusually poetic or overly dramatic.

Could I place sitä paitsi at the very end of the sentence instead?

Technically you could be understood, but it sounds quite unnatural. Because sitä paitsi is a transitional phrase specifically meant to link back to a previous thought and introduce a new argument, its natural home is right at the beginning of the clause.