Kaikki tulivat paitsi minä.

Here kaikki means everyone / all the people.

Finnish kaikki can also mean everything, so context matters:

• Kaikki tulivat. = Everyone came.
• Kaikki meni hyvin. = Everything went well.

In your sentence, because the verb is about people coming, kaikki is understood as everyone.

Finnish does not follow English here.

When kaikki means all the people / everyone, Finnish normally uses a plural verb:

• Kaikki tulivat. = Everyone came.

When kaikki means everything, singular is common:

• Kaikki on valmista. = Everything is ready.

So tulivat is plural because kaikki refers to a group of people.

Tulivat is the past tense, 3rd person plural form of tulla (to come).

So:

• tulla = to come
• tulivat = they came

That is why Kaikki tulivat literally works like All of them came.

Paitsi means except or apart from.

It marks an exception to the main statement:

• Kaikki tulivat paitsi minä. = Everyone came, with one exception: me.

It is a very common Finnish word for excluding one person or thing from a group.

Because here minä is understood as the subject of an omitted clause.

The full idea is something like:

• Kaikki tulivat, paitsi minä en tullut.

Since minä would be the subject of en tullut, the nominative form minä is used.

This is different from a sentence where the excluded word would be an object:

• Hän kutsui kaikki paitsi minut. = He invited everyone except me.

There, minut is correct because it matches the object role.

Yes. That is also correct and very natural.

Compare:

• Kaikki tulivat paitsi minä.
• Kaikki paitsi minä tulivat.

Both mean the same thing. The difference is mostly emphasis:

• Kaikki paitsi minä tulivat puts the exception inside the subject phrase from the start.
• Kaikki tulivat paitsi minä first gives the general statement, then adds the exception at the end for extra focus.

Finnish often leaves out words that are easy to understand from context. This is called ellipsis.

So instead of saying the full version:

• Kaikki tulivat, paitsi minä en tullut.

Finnish usually prefers the shorter, more natural version:

• Kaikki tulivat paitsi minä.

The missing part is understood automatically.

Not in this sentence.

• Kaikki tulivat paitsi minä. → no comma

Here paitsi minä is just a short exception phrase.

If paitsi introduces a full clause, then a comma is normally used:

• Kaikki tulivat, paitsi minä en tullut.

So the comma depends on whether a new clause follows.

Yes, if the context makes it clear that you are the one left out.

• Kaikki muut tulivat. = Everyone else came.

This is close in meaning, but slightly less explicit than Kaikki tulivat paitsi minä, because it relies on context to show everyone else than whom.

So:

• Kaikki tulivat paitsi minä. = fully explicit
• Kaikki muut tulivat. = natural if the missing person is already obvious