Learn Finnish: Kokoelma, jota myyn, on iso. - The collection that I am selling is large.

The relative pronoun gets its case from what it does inside its own clause. Here, it is the object of the verb myyn (I am selling). Because selling is an ongoing, incomplete action in this context, the object must be in the partitive case.

Could I use mitä instead of jota here? Both can mean 'that' or 'which'.

No, you need a form of joka here. You use joka (and its forms like jota) to refer back to a specific noun, which is kokoelma (collection) in this sentence. The word mikä (and its partitive mitä) is used when referring to abstract concepts, whole sentences, or indefinite pronouns like kaikki (everything).

Are both commas around jota myyn necessary?

Yes, Finnish comma rules are very strict about this. A relative clause (the part of the sentence starting with joka or its forms) must always be completely separated from the rest of the sentence by commas on both sides.

Why is the adjective iso in the basic nominative case, not partitive?

iso describes the main subject, kokoelma (the collection), which is in the basic nominative case. The partitive word jota only affects the relative clause (jota myyn). If you take out the relative clause, the main sentence is simply Kokoelma on iso.