Learn Russian: Что за концептуальная куча! Это просто свалка. - What kind of conceptual pile is this! It is simply a dump.

You are exactly right that the standalone preposition за normally takes the accusative or instrumental case. However, что за is a fixed idiomatic particle used for exclamations or asking "what kind of". Because it acts as a single particle rather than a true preposition, the noun phrase that follows it bypasses standard preposition rules and always stays in the nominative case.

Can I use Какая instead of Что за here?

Yes, you could say Какая концептуальная куча! to mean "What a conceptual pile!" Both express strong emotion or surprise. However, что за often carries a slightly stronger tone of bewilderment, disbelief, or even slight irritation, which perfectly matches the sarcastic realization in the second sentence.

What is the exact difference between куча and свалка in this context?

Куча simply means a "pile" or "heap" of anything, good or bad. Свалка specifically refers to a "dump," "landfill," or a place where garbage is thrown. The speaker is sarcastically looking at what is claimed to be modern art (a conceptual pile) and bluntly dismissing it as literal trash.

How should I intonate or stress что за in an exclamation like this?