Ovaj kaput je najskuplji u prodavnici.

Because ovaj has to agree with kaput.

• kaput is masculine
• it is singular
• here it is in the nominative case because it is the subject of the sentence

So the correct form is:

• ovaj kaput = this coat

Compare:

• ovaj for masculine
• ova for feminine
• ovo for neuter

For example:

• ovaj kaput = this coat
• ova jakna = this jacket
• ovo odelo = this suit

In Serbian, many nouns that end in a consonant are masculine, and kaput ends in -t.

So:

• kaput = masculine
• nominative singular form: kaput

Because it is masculine, words that describe it must also be masculine:

• ovaj kaput
• najskuplji kaput

je is the 3rd person singular present tense of biti = to be.

So here:

• je = is

The structure is:

• Ovaj kaput = This coat
• je = is
• najskuplji = the most expensive
• u prodavnici = in the store

Serbian often leaves out subject pronouns like he, she, or it, but here the subject is already stated: ovaj kaput.

Because najskuplji is the superlative form of the adjective skup = expensive.

The pattern is:

• skup = expensive
• skuplji = more expensive
• najskuplji = most expensive

So Serbian forms the superlative by adding naj- to the comparative form.

This is very common:

• lep = beautiful
• lepši = more beautiful
• najlepši = most beautiful

Because Serbian has no articles like English a and the.

So English says:

• the most expensive

But Serbian simply says:

• najskuplji

The meaning is clear from the grammar and context. You do not need a separate word for the.

Because after u meaning in or inside, Serbian uses the locative case for location.

Base form:

• prodavnica = store, shop

Locative singular:

• u prodavnici = in the store

So:

• u prodavnici = location
• u prodavnicu = motion into the store

This is an important contrast:

• On je u prodavnici. = He is in the store.
• On ide u prodavnicu. = He is going into the store.

In many situations, u prodavnici is best translated as in the store, but in natural English it can sometimes correspond to at the store, depending on context.

In this sentence, the idea is that among the coats available there, this one is the most expensive. So in the store is the most direct translation.

Yes. Just like ovaj, the adjective najskuplji agrees with the noun it refers to.

Here it is:

• masculine
• singular
• nominative

So we get:

• ovaj kaput je najskuplji

If the noun changed, the adjective would change too:

• Ova jakna je najskuplja. = This jacket is the most expensive.
• Ovo odelo je najskuplje. = This suit is the most expensive.

Yes, Serbian word order is more flexible than English word order. But the given sentence is the most neutral and natural way to say it:

• Ovaj kaput je najskuplji u prodavnici.

You may also hear other orders for emphasis, but they can sound marked or stylistically different. For a learner, the given order is the safest one to use.

They are very close in meaning, but the focus is a little different.

• Ovaj kaput je najskuplji u prodavnici.
  = This coat is the most expensive in the store.
  You start with a specific coat and say something about it.

• Ovo je najskuplji kaput u prodavnici.
  = This is the most expensive coat in the store.
  You start with this as a thing you are pointing to, and then identify it as the most expensive coat.

So the Serbian sentence you were given focuses first on this coat.

Because ovo does not agree with kaput.

• kaput is masculine
• so the correct demonstrative is ovaj

Use:

• ovaj kaput = this coat

ovo is used for neuter nouns, or sometimes on its own as this when pointing to something in a general way:

• ovo odelo = this suit
• Ovo je skupo. = This is expensive.

Yes, often you can.

• prodavnica = store/shop
• radnja = shop/store

So:

• Ovaj kaput je najskuplji u radnji.

is also possible.

Prodavnica is a very common standard word in Serbian. Radnja is also common, but exact usage can depend on region and context.

A rough English-style guide is:

O-vai ka-poot ye nai-skoop-lyi oo პრო-da-vni-tsee

A few useful notes:

• j in Serbian sounds like English y
• je sounds roughly like ye
• c in prodavnici sounds like ts
• lj in najskuplji is a soft sound, something like ly

So:

• ovaj ≈ o-vai
• je ≈ ye
• prodavnici ≈ pro-da-vni-tsi