Na gornjoj polici je mala kutija.

Because na is expressing a location, and with that meaning it takes the locative case.

• dictionary form: polica = shelf
• in this sentence: na polici = on the shelf

So polica changes to polici because it is in the locative singular.

A useful contrast:

• na polici = on the shelf, already there
• na policu = onto the shelf, movement toward it

Adjectives in Croatian must agree with the noun they describe in:

• gender
• number
• case

Here, polici is feminine singular locative, so gornja also has to become feminine singular locative:

• gornja polica = upper shelf
• na gornjoj polici = on the upper shelf

So gornjoj is simply the adjective form that matches polici.

Because na usually means on or on top of, while u usually means in or inside.

A shelf is treated as a surface, so Croatian says:

• na polici = on the shelf

By contrast:

• u kutiji = in the box

So na is the natural choice with polica.

Because mala kutija is the subject of the sentence, and subjects are normally in the nominative case.

• mala kutija = nominative, used for the subject
• malu kutiju = accusative, often used for a direct object

Compare:

• Na gornjoj polici je mala kutija. = A small box is on the upper shelf.
• Vidim malu kutiju. = I see a small box.

So the form changes depending on the role of the noun in the sentence.

Je is a clitic, an unstressed short form of biti = to be. Croatian clitics usually go in second position in the clause.

In this sentence, the opening phrase Na gornjoj polici counts as the first unit, so je comes right after it:

• Na gornjoj polici je mala kutija.

That is why it does not appear at the very beginning or the very end.

Yes. That is also correct.

Both versions are natural, but the emphasis is slightly different:

• Na gornjoj polici je mala kutija.
  Focuses more on the location.

• Mala kutija je na gornjoj polici.
  Focuses more on the box.

So the word order can change, but the basic meaning stays the same.

Croatian does not have articles.

So kutija can mean:

• a box
• the box

The exact meaning depends on context, not on a separate word like a or the.

That is why mala kutija can mean either a small box or the small box, depending on the situation.

Gornji is an adjective meaning upper or top, so it can describe a noun:

• gornja polica = upper shelf

Gore is usually an adverb, meaning something like up, upstairs, or above. It does not work the same way as an adjective.

So here you need gornjoj, not gore, because the word is describing polici.

The dictionary forms are:

• na = on
• gornji = upper, top
• polica = shelf
• biti = to be
• mali = small
• kutija = box

In the sentence, some words change form:

• gornji → gornjoj
• polica → polici
• mali → mala
• biti → je

Those changes happen because of case, agreement, and sentence structure.

Then Croatian would normally use the accusative after na, not the locative.

Compare:

• Na gornjoj polici je mala kutija. = The small box is on the upper shelf.
  This describes location.

• Stavio sam malu kutiju na gornju policu. = I put the small box onto the upper shelf.
  This describes movement.

So this is an important contrast:

• na gornjoj polici = on the upper shelf
• na gornju policu = onto the upper shelf