Learn Danish: Jeg skriver en liste over alt vi skal købe. - I am writing a list of everything we have to buy.

In Danish, when you are talking about the contents, subject matter, or coverage of a list, map, or chart, you use over. Think of it as having an overview of the items. The preposition af is generally used for physical composition or origin, which doesn't fit a shopping list.

Could I add a word like hvad or som after alt?

Yes, you could absolutely say alt, hvad vi skal købe or alt, som vi skal købe. However, because the relative pronoun acts as the object of the clause (you are buying the items), it is very common and natural to drop it in spoken and written Danish, just like you can drop the word "that" in English.

Why use skal for "have to" instead of må?

The modal verb skal is the standard way to express a firm plan, an intention, or an external necessity, making it the perfect choice for a shopping list. må is more often used to express permission (may/allowed to) or a very strong personal, internal obligation.

Why is the neuter form alt used instead of al or alle?

Here, the word functions as an independent pronoun meaning "everything". When referring to an abstract, general concept of "everything" rather than a specific countable plural group (which would use alle) or an uncountable common-gender noun (which would use al), Danish always uses the neuter form alt.