Learn French: Je l'attends au guichet. - I am waiting for him at the counter.

In French, the verb attendre already means 'to wait for.' You do not need an extra preposition like pour. You just say that you 'wait him' (Je l'attends).

What exactly does the l' stand for, and why is it in front of the verb?

The l' is an object pronoun standing in for le (him). In French, object pronouns go right before the verb. Because attends starts with a vowel, le drops its vowel to become l'. Note that l' could also mean 'her' or 'it' depending on the context!

Why do we use au instead of à le guichet?

Guichet is a masculine noun, so we would normally use le (the). However, in French, the preposition à (at/to) and le always combine to form the single word au.

We are learning the conditional tense for polite wishes. How would I change this sentence to say 'I would like to wait for him at the counter'?

You would use the conditional form of aimer (to like) plus the infinitive of our main verb. It becomes J'aimerais l'attendre au guichet. Notice that the pronoun l' stays right in front of the verb it belongs to (attendre).