Learn Dutch: We willen graag gitaar spelen. - We would like to play guitar.

Yes, on its own willen means 'to want'. However, adding the word graag (which translates to 'gladly') softens it. Combining willen and graag is the standard, polite way to say 'would like' in Dutch.

Why is the verb spelen at the very end of the sentence?

This is a strict rule in Dutch word order. When you use a modal verb like willen (want) as the conjugated verb, the second verb in the sentence (spelen) gets pushed all the way to the end in its full dictionary form.

Shouldn't there be a word for 'the' or 'a' before gitaar?

In English you can say 'play the guitar', but in Dutch, you usually drop the article when talking about the general activity of playing a musical instrument. So it is simply gitaar spelen without de or een.

Why does the sentence use willen instead of wil?

Because the subject is We, which is plural. In Dutch, all plural subjects (we, jullie, ze) use the full infinitive form of the verb, which is willen. The form wil is used for singular subjects like ik (I) or hij (he).