Learn Portuguese (Portugal): Os vossos livros estão na mesa.

In Portuguese, you almost always put a definite article (o, a, os, or as, meaning "the") before a possessive word. So instead of just saying "your books" (vossos livros), you literally say "the your books" (os vossos livros).

Does vossos match the people I am talking to, or the books?

The ending of vossos matches the things being owned, not the owners! Because livros (books) is masculine and plural, you use the masculine plural form vossos. If it were a single book, it would be o vosso livro.

I learned seus means "your". Why use vossos here?

In Portugal, seus usually means "his", "hers", or "theirs". To say "your" when talking to multiple people, it is very common to use vossos to avoid confusion, even if you are talking to them using the everyday vocês.

Why use estão instead of são for "are"?