Learn Russian: Вчера мы не смотрели телевизор.

Because the subject is мы (we), which is plural. To form the past tense of a regular verb like смотреть (to watch), you drop the -ть ending from the dictionary form and add a past-tense ending. Since the subject is plural, you use the plural ending -ли.

There is no word for 'did' in the Russian sentence. How do we know it means 'did not watch'?

Russian does not use helper verbs like 'do' or 'did' to make negative sentences or questions. You simply put не (not) directly in front of the past tense verb. So, не смотрели automatically translates to 'did not watch'.

Does не always have to go right before the verb?

Yes, to state that an action did not happen, не must immediately precede the verb. If you moved it and said Не мы смотрели телевизор, it would change the meaning to 'Not we watched television' (implying someone else did).

Why doesn't the word телевизор change its ending here? Isn't it the direct object?

It is the direct object being watched, so it is in the accusative case. However, телевизор is a masculine, inanimate noun (a thing, not a living person or animal). In Russian, the accusative case for masculine inanimate nouns looks exactly the same as the basic dictionary form.