Learn Russian: Этот мягкий коврик мы тоже возьмём с собой.

The reflexive pronoun себя (here in the Instrumental case as собой) always refers back to the subject of the sentence. Since the subject is мы (we), с собой literally means "with ourselves". In Russian, you must use собой instead of a regular pronoun like нами when the person doing the action is also the one being accompanied.

Why do этот мягкий коврик look like they are in the dictionary form (Nominative case) if they are the object being taken?

They are actually in the Accusative case because they are the direct object of the verb возьмём (will take). However, for inanimate masculine nouns like коврик (mat) and their matching adjectives like мягкий (soft), the Accusative case perfectly matches the Nominative case. Only animate masculine nouns change their endings in the Accusative.

The translation says "will take", but возьмём is just one word. Where is the word for "will"?

возьмём is the "we" form of the perfective verb взять (to take). Because perfective verbs focus on the result or completion of an action, they do not have a present tense. When you conjugate them, they automatically refer to the future. You never use the future helper word быть (will be) with perfective verbs!

Does the word order matter here? Why is мы in the middle of the sentence?