Learn Spanish (Latin America): Ellos dudan que el monstruo sea imaginario. - They doubt that the monster is imaginary.

The verb dudar (to doubt) triggers the subjunctive mood because it expresses uncertainty. Whenever you use dudar que, the verb in the following clause must be in the subjunctive. Here, the present subjunctive of ser is sea.

Would we still use the subjunctive if they didn't doubt it?

No. If you say no dudan que (they do not doubt that), you are expressing certainty or a stated fact. Certainty takes the indicative mood, so the sentence would become Ellos no dudan que el monstruo es imaginario.

Why is it sea (from ser) and not esté (from estar)?

We use ser because being imaginary is a fundamental characteristic or defining trait of the monster. We use estar for temporary states, emotions, or locations, which does not fit here.

Can I leave out the word que like we often do with the word that in English?

No, you cannot drop que in Spanish. In English, we can optionally drop the connecting word and just say They doubt the monster is imaginary. In Spanish, the connecting word que is always required to link the two halves of the sentence.