Learn Russian: Кстати, ты помнишь нашего соседа? - By the way, do you remember our neighbour?

It is actually in the accusative case! The verb помнить takes a direct object in the accusative. However, because сосед (neighbour) is a masculine, animate noun (a living person), its accusative form is identical to its genitive form. Therefore, both the noun and its modifier наш take the animate accusative endings (-его, -а).

Does помнить always take the accusative case directly?

Yes, помнить (to hold in your memory) and вспомнить (to actively recall) take the accusative case directly without a preposition. Learners sometimes mix this up with напоминать (to remind), which works differently: it takes the dative case for the person being reminded and the accusative for the thing being reminded of.

Is кстати used exactly like 'by the way' in English?

Yes, in conversational Russian, it is the most common equivalent for 'by the way' or 'speaking of which.' Literally, it comes from к (to) plus стать (an old word for shape or suitability), meaning something is 'to the point' or 'apropos.' You can also use между прочим, but кстати is punchier and more common for suddenly introducing a related thought.

Why use the imperfective помнишь instead of a perfective verb here?