Learn Farsi: ماشین قربانی محاصره شده بود.

To make a compound verb passive in Farsi, we replace the active part kardan (to do) with the passive part shodan (to become), giving us mohasere shodan (to be surrounded). To put this into the past perfect passive ("had been surrounded"), we use the past participle shode followed by the past auxiliary bud, resulting in mohasere shode bud.

Could I just say mohasere bud instead of mohasere shode bud?

Not if you want the specific "had been" meaning. Mohasere bud is the simple past state, meaning "it was surrounded." Adding shode creates the past perfect passive, which emphasizes that the action of surrounding was fully completed before another point in the past.

How is the possession "the victim's car" indicated since there is no written word for "of" or "'s"?

Possession is shown by the invisible ezafe vowel linking the two nouns, pronounced here as mashin-e qorbani. It literally translates to "car of [the] victim." Because it is just a short vowel sound, it isn't written in the standard Farsi script, but you must speak it to connect the noun to its owner.

Does mohasere just mean "surrounded" in a casual sense, like being surrounded by friends?

Usually no. Mohasere has a heavy, serious tone and often translates to "siege," "blockade," or a military/police encirclement. In this sentence, it implies the car was deliberately trapped or cornered, which perfectly fits the context of a (victim).