Learn Italian: Ha rifiutato la possibilità senza nessun'esitazione. - He rejected the possibility without any hesitation.

It modifies the feminine noun esitazione. The base feminine form of the adjective is nessuna. When nessuna comes before a feminine noun that starts with a vowel, it drops the final a and takes an apostrophe, becoming nessun'.

If the noun were masculine and started with a vowel, like amico, would it still use an apostrophe?

No, and this is a very common trap! For masculine nouns starting with a vowel, you use the form nessun without an apostrophe (for example, nessun amico). The apostrophe is strictly used for feminine nouns to show where the a from nessuna was removed.

Words like nessuno usually require a non before the verb. Why is there no non here?

The preposition senza (without) already carries a negative meaning. When you pair senza with a negative word like nessun', it acts as the sentence's negative trigger, so you do not need to add non to the verb.

How can I tell that esitazione is feminine?

Almost all Italian nouns ending in -zione are feminine. Recognizing this ending is a highly reliable shortcut for knowing when to use feminine modifiers like la, una, or in this case, nessuna (which then becomes nessun').