Learn Hardcore Japanese: watasi ha tomodati to kenka wo sinai you ni ki wo tukemasu. - I try to be careful not to fight with my friends.

In everyday speech, both けんかする and けんかをする are natural. In your sentence, it appears in the negative: けんかをしない = to not have a fight / to not quarrel.

Why is it しない and not しません before ように?

Before ように, you normally use the plain (dictionary) form of the verb, not the polite ～ます form.

So:

Positive: するように (so that I do / in order to do)
Negative: しないように (so that I don’t / in order not to)

Using しませんように is possible in some special, more formal/prayer-like contexts (e.g. in written wishes), but in a normal sentence like this, the natural grammar pattern is:

plain form + ように + main verb

Hence: けんかをしないように気をつけます (not しませんように気をつけます).

What exactly does ように do in this sentence?

ように here expresses purpose or desired result: “so that / in order that / so as to”.

Breakdown:

友達とけんかをしない = (I) do not fight with my friends
友達とけんかをしないように = so that I don’t fight with my friends / in order not to fight with my friends

Grammatically, 友達とけんかをしないように is an adverbial clause modifying 気をつけます:

(I) will pay attention / be careful in such a way that I don’t fight with my friends.

With a negative verb before ように, the nuance is often “to avoid X / to prevent X from happening”, which fits this sentence very well: to avoid fighting with my friends.

What does 気をつけます literally mean, and how is it working here?

Literally:

気 = spirit, mind, attention, awareness (a very broad word)
気をつける = to attach your attention / mind to something → to pay attention / to be careful

So 気をつけます is the polite non-past form:

気をつけます = I be careful / I pay attention / I will be careful

In this sentence:

友達とけんかをしないように気をつけます。
I’ll be careful so that I don’t fight with my friends.

The main action is 気をつけます; the ように-clause explains for what purpose you are being careful.

Why is it 友達と and not 友達に or something else? What does と mean here?

The particle と often marks a partner in actions done together with someone, whether cooperative or reciprocal:

友達と遊ぶ = play with a friend
彼と結婚する = marry him
友達と話す = talk with a friend

For reciprocal actions like fights, arguments, and the like, the other person is usually marked with と:

友達とけんかする = fight / quarrel with a friend

So in your sentence:

友達とけんかをしない = not fight with my friends

Using 友達にけんかする would not be natural; と is the normal particle for “fight with (someone)” in this pattern.

Is the word order fixed, or can I move 私は or other parts around?

Japanese word order is fairly flexible, as long as the particles stay with the right words. The most important part is usually the verb at the end.

Your original sentence:

私は友達とけんかをしないように気をつけます。

Natural variations include:

友達とけんかをしないように、私は気をつけます。
友達と、私はけんかをしないように気をつけます。 (a bit more marked emphasis on 友達と)

The most neutral and common is probably the original order or:

友達とけんかをしないように気をつけます。 (dropping 私は)

What you normally don’t change is putting the main verb 気をつけます anywhere except near the end; Japanese strongly prefers the verb at the end of the sentence.

Can I omit some parts, like 私は or the を in けんかを?

Yes, quite a bit can be omitted naturally.

Dropping 私は

Japanese often omits the subject when it’s obvious from context:

友達とけんかをしないように気をつけます。
(I will be careful not to fight with my friends.)

This is very natural and probably the most common way to say it.

Dropping を after けんか

Because けんかする is a common verb-like expression, people often omit を:

友達とけんかする
友達とけんかをする

Both are fine. So in your sentence:

友達とけんかしないように気をつけます。

is also perfectly natural.

Dropping 友達 or other pieces

You can drop 友達 if it’s understood from context:

けんかしないように気をつけます。
I’ll be careful not to fight. (Who you’d fight with is left unsaid.)

But if you want to be clear that it’s friends in particular, keep 友達と.

Why is it 気をつけます and not something like 気をつけています or けんかをしないようにします? Do they mean different things?

All are related but have slightly different nuances:

気をつけます

Neutral polite non-past:
- Can mean I will be careful (from now on)
- Or I am (generally) careful / I make a point of being careful

In your sentence, it sounds like a decision / intention: I’ll be careful so I don’t fight with my friends.

気をつけています

Emphasises an ongoing state:
- I am careful / I am being careful (these days / right now)

Example:

友達とけんかをしないように気をつけています。
I’m (currently) making sure not to fight with my friends.

～ようにします

V-plain ようにする means to make an effort to V / to arrange things so that V happens.

Example:

友達とけんかをしないようにします。
I’ll try not to fight with my friends / I’ll make sure not to fight with my friends.

Your original with 気をつけます focuses on the carefulness itself; ～ようにします focuses more on making a conscious effort to achieve that result. They overlap in meaning but are not identical in nuance.

Why is けんか written in hiragana? Is 喧嘩 also correct?

Yes, 喧嘩 is the full kanji form, and it is correct.

However:

The kanji 喧 and 嘩 are relatively difficult and uncommon, especially for learners.
Many everyday words with tricky or rare kanji are commonly written in hiragana in modern Japanese.

So you will often see:

けんかする / けんかをする (very common)
喧嘩する / 喧嘩をする (also correct, more likely in novels, manga, or by people who like using kanji)

For study purposes, it’s useful to recognize 喧嘩, but using けんか in your own writing is perfectly natural.