Learn Hardcore Japanese: apuri de ryoukin wo keisansimasu.

It marks the means or place of the action. Here it means “by means of/using,” so アプリで = “using the app.” You could make the “using” idea explicit with アプリを使って料金を計算します. If you want to highlight “inside the app,” say アプリ内で.

Why is を used with 料金?

Because 計算する is a transitive verb and takes a direct object marked by を. So 料金を計算します = “(I) calculate the fee.” In passive voice the fee becomes the subject: 料金が計算されます. You can also topicalize: 料金は（アプリで）計算します where は replaces を to mark the topic.

Can I change the word order?

Yes. Both are natural:

アプリで料金を計算します。
料金をアプリで計算します。 Japanese relies on particles, not fixed order. The second puts a bit more focus on 料金 by mentioning it earlier, but both are common.

Who is doing the calculating? There’s no subject.

Japanese often omits subjects when obvious from context. 計算します could mean “I/we/you/they will calculate,” depending on the situation. If you need to specify:

“I”: 私はアプリで料金を計算します。
“This app”: このアプリは料金を計算します。 (or with が: このアプリが…)