Every apply_filters() call has overhead — even with no hooked callbacks. On a hot path running thousands of times per request, that adds up.
For inner-loop code, check has_filter() before calling and skip if no one is listening. Or batch the work: filter once on a collection rather than once per item.
Most filter overhead is invisible. Reach for these patterns only when profiling shows it matters.