Your out of pocket at closing can genuinely be zero when the seller's side covers the full fee. That's the cash flow answer, and it's the reason sellers keep paying it. Buyers are already stretched by down payment and closing costs, so a fee they'd have to fund separately is money most of them don't have.
What your lender means is different. A seller who expects to pay 5% total commission sets an asking price with that in mind, so the fee shows up in what you finance rather than in a check you write. On a $300,000 purchase, 2.5% is $7,500, and if that's in the price, you're borrowing it and paying interest on it for as long as you hold the loan. That's why the number matters even when you never see it leave your account.
The practical version of this after the August 17, 2024 rule change is that the fee has become something you can put on the table. If you're negotiating price, the buyer-side compensation and the price are two dials on the same machine. An agent who agrees to a flat $6,000 instead of a percentage gives you room, and some agents will do reduced rates when you're finding properties yourself.
The piece that catches people is the shortfall. If your agreement says 2.5% and the seller's side offers 2%, you owe the difference in cash unless your agreement caps your obligation at whatever the seller pays. Read that clause before you sign, and ask for a dollar cap if the percentage makes you nervous.