You can do it, and plenty of people do. What you'd be doing is called going unrepresented, meaning no licensed agent owes you a duty of loyalty in that deal. The listing agent has a signed contract with the seller and owes their loyalty there. If that same agent also formally represents you, that's dual agency, and it's handled very differently from state to state. Some states allow it with written consent from both sides, some allow a version where two agents in the same brokerage each represent one party, and a handful ban it outright. Whether it's even available to you is a state question, so ask the agent to show you in writing what your state permits.
The money part is where people get surprised. The seller already agreed a total fee with the listing brokerage before the house went up. If no buyer's agent turns up, that doesn't automatically refund anyone. The listing brokerage may simply keep the full amount. So the saving you're picturing only exists if the seller agrees to reduce the price or the fee, and you have to ask for that specifically, early, before you're emotionally attached to the house.
Since the August 2024 rule changes, an agent who's actually working with you as a buyer has to have a written representation agreement signed before touring. Walking into an open house or calling on a sign as an unrepresented buyer isn't the same thing. If someone hands you a form at the door, read what it says about exclusivity before you sign, because a one-page form can commit you to that broker for months.
