Cap rate has a strict definition: net operating income divided by price. Net operating income is the rent you actually collect after vacancy, taxes, insurance, water, repairs, and management, before any mortgage payment. So on a $340k duplex, a real 6.8% cap means about $23,000 of NOI a year. The mortgage is deliberately left out, which is why the number says nothing about what lands in your pocket. That is your cash-on-cash return, and it depends entirely on your loan.
The loose usage is the one on your listing sheet. Plenty of brokers compute "cap" from gross rents minus taxes and insurance, sometimes using rents the current tenants aren't paying yet. Take that $23,000 and add back a 5% vacancy allowance, 8% management, and a repair reserve, and a 6.8% quickly becomes a 5.2%. Rebuild it yourself from the actual leases and the actual tax bill before you treat it as a number.
Small multifamily generally trades around 6 to 7% when the expenses are honestly counted, a little above single-family, because two rents produce more cash flow than one.
Your lender ignored it because a two-unit is valued off comparable two-unit sales, not off income the way a 20-unit is. Cap rate is your screening tool for whether the price makes sense to you. It won't move the appraisal. What can move your deal is the appraiser's opinion of market rent on the other unit, since some loan programs let a share of that rent count toward qualifying you.