AI creative came out close enough to my manual ads to be annoying

Set the marketing question aside for a second. You ran ads showing a house that doesn't exist, to sellers, and two of them referenced it on a call. Depending on how the image was presented and what state you were advertising in, misleading advertising rules and in some states real estate advertising rules can reach that, and if you're doing this under a license the exposure is different again. Worth a real conversation with an attorney in your state before Group C, not after.

Separate from that, your Group B copy was written by a model optimizing for clicks. Of course it drew people with a different understanding of the offer. That's the mechanism, and it's independent of the image question.

The attribution problem you named is bigger than the creative question. With a 5 to 7 week lead to contract cycle and a 90 day test, roughly the last third of your spend can't have produced a contract yet. Group B may have contracts in flight you haven't counted. I wouldn't draw the honesty conclusion from 1 versus 0 when the window can't hold the cycle.

The two callers mentioning a house that doesn't exist would have ended it for me, no question. Not as a compliance thing, as a first impression thing. You get one.

You're comparing 6 creatives against 34 and calling it manual versus AI. It's also low variant count versus high variant count, and the platform's optimizer behaves differently when it has 34 things to test. Group B might be winning on cost per lead purely because the algorithm had more to work with, and you'd get the same effect from 34 manual creatives if you could make them.

That confound has to come out before Group C means anything. Run the next test with equal variant counts.

@windlass is right and it's the strongest point in the thread.

On the actual decision, I'd bid on cost per conversation. I know you don't want to rebuild tracking. Every time I've optimized on the cheap upstream metric I've ended up paying for volume that cost me labor downstream, which is the same shape as scope creep. You spend the savings answering calls.

The rebuild is a week of annoying work and then you have the number that matters for years.

How are you defining conversation to price? If it's your own judgment on a call, that's the metric you're about to bid a budget against and it isn't defined anywhere but your head. Write the definition down first, then rebuild tracking.

The two agencies pitching AI creative at a lower retainer are pricing on production cost, which is exactly the thing that got cheap. What didn't get cheap is knowing which offer to make and reading what the leads mean. If their pitch is mostly about volume of variants, you're paying a retainer for something you just did yourself in four hours.

On Group C, run it. Real photos with AI assisted copy and layout is the configuration most shops will land on anyway, and you'd rather find out on $2,100 of your own money. Match the variant count per @windlass and keep the price conversation definition @plumb asked for written down before you start.