Rough numbers first, then the licensing part, since that's where the surprise is.
A licensed residential loan servicer typically charges a setup or boarding fee per loan, often 50 to 150, then a monthly fee. Performing loans commonly run 20 to 40 a month. Non-performing and loans in active workout cost more, often 75 to 125 a month, because someone is making borrower contact attempts, sending required notices, and tracking a foreclosure referral. Add fees for specific events: a modification package, a payoff quote, a foreclosure referral, escrow analysis. Some servicers charge for advancing taxes and insurance on your behalf.
So one non-performing loan can carry 1,000 to 1,800 a year in servicing before any legal cost. Legal for a foreclosure is separate and varies enormously by state and by whether the borrower contests.
On doing it yourself: collecting on a defaulted residential mortgage is regulated activity, and several states require a license to service or even to acquire residential mortgage loans, with the specifics differing state by state. There are federal rules about notices, error resolution, and how you communicate with a borrower in default. Whether your particular setup needs a license is a question for an attorney in that state before you close. Most small note buyers use a licensed servicer because it's cheaper than getting the compliance wrong once.
The cost people forget is force-placed insurance. If the borrower's policy lapsed, you'll want coverage on collateral you may end up owning, and lender-placed policies on a vacant or poorly maintained house cost far more than a normal homeowner policy. Get a quote on a sample property before you build the model.