The theory behind these rules is that the lender had a right to receive a minimum amount of interest on the loan, and should be treated as is (1) the lender received that interest and (2) made a gift of the same amount to the borrower. You’re focusing on the consequences of part 2, the gift. It’s part 1 that creates the problem: the lender has to report imputed interest income.
If the loan is between natural persons (you aren’t borrowing from a business or a trust, for example) and the total amount borrowed is no more than $100,000, under a proposed reg, imputed interest is limited to the amount of the borrower’s investment income. The idea here is that the IRS doesn’t have to worry about these loans unless they’re being used to shift investment income from one person (usually someone in a high tax bracket) to someone else (usually in a lower one). So, if you and your spouse can get by with each borrowing no more than $100,000, and you have little or no investment income, you should be able to work this out.
The loan doesn’t have to be secured. There’s a requirement for a loan to be secured if you’re looking to claim an itemized deduction for interest paid on the loan, but that’s obviously not an issue here.