You are right. This is the decision of the custodian, and the IRS has no rule barring savings accounts investments in an IRA. Many institutions indeed offer such accounts.
As for the choice custodians make, savings accounts are historically high frequency contribution and withdrawal accounts for small amounts, neither of which occurs to that frequency for an IRA, particularly the withdrawals. Offering savings accounts would probably aggravate excess contributions and rollover issues with the one rollover limit. It might also cause more confusion for some people keeping track of whether their savings account was an IRA or not. There are probably other factors as well, such as processing platform limits, etc.