Should I pay off my mortgage?

Should I pay off my mortgage?

One of the most common questions we get, especially as people near retirement is, “Should I pay off my mortgage?”

From a purely conceptual standpoint it’s simple. If you can earn more on your money than you pay by borrowing, your net worth is better by NOT paying off the mortgage. The odds of this working are improved if your mortgage interest rate is low or your investments are structured and managed properly.

The cost of a mortgage

Although the concept regarding paying off your mortgage may be simple, the math is more complex.

We’ll start by determining how much interest you pay when you hold a mortgage. The starting point is, of course, the stated interest rate on your loan. Most mortgages have a fixed rate, but some are variable. In the case of a variable rate, the return required on your investments to come out ahead changes over time. For the duration of this discussion, we will assume a fixed rate mortgage.

The basic formula for your cost of borrowing is: interest rate plus private mortgage insurance (PMI) less tax savings.

PMI is required to protect mortgage lenders. It usually only applies if the home was purchased with a down payment of less than 20% and the current equity in the home is below a certain level. Many people who consider paying off a mortgage have enough equity to avoid PMI.

Mortgage interest is deductible but as an itemized expense on Schedule A of Form 1040. With recent changes to the tax code, most taxpayers no longer itemize, so those homeowners do not get tax savings from their mortgage. One’s ability to itemize deductions can change from year to year, complicating the evaluation somewhat.

Let’s consider a couple who owns a $400,000 residence with $100,000 left on a 4% mortgage scheduled to be paid off in ten years. The equity is high enough that PMI would not apply. Further, with a mortgage that size, they would need to be making thousands of dollars in charitable donations or incurring thousands in unreimbursed health care expenses in order to itemize deductions. Therefore, the cost of borrowing is likely 4%.

Investment considerations

If your investments make 4% after-taxes, the effect on net worth is the same as if you had paid off a mortgage charging 4%. Put another way, the effective rate of return for paying off the mortgage is 4%.

Can our hypothetical homeowner make more than 4% on $100,000 somewhere else? Maybe. That depends on where those investments are made and how they perform.

The first consideration is the type of account in which the $100,000 resides. If the funds would be coming from an IRA or retirement account, the tax bill can swamp the interest savings.  A $100,000 loan at 4% for 10 years requires annual payments of roughly $12,330. A total of $23,330 of interest is paid spread over 10 years.

In order to net $100,000, an IRA withdrawal will incur an upfront tax cost greater than $23,330 if the taxpayer is subject to a marginal tax rate of about 19% or more. In 2020, the marginal rate jumps to 22% at a mere $40,125 of taxable income to a single filer or $80,250 for a married couple filing a joint return. It is usually much better if the money to pay off the mortgage comes from non-retirement accounts.

It is usually much better if the money to pay off the mortgage comes from non-retirement accounts.

The next consideration is the type of investments in which the $100,000 is placed. If the funds are placed in a bank account, the interest received will fall far short of the 4% threshold. There are simply no safe, stable instruments that will net 4% after taxes.

If the $100,000 is in a diversified array of stocks, there is a good chance that those funds will net more than 4% over the next ten years, but it is not guaranteed and there is no chance the growth will come steadily. Stocks and other investments with growth potential are simply not stable in the short term. There have been ten-year periods in which the S&P 500 index of the stocks of large U.S. companies lost money.

A balanced approach is usually a better choice. For instance, a portfolio split 50/50 between stocks and bonds returned an average of 5% or more in 93% of 10-year periods since 1926[i]. The worst 10-year period returned 3.55%, which is close to mortgage rates available today for borrowers with good credit ratings. No one knows what a balanced portfolio will return over the next ten years, but that record is encouraging.

Cash flow considerations

First-time homebuyers often find that the monthly mortgage payment consumes a significant percentage of their income. This level of expense often makes the idea of having no mortgage payment appealing and for many, it becomes a goal.

Our hypothetical homeowner, however, has only ten years left on their mortgage. If they originally had a 30-year loan, their $12,330 of payments feels more like $6,826 today, assuming a 3% inflation rate. As a result, how one values the effect on their cash flow is influenced by each household’s situation.

with many more years to go on their mortgage, there are often far better places for them to direct their cash flow given today’s historically low mortgage rates.

If we change the homeowner to one with many more years to go on their mortgage, there are often far better places for them to direct their cash flow given today’s historically low mortgage rates. Most families like this would be better off keeping their mortgage and doing things like paying down higher interest rate debts, building a cash reserve, adding to tax deductible retirement plans (especially if their company matches contributions), contributing to a Health Savings Account, saving for college, or spending money on experiences that enrich their lives.

Liquidity considerations

What if our hypothetical homeowner only had $100,000 to his name? Paying off the mortgage would mean he would have no liquid reserves and no potential for investment income or significant gains. Real estate does tend to appreciate over time, but usually nowhere near as well as a sound portfolio.

When additional cash is needed, he can’t sell a window. He either sells the whole house, rents out part of the home or borrows the money. Most people who pay off a mortgage want to stay in their home. The potential problems with renting are too numerous to go into here. His situation is probably not attractive to lenders. If he could get a loan, it would probably not be at a good rate.

A common but less extreme example of paying off a mortgage and causing a liquidity issue is the homeowner who pays off the mortgage with non-retirement money and is left with the rest of his investments in IRAs or retirement accounts.  He may not have to borrow to get it, but extra cash might be expensive.

A taxpayer in the 22% bracket ($40,125 of taxable income to a single filer or $80,250 for married filing a joint return) needs to distribute $1,282.05 to net $1,000. Basically, everything costs 28.2% more than the price tag. The higher the bracket, the worse this markup becomes.

This tax cost can cause people to spend through their savings at a quicker pace. Similar to taking a lump sum from an IRA to pay off the mortgage, the tax cost can be so high that paying off the mortgage is a bad move even though interest expense is saved.

Intangible considerations

One of the most common reasons to pay off a mortgage, even if the interest rate is low, is the value many people find in owning their home free and clear. For many, it brings great peace of mind, reduces stress, and feels like a safety blanket. This value is hard to quantify, but it is real.

On the other hand, intangible factors such as wanting easy access to assets lead many people to keep their mortgages.  We see this often with clients who contribute to charity regularly, travel a lot, are paying for college or are expecting a significant change to their employment or living arrangements.

People who can maintain a sound, tax-efficient, and moderate-to-aggressive investment portfolio over several years have excellent odds of coming out ahead. This is especially true if they have a low interest rate mortgage loan or can refinance into one.

Like so many of life’s big decisions, the math isn’t the only factor when deciding whether to pay off your mortgage. Your family’s circumstances and preferences can heavily influence the choice. We are here to help you assess the tradeoffs and serve as “A sanctuary from the noise®”.

[i] 50% CRSP 1-10 index, 50% 5-year U.S. Treasuries securities, rebalanced annually, rolling calendar year periods. See additional disclosures for other important details