Barry Ritholtz had a hard time writing his first book, “Bailout Nation.”

Drafted in the midst of the 2008 financial crisis, the biggest challenge, he said, was that a different company “would blow up” every week.

It felt as if the writing “was never over,” said Ritholtz, the chairman and chief investment officer of Ritholtz Wealth Management, an investment advisory firm that manages more than $5 billion of assets.

By comparison, the new book was a “joy” to write, largely due to the benefit hindsight, said Ritholtz, who is also a prolific blogger and creator of the long-running finance podcast “Masters in Business.”

The book, “How Not to Invest: The Ideas, Numbers, and Behaviors That Destroy Wealth — And How to Avoid Them,” published March 18, is a history lesson of sorts.

More from Personal Finance:
Why the father of ‘life planning’ says you’re managing money the wrong way
It’s not always ‘a sexy thing’ to be a millionaire, former NFL linebacker says
Take a look inside a $1.1 million ‘zero emissions’ home

Ritholtz looks back at anecdotes across pop culture and finance — touching on Hollywood titans like Steven Spielberg, music sensations like The Beatles, and corporate pariahs like Elizabeth Holmes of Theranos — to illustrate the disconnect between how much people think they know and what they actually know. (Ritholtz’ point being, The Beatles and films like “Raiders of the Lost Ark” were initially panned; Holmes, initially lauded, is now serving jail time.)

“It’s a huge advantage to say, ‘I know how the game ended,'” Ritholtz said. “What the analysts were saying in the second, third, fourth inning, they didn’t know what they’re talking about.”

CNBC spoke to Ritholtz about why people are often bad investors, why famous investors like Warren Buffett are “mutants,” and why financial advice about buying $5 lattes is the cliché that just won’t die.

This interview has been edited and condensed for clarity.

GI: There’s this great part in the book where you talk about the $5 coffee: The thought being, if you invest that money instead of buying coffee, you’ll basically be a millionaire. You write that it’s the “cliché that refuses to die.” Why do you think it’s detrimental for people to think this way?

BR: $5, really? I don’t want to come across as a completely detached one percenter, but if a $5 latte is the difference between you having a comfortable retirement or not, you’ve done something very, very wrong.

Let’s say you do put $5 away. If you saved $5 every day and invested it, it adds up to something. But when you look out 20, 30, 40, years, the other side of the spending equation is, what’s my income going to be? How much am I going to earn? If you’re going to show me $5 compounding over 30 years, you also have to show me where my income is going to be. If I’m looking at this as a 30-year-old, what’s my income going to be at 60? How will my portfolio, my 401(k) — and if I have kids, my 529 [college savings] plan — how will that have compounded over the same time? If you’re only looking at the $5 latte but ignoring everything else — and that’s before we even get to inflation — it looks like a chunk of money but it really isn’t.

The big philosophical problem that I’ve found is most of the spending scolds don’t understand what the purpose of money is.

GI: What is the purpose of money?

BR: Money is a tool. First, lack of money certainly creates stress. You can worry about paying the bills, and if you have a kid, how am I going to pay for their health care? Not having sufficient money to pay the rent, buy food, pay for health care, is certainly stressful. The first thing money does is it chases away the lack-of-money blues.

Money [also] creates optionality. It gives you choices. It gives you freedom. It allows you to not do many of the things you don’t want to do. And it allows you to buy time with friends and family experiences and to create memories.

It’s the ability to spend your time how you want, with who you want, doing whatever work you want, or no work at all, if you eventually get to that point.

GI: What should people do to make investing as simple as possible and have good outcomes?

BR: [Vanguard Group founder] Jack Bogle figured this out 50 years ago. If you want to find the needle in the haystack — if you want to find the Apples, Amazons, Microsofts, Nvidias, J.P. Morgans, United Healthcares and Berkshires [of the world] — don’t look for the needle in the haystack. Just buy the whole haystack. (Editor’s note: The “haystack” here refers to buying an index fund that tracks the broad stock market rather than trying to pick winners.)

You make the core part of your portfolio a broad index, and then you put whatever you want around it.

So, start out with a basic index, be very tax-aware of what you do, and then back to the behavioral stuff: Don’t interfere with the market’s ability to compound.

The crazy thing about Warren Buffett: His wealth has doubled over the past seven years. Think about how insane that is. He’s 94, like half of his wealth came about from zero to [his late eighties], and the other half came about in the last seven years. That’s the miracle of compounding.