It’s not nice to tell people “I told you so.” But if anybody has the right to say that, it’s Nina E. Olson, the national taxpayer advocate.

Olson recently submitted her annual report to Congress and top on her list of things that need to be fixed is the complexity of the tax code, which she calls the most serious problem facing taxpayers.

Let’s just look at the most recent evidence of complexity run amok. The Internal Revenue Service had to delay the tax-filing season so it could update forms and its programming to accommodate recent changes made under the American Taxpayer Relief Act. The IRS won’t start processing individual income tax returns until Jan. 30. Yet one thing remains unchanged – the April 15 tax deadline.

The IRS said more extensive changes could result in some people not being able to file their returns until late February or into March.

Because of the new tax laws, the IRS also had to release updated income-tax withholding tables for 2013. These replace the tables issued Dec. 31.

As part of the recent “fiscal cliff” deal, the AMT is now fixed, a move the IRS was anticipating. It had already decided to program its systems on the assumption that an AMT patch would be passed, Olson said. Had the agency not taken the risk, the time it would have taken to update the systems “would have brought about the most chaotic filing season in memory,” she said in her report.

The tax code contains almost 4 million words. Since 2001, there have been about 4,680 changes, or an average of more than one change a day. What else troubles Olson (and most of us)? Here’s what:

• Nearly 60 percent of taxpayers hire paid preparers and another 30 percent rely on commercial software to prepare their returns.

• Many taxpayers don’t really know how their taxes are computed and what rate of tax they pay.

• The complex code makes tax fraud harder to detect.

• Because the code is so complicated, it creates an impression that many taxpayers are not paying their fair share. This reduces trust in the system and perhaps leads some people to cheat.

• In fiscal year 2012, the IRS received around 125 million calls. But the agency answered only about two out of three calls from people trying to reach a live person, and those taxpayers had to wait, on average, about 17 minutes to get through.

“I hope 2013 brings about fundamental tax simplification,” Olson pleads in her report. She urges Congress to reassess the need for tax breaks we know as income exclusions, exemptions, deductions and credits. It’s all these tax advantage breaks that complicate the code. If done right, and without reducing revenue, tax rates could be substantially lowered in exchange for ending tax breaks, she says.