This month, Personal Capital, an online firm that looks at your investment holdings and recommends improvements, released a study based on more than 100,0000 of those portfolios. Its conclusion: you're paying a lot if you're using one of the larger brokerages to manage your wealth.

The average annual total fees you'll pay at one of these brokerages, according to Personal Capital data, ranges from 1.06 percent annually at USAA, to nearly 2 percent annually at Merrill Lynch. And while that may not sound significant, over time the fees accumulate. For an account of $500,000, for example, even the least expensive of these brokerages will cost an investor nearly half a million dollars over a 30-year investing career—as much as the initial investment.

Man on plane with laptop. Adviser fees are costly, according to a new study.

The bulk of these fees are in the form of advisory fees, which range from 0.82 percent to 1.53 percent. Then, on top of that, are the fees you'll pay for the funds that these hypothetical investors own. Scottrade investors were found to own the least costly funds at 0.17 percent. But Merrill Lynch investors were paying four times that, an average expense ratio of 0.68 percent. 

The conclusion may feel familiar. Two years ago, PBS television's investigative news program, Frontline, reported that high fees and higher-cost mutual funds can cost retirement savers more than $155,000 over a lifetime.