There's a really interesting discussion (via BookNinja) going on over at John Green's blog about advances vs. bigger royalties, and why the latter might be better as a whole for both authors and publishers. He also has a follow-on based on some feedback received.
I find it really interesting when compared to the video game (AKA my day job) industry's publisher/developer model (which is somewhat similar to the publisher/author model). The developers that don't survive are the ones that sign contracts with publishers that don't actually cover the development costs of a game, counting on a major hit's sales to earn out their advances and bring in royalties - a rarity in the industry. It's actually a very low margin industry since most of the costs are labour.
A smart developer will structure a contract such that the advance pays for the costs of development plus a minimum built-in margin (which covers R&D, a little profit, etc.), under the assumption that there will never be any royalties earned.
I'm wondering if the problem with author advances is that they are typically more random. Perhaps if advances were actually calculated based on an author's development costs (ie, some number calculated from time spent writing multiplied by some reasonable hourly rate), if a smarter royalty structure could then be created. This would normalize advances based on actual cost of labour, rather than the current structure which is based on some imaginary expected return pulled from thin air, or a good agent's negotiating tactics. With more normalized advances that take into account the true cost of creating a book, perhaps there'd be room for higher royalties in contracts.