Have you ever eaten an egg from a chicken that lives outside, eating bugs and worms and grubs instead of just chicken feed? You notice the difference as soon as you crack the shell--it's twice as strong as a regular grocery store egg. The difference is visible, as well--the yokes are firmer and stand up better. I won't even bother trying to compare the taste in words--I'll just say, find a farmer who sells eggs from grass-fed chickens and eat some yourself.
Don't settle for so-called "free-range" chickens. That's a term defined by the federal government to mean chickens raised in a pen that includes a door that's open to the outside. There's no rule that there be anything outside that door (such as grass, for example), and the rules allow the door to be kept closed when the chickens are small. The result is that most "free-range" chickens have never seen a blade of grass.
My wife and I get good, grass-fed eggs at the farmer's market. We don't get them very often, though, as they are not a frugal alternative. On the other hand, they're so much better tasting, it's hardly fair to consider them an alternative at all--they're effectively a completely different thing from grocery store eggs.
The excellent book The Omnivore's Dilemma first piqued my interest in eating real eggs. It's a book that deserves a full-blown review, which I'll try get done. In the meantime, let me just say that, if you care about what you eat and how it's grown or raised, you should read this book.
Keeping two or three chickens for eggs used to be a perfectly ordinary thing for people to do, even in cities. Modern lifestyles and local ordinances have made that more difficult (although raising chickens for eggs is still legal in many towns). More important, modern factory farming techniques drove the price of eggs so low, it didn't make much sense to go to all the trouble to keep chickens any more.
Recently, though, egg prices (like all commodity prices) have shot up. (Egg price data from the USDA.) Maybe keeping a couple of chickens makes sense again. If you let them scratch for worms and bugs in your yard (as well as feeding them chicken feed), you'll get much better-tasting eggs.
If that's not the choice for you, be sure to check out Myscha's Egg-cellent ideas for money saving and menu planning.