I'd say go for a camera of a famous brand name, good optical zoom options (I agree wholeheartedly: Digital zoom usually stinks to high heaven), standard AA or AAA batteries and SD-card, plus most accessories for the price.
I had to go buy a camera bag for my Konica Minolta Z6, and they're expensive in that size - annoying, so if you get a free camera bag, camera hand strap, extra card or something like that, it's good.
Try searching for offers - usually some sellers offer camera packages with accessories for a special price.
It's cheap to buy rechargeable AA or AAA batteries, go for the good, long lasting ones.
If the camera comes with a small memory card, buy a bigger one at once - you'll need it, once you get started.
Once you've chosen three or four models you like, read camera tests of the models - even though the cameras may look and have mostly the same functions, there can be differences.
I usually use
Steve's Digicams - Main Menu, as his reviews are well done and thought through, and they give you a good overview of what the various tested cameras can and won't do.