In most countries, it's perfectly legal to take pictures of people in public places. However, it's very important to be sensitive to their feelings. There are good guides to the dos and don'ts of "street photography" on the Web, for example a street photography article at Luminous Landscape by Michael Reichmann. I'm no guru at street photography, but I've stayed out of trouble so far by following these guidelines:
Don't sneak around
Make it obvious that you're taking pictures. Take your time, so that people have a chance of letting you know they don't like their photo taken, if that is the case. Don't go out of your way to ask permission, but if you meet their eyes, do so -- for example, point to your camera and make a questioning look, and acknowledge their response. (I.e., if they make a "no" gesture, nod and point your camera elsewhere. This rarely happens: I missed one photo for this reason on my December, 2002 trip to Lebanon, for example.)
Use a short lens (35 to 50 mm) and get pretty close to the action: using a long tele will make you look like a voyeur. Smaller cameras are less intimidating than big ones.
When people see you taking pictures, they wonder why you're doing it. For this purpose, it helps to give cues about it -- act out a role, if you will (see postscript).
People get tired of mugging or acting self-conscious around a camera after a few minutes. The ones who don't like it have let you know they don't want their picture taken or have left, and the rest will go about their business. You can go about yours quietly without bothering anyone.
Be a lady -- or take one along
A man all by himself shooting pictures of passers-by might look somewhat suspicious, but a man and a woman taking pictures is completely harmless.
If asked, shoot
Even if you don't want to, and even if you're shooting film or running out of CF: one "wasted" frame can make somebody happy, and bring gobs of positive karma your way, letting you do your stuff in peace.
Use your common sense
Think "Would I mind if that was me in the viewfinder?" Don't take pictures of sensitive subjects. I wouldn't photograph a smooching couple except if they expressly asked me to. I'd avoid shopping malls, schools, and governmental buildings, and I'd be very careful about photographing unknown kids -- I'd be extra certain about having their parents' permission before shooting any frames. As Michael Reichmann puts it:
A picture of a little girl sitting on her daddy's shoulders at the Santa Claus parade is one thing. A child alone in a public place is quite another. As commented on recently to me by street photographer John Brownlow, imagine what you would think if your little girl came home one day and said, "Daddy, a man took my picture in the park today." Needless to say, use mature judgment.
The Dumb Tourist
Props: Hawaii shirt, Panama hat, sunglasses (in summer), ridiculously big down-stuffed coat, silly wool cap, and ski glasses (in winter).
Location: Anywhere on the globe. Might not be the best role to pick e.g. in the Southern Suburbs of Beirut.
Mannerisms: Point at things, stare at them, take pictures all the time of everything around you.
Advantages: People will utterly ignore you, and you can get all the pictures you want.
Disadvantages: You might get mugged. People will try to sell you stuff, some of which may be unsavory.
The Press Photog On Assignment
Props: Scruffy clothes, unkempt appearance, three-day stubble (for men), bleary and bored look. Big camera bag. Keep your lunch in it if you don't have enough equipment.
Location: Anywhere on the globe.
Camera: Biggest, blackest, most expensive you can get your hands on. You're not supposed to own it.
Mannerisms: Look bored, sling around your expensive gear very casually as if it doesn't belong to you, shoot at least five-ten pictures a time.
Advantages: People will stay out of their way, if they don't want their face in the press.
Disadvantages: People will elbow themselves in front of you, if they want their face in the press.
The Hero Photojournalist
Props: Outdoorsy, worn but good-quality clothes, healthy tan, sweat. Think Indiana Jones with a camera.
Location: Rough areas.
Camera: Something pretty small and beat-up looking. Russian or Ukrainian cameras (Zenit, Kiev) for extra credibility. Vintage Leica or Voigtländer is excellent, too. Vintage manual-focus SLR with normal lens in a pinch.
Mannerisms: Examine everyone and everything intently. Concentrate on every individual photo. Take notes.
The Artist Pursuing Inspiration
Props: Black clothes, rings under eyes, long hair, intense look.
Camera: Anything black. Medium-format or vintage equipment for extra credit. Big is preferable to small, unless it's a Leica or a Voigtländer.
Location: Anywhere s/he won't get thrown out of.
Mannerisms: Stare enrapturedly at utterly commonplace things. Photograph things from odd angles (camera held overhead or tilted at a wild angle is good). Crawl under objects, climb on them, glower at people who get in your way, otherwise pretend they don't exist.
Advantages: You might make interesting friends.
Disadvantages: You might make interesting enemies.