I've been taking pictures as a hobby for a number of years. In the beginning I was completely clueless. I had no idea about composition, I did no post-processing, I knew very little about optics(1) and even less about image sensors or even digital imaging and the issues that come with the territory. I shot pictures of people, places and things that I thought I might want to remember some day.
Somewhere along the way I started developing more of an interest in understanding how you get better images. What worked for me was a) developing a better understanding of how the technology works, and b) looking at other people's pictures with an analytical mind.
I am not by any means what I would rate as a good photographer(2), but I am eager to learn and when I look at pictures I've taken over the years I do see that I am getting better. And that is good enough for me.
There are a lot of hotshot photographers that will turn up their noses at gadget geekery and say that "it is not about the gear". Sure, there is some truth to that (though it sounds bit hollow when coming from people who have $5000 camera bodies with a $20.000 assortment of lenses to go with it). There are people who make great shots with their mobile phones. But that still doesn't mean that actually knowing how stuff works isn't a prerequisite to evolving as a photographer. You need to know some basics and I think it is hard to get a good grasp of the basics quickly without at least a good SLR.
If you shoot an SLR in manual mode and you pay attention to how the different parameters influence the end result, you will eventually develop some intuition. It will also force you to be a bit more analytical when taking pictures. There are compact cameras that have manual modes (such as the Canon G11 and its predecessors), but the optics and sensors on these cameras do not produce as obvious results. They are okay cameras, but I don't consider them very good learning tools. In part because these cameras are aimed at people who have no clue and thus come with a whole battery of features to produce the best possible pictures with the least amount of manual intervention. (Same goes for modern SLRs. If you want to learn, turn off auto-ISO, matrix metering, fancy focus modes and all that. Use manual (M) mode as much as possible. In fact, you are probably not going to use any of that most of the time once you've gotten a good grasp of the basics).
So if you have a tight budget, get a used SLR or an entry level model. If you have to choose, get a 18-70mm zoom lens. A lot of people recommend only having a 50mm, but if your goal is to learn, the 18-70mm or similar is more important. (Your next lens should probably be a wide aperture 50mm lens though. By the time you are getting ready to buy one you'll know why).
Develop a vocabulary.
As for looking at other people's pictures to learn, it does help to have a vocabulary. There is a number of tutorials, articles and blogs that explain, for instance, compositional rules. If you know about the rule of thirds it helps when looking at other people's pictures in an analytical way. Having a vocabulary helps you see how different techniques have been used to achieve somthing that works visually -- or doesn't. If you can describe what is going on in a picture it'll be easier to learn from it.
There's also a whole vocabulary to describe the technical aspects. You need to understand concepts like shutter speed, aperture, focal length, depth of field, dynamic range etc.
Exposure and inspiration.
A great way to get exposed to a lot of photos, and to expose yourself (pun thoroughly intended) is to join a social website that centers around photos. Flickr has been a very useful place for me both to publish my own photos and as a source of inspiration.
There are some groups on flickr where you can enter your photos to have them criticized by other people. This can be a good learning experience, but it can also be a bit intimidating. Just keep in mind that most people are just trying to be helpful, so don't get put off by the fact that most of the feedback you'll get will be of the form "nice, but...". People usually focus on what they can say that is helpful to you in terms of how they think you can improve your photography. Don't take it the wrong way.
Of course, a lot of people won't share your taste, or indeed, have any. Before taking any criticism to heart, at least have a look at their photos. You'll soon notice that sometimes the worst nit-pickers are people who tend to take supremely dull photos. (Don't insult their photos in return. Just do better).
Post-processing or not?
I post-process my photos. I consider people who are militantly opposed to even adjusting brightness or contrast in their photos to be odd.
Post-processing has been part of photography from the very beginning, the main difference is that the means by which photos are post-processed. In fact, in the early days there were people arguing that photography could not be art if it were mere depictions of reality -- unaltered by artful intent.
I think the idea that you shouldn't post-process is a combination of decades of amateurs who never had access to the right equipment, historical ignorance, and the increasing trend of "photographic dishonesty" that has given image processing a bad reputation. I am not arguing that you have to post-process your photos, I am merely saying that the obsessive-compulsive people who view post-processing as "impure" do not represent any sort of norm. You should at least learn what is possible and how to do basic post-processing before you dismiss it.
Of course, there are degrees of post-processing. On one hand you have people who twiddle the colors and contrast to get that National Geographic look, and on the other hand you have a whole industry of people who turn healthy looking people into emaciated plastic dolls(3). And somewhere in the middle of that you have press photographers who take liberties in ... interpreting the truth.
Much of the time, I shoot pictures in a way that both necessitates and anticipates post-processing. Sometimes the light can be tricky, so what I worry about is that my shot captures the raw data I need to later achieve the end result I desire. Knowing a bit about how digital imaging works and what is possible in your post processing tools can help when you have to deal with the rather limited sensory capabilities of a camera.
Lastly, be patient. I've been somewhat serious about learning photography for the past 5-6 years and I've seen a steady improvement. I am still in the process of learning and figuring things out, and the photo-related sites, blogs, podcasts I've been exposed to over the past few years have really been a catalyst.
1) Well, a physics course covering optics certainly gave me a theoretical basis, but brilliant though my physics teachers were, none of them ever ventured into the practical beyond splitting light into simple spectra and calculating refractive indices of various transparent materials. None of which will provide you with any intuition as to why a 50mm lens is such a favorite among most photographers.
2) No this isn't false modesty. I am fully aware of the fact that I am a better photographer than a quite a few professionals, however that doesn't really mean I am a good photographer -- it just means that there are professional photographers who aren't really all that good. Also, with the digital revolution in photography, there are now more excellent amateur photographers in the true sense of the word "amateur" than ever before. Every day I see pictures by amateurs on flickr that are way better than what most professionals produce.
3) I've watched Top Model a few times in the naïve hope that the show would provide even the tiniest fragment of useful knowledge on how to take pictures of people. The only thing I've deduced from the show is that the fashion industry is a perverse universe where expensive cameras are pointed at braindead morons and that the output is then de-humanized and sterilized by yet more morons and the end result is then evaluated by really creepy people who make judges at cat shows seem like well-adjusted individuals. But hey, to each their own.