Read this, then ask questions. There's always more to learn and we're here to help.
Too, I'll be adding to this thread as I can, tips on mic selection for certain jobs, placement, etc. I'm also going to grovel slightly right here for others to contribute their ideas (Buffalo, looking at you

Microphones
First and foremost in the recording chain is the microphone. It's the most important and where you should spend as much of your budget as possible while keeping the rest of some quality. Basic guitar/vocal recording can be done, and quite well for that matter, with just one microphone. There is a fundamental difference in the two main types of mics you'll see, those being dynamic mics, e.g. Shure SM57 and condenser mics, e.g. Neumann KM184.
Suffice it to say that they are very different, though aimed ostensibly at the same purpose--capturing noise. On a whole, for music recording, condenser mics are better. They are also generally more expensive. There are many great dynamics out there, but condensers are usually greatly more versatile and this is the direction in which I'd recommend you head.
Microphone Power
Buying a condenser microphone requires some way to power it. (This is the most quickly appreciable difference between condensers and dynamics, fyi.) The vast majority of true condenser mics require 48V Phantom Power.
For this, one has a tremendous number of options. Among a couple other methods, the main two approaches to powering mics are microphone preamps and dedicated phantom power supply boxes.
Preamps are almost always the better choice, as they give you gain (volume, basically) options (among a great deal else), which you absolutely need, always.
The dedicated supply boxes will be cheaper, but will leave you with only the strength of signal you can provide while playing/singing, which (again, among a lot else) will be too quiet. Way too quiet.
Analog/Digital (A/D) Conversion
Since I highly doubt anyone here is recording to cassette and then letting all those pile up in the corner, there always is some sort of A/D conversion. Microphone signals always start as analog signals, so they have to get digital somewhere in the chain to be usable with a computer.
A lot of people will let their soundcard take care of it. Just plug some stuff into the 1/8" inputs and it will do it. It won't necessarily do it well, but you can get a .WAV file out of it and be alright. There are much, much better ways to do it.
Probably the most common way these days, especially for someone in your situation, Ian, is the USB or Firewire recording interface, e.g. Digidesign MBOX.
The great thing about these is that you get a (usually) fairly solid preamp and A/D in the same little box.
There are also tons of dedicated A/Ds that one would pair with a dedicated mic pre. It starts to get (even) more expensive in this realm. But the price often yields better results.
There is also the fairly new set of USB microphones that have hit the market. I have not used these, but am hesitant to because of the application of only digital gain, as the mic plugs directly into the computer. Easy? Sure. Questionable? Definitely.
Back-end/recorder/software
I'm lumping all this in because of the audience I'm talking to at this site. Most of us are using our computer as a back-end to the recording rig, so I'm bypassing the discussion of portable digital recorders unless someone asks me about it. (I do know a lot here; I do a good bit of location recording.)
Using one's computer is great. It's opened up recording so much for the little guy just wanting to lay some tracks down. You'll often see a recording computer called a Digital Audio Workstation (DAW) in industry publication and product copy.
I'm sure most of us here have a computer totally capable of handling the bits we throw at it during recording. So it comes down to software.
Just like mics, pres, A/Ds, cables, et cetera ad nauseam, there are myriad recording/editing software packages out there.
One of the most popular for our general purposes (and quite capable), Audacity, is totally free. If you have a Mac, you get GarageBand for free. (I personally hate this piece of software and wish it would go away forever.) The industry standard is ProTools. It's heavy, expensive and has a steep learning curve. But it does everything quite well. Several people on this board (including me) are using Adobe Audition (or its former iteration called Cool Edit Pro).
Your software choice is a question of budget and capability. If you buy one of those USB/Firewire boxes, it may well come with its own software (which is probably a "lite" version of a mainstream software package). Issues arise when trying to cross software platforms with unsupported peripheral devices. These problems are usually solved by just sticking with the softwares you know to be compatible with your gear, if you're going into the computer via USB, etc.
Personally, I have an interface that I could use with USB. But I choose to run it into my computer via digital coax cable and a digitally-capable sound card (M-Audio Audiophile 24/96). This way, I have external digital conversion, but never have to deal with USB support issues, etc. as I'm connected through the sound card that has its own set of drivers.
In this fashion, recording software functions essentially as a bit-collection tool, since everything is in the digital realm.
_______________________________________________________________________
A side-note
When asking for gear advice, have a budget in mind. In the exact same amount of time, I could recommend three different rigs costing $200, $1000, $5000.
Now get to it and happy recording!