I would first like to say that I am no expert at this sort of thing.
I think the way which it would be best to try and make the sound fx depends on what sort of sound effects you want. If you want distorted "retro" sound effects like those in cave story, it might be an idea to learn how to use some sort of synthesizer software. From here you can choose a wave form such as triangle or sine, and modify it with LFOs and the like. You can get some pretty diverse sound fx from experimenting with synthesizers!
A second option, which could be used, would be to use already existing VSTi instruments. The best example I can think of atm is the "Peach" and "Toad" VSTis, which are SNES instrument and drum emulators respectively. Especially the "Peach" VST, which I have used before, would be invaluable for making 8-bit sounding fx. There are probably many VSTs, which may be designed for making sound effects.
Finally, if you want "realistic" sounding effects, you could download or record actual sounds, then edit them with audacity. One website which is really good for this is freesound.org.
Just my 2 cents, good luck