You have a few options:
- From a local store (in my case, they are too far from my place)
- By salvaging old equipment (TVs, radio, etc...)
- From Ebay
- From small online stores
- From a large online electronic component distributor
There are a lot of specialized shops to buy components. Usually, the prices are a lot higher than the other places, and the choice of component is limited. They are useful when you need a component right away and do not want to wait several days for the parts to arrive.
Salvaging and scavenging
Sometimes, you can find some little gems in broken and old electronic equipment. It may be hard to remove the component without breaking it though. You might want to search for power supplies, toroids, LCD screen, resistors/capacitors, connectors, switches etc...
This one is my favorite! Components on Ebay are cheap... really cheap. Most of the time, you can get bulk parts for 1$, free shipping. I recommend Tayda2009, he got a lot of stuff at a bargain price. The only problem with Chinese and Thai sellers is that the shipping time is 2-3 weeks. For even better prices, you can visit his online store and use the voucher EB99922 for a 10% discount. ;)
Small online stores
These stores are handy for specialized parts like sensors, motors and various purposes boards. Shipping prices can sometimes be a problem; maybe not in the US but in Canada it can often be something like 30$. On Cyber Monday, shipping was free at Sparkfun so I bought some interesting stuff. Take a look at this resistor kit. I suggest you try them by yourself and pick your favorites.
Large online distributors
For a professional service, try some of the large electronic retailers. Their parts selection is HUGE and shipping time are blazing fast (2-3 days). They are often the only choice when looking for a particular component. One nice tip : try findchips website, you can enter a part number and you will get a list of all the distributors that stock the component you are looking for.
If you do buy some parts, leave a comment and tell me which method worked for you :)