The cheapest way to collect stamps is to soak them off envelopes. You can also buy packets of new or used stamps. Recent stamp issues can be bought directly from the post office. Most stamps can be ordered through the mail or bought from dealers.