Individual cities bid for the Olympics. Some richer countries have hosted multiple Games, a few cities have even done so (London, Los Angeles, even Lake Placid)
These days, there is much competition, which is why the IOC is working to spread the Games to various countries and continents. A new city will have a better chance (all other factors being equal). For example, if an African country make a reasonable bid, they would be more likely to win that say a US city or some place in Europe.
The process to get the Olympics is more complicated than just going to some random country every now and again. It's a bidding process, where various cities request the right to host the Olympic games. Then International Olympics Committee (122 representatives in the article below) goes through a process to select where the next Olympics will be held.
It's very competitive to win the bid to host the Olympics. The reason is because the winning city wins about 1.2 billion dollars in funds from tourism, advertisement, and rights to sell airtime for the Olympics to TV stations.
To my knowledge, there is no rules that it must go to every country in the world in a given millennium or anything like that.
There is no set schedule as to when they go where
Countries bid on the Olympics and make presents to the Olympic Committee, who then chooses the country they see as the best fit
The 2014 Winter Olympics will be in Sochi, in Russia and Rio De Janiero will hoist the 2016 summer games
Beyond that, future sites have not been decided