Bus: Greece has got a good net of buses, except in the south of Peloponnese. In the north coast of Crete (the most touristic), there are buses to each important town (Hania, Retimno, Iraklio, Agios Nikolaos, Sitia). You can have a bus every thirty minutes (every hour or less to others places, with only one bus per day to some places!). Warning: in Irakio, there are two main bus stations: the A near the port, and the B in the west of the town. At the first one, you'll find buses to the important towns of the north coast; at the second one, there are the buses to the towns of the south (Festos, Matala, Agia Galini, Mires...) and the airport.
At Santorini, Thira is the place where you'll find the buses for all the island.
Globaly, buses are cheap.
In every bus, there is a controller who sell tickets.
Webite of the greek buses compagny Ktel (with times, in english and greek)

Plane: From Athens, there are many flies to main towns and some islands: Corfou, Crete, Mykonos, Santorini, Paros... It goes quickler than boat (30 mns to go to Santorini from Athens, instead of about ten hours by boat), but of course, it's more expensive.
Website of the international airport of Athens (with times of the international and domestics flights, from and to Athens)

Car, motorbike: You can rent a car or a motorbike in many places. But be careful: the Greek people has a particular way of driving.
In general, if you rent a motorbike, you won't have a helmet : they don't have any...
Last but not least: Greece has the highest number of death on roads in Europe...

Boat: You can go to every Cycladic island from Athens, and some islands have boats between them. There are two kinds of boat: ferrys and jetfoils. Those last one go quickler than ferrys, but they only go if the sea is quiet. And in the Egean sea, the meltemi (summer wind from the north) often blows.
All the greek ferries (with times, in english)

Train: Don't look for train in islands, even in Crete: you won't find anyone. The main train station is in Athens. From there, you can go to the north (Livadia, Volos, Mont Olympe, Météores, Thessalonic...), and to Peloponnese (Corinthos, Nafplio, Argos, Tripolis, Patras, Olympia...). Trains are the cheapiest transports.
The greek railway network (prices and times, in english and greek)

Donkey: Maybe you'll get on a donkey! They are still used in the country and islands by shepherds. In some villages, if you don't want to go by walk, it's the only other way to go! But it became a commercial market, and in Santorini, at Thira, if you pay, you can go downstairs or upstairs getting on a donkey (by foot, going downstairs takes 20 mns, and going upstairs 40 mns).