I think my favorite times to visit the Mediterranean are the end of May/the beginning of June and September. As these periods are outside the school holidays, there are a lot less people everywhere. A beach to yourself is not difficult to find. Historical sites are not yet loaded with dozens of tourist buses. One can find a luxury hotel for two or three times cheaper than during the peak season. In May there is this hope about a successful summer, so the locals are very friendly and have patience and energy to chit chat and to serve. At this time of the year flowers are blossoming and mountains are covered with dense vegetation. The sun is already strong but in evenings the temperature drops to more bearable numbers.
All this applies also to Crete, where we have now been traveling for one week. We have almost constantly had 30C (even 32C). It has rained once, bringing the temperature down to 20C. We have swum in the sea and the sea water must be around 22C. In September the sea is definitely warmer, but 22C is good enough for us, especially since Crete has so much to offer and staying on the beach was not our main and only goal. There is such a density of archeological sites to see in Crete that it would be a pity to just lie on the sand (I think).