Elie occupies the eastern half of a mile long south-facing sandy bay framed at either end by rocky points. Since 1929 it has been formally joined with Earlsferry, which runs along the western half of the same bay. You can read the full text of the chapters about Elie and Earlsferry in D Hay Fleming's 1886 book: Guide to the East Neuk of Fife.

Of the two settlements, Earlsferry has the longer history, being made a Royal Burgh by Robert II in 1373. By then it had been an established ferry port for crossings to North Berwick for hundreds of years. It is said that MacDuff, the Earl of Fife, crossed from here in 1054 while fleeing from King Macbeth.

Earlsferry ceased to operate as a port after a serious storm in 1766. This drowned seven Earlsferry fishermen and completely filled the village harbour with blown sand. By this time Elie, which was much better protected from weather coming in from the east, had already become the more important of the two. The name is said by some to come from Eilean Ardross the original name of the island on which Elie's Harbour, now long connected to the mainland, stands. As the slight remains of the ancient Ardross Castle can be found just a mile along the coast, this theory is fairly convincing.

On the east edge of Elie is Elie House. Originally built on the site of an earlier house in 1697 for Sir William Anstruther, this saw a series of changes over the centuries. It was used as a convent during part of the 1900s before being placed on the market at the end of the century. It has since been restored and since 2006 has provided self catering accommodation.

In 1850 Elie's harbour was expanded, and a road was laid along the headland leading out to it from the village. Elie's fortunes improved further in 1863 with the arrival of the railway. The visitors brought by the trains were supplemented by those on board the regular steamers from North Berwick and Leith, and suddenly Elie and Earlsferry became desirable destinations for Victorian trippers. The railway, and the steamers, have all long gone, but the villages remain significant centres for yachting, and together serve as a local resort whose beaches are matched as attractions by the golf available on Earlsferry links. It was here that the famous golfer James Braid first played the game.

The twin settlements of Elie and Earlsferry remain extremely attractive and charming places to visit. The oldest buildings are found on South Street in Elie, whose south side backs onto the beach. Parallel to it and a little inland is the main street, operating under a number of different names including, from east to west, Elie High Street, Bank Street, Links Place, Liberty, and Earlsferry High Street.