I was surprised to learn that the first Olympics wasn't filmed (neither was the second). This is such a shame since film came in just in time for it to be possible, and especially since, sporting events were typical of early film subjects.
Here is Bioscope's very detailed comments about the various filmed records of Olympic Games.
Here is a link to the olympics.org site with stills from the 1896 games.
Below is a You Tube compilation showing stills from the 1896 game: