Updated 15 May 2008: Bust is probably not that of Caesar, see comments at the end of the post.
While I am abroad, CNN.com remains my primary source of news. This morning I saw an interesting headline: "Divers find marble bust of Caesar that may date to 64 B.C." Upon reading this article it is clear that "64 B.C.," which indeed would be a very early bust for Caesar, is meant to be "46 B.C." - just two years before his assassination on the Ides of March in 44 B.C. Apart from the coins, I believe virtually every known portrait of Caesar is dated posthumously - but I don't have my references on sculpture and portraiture at hand, so that statement needs to be checked.
The bust was recovered from the Rhône River by underwater archaeologists near the town of Arles, which was founded by Caesar in 46 B.C. At present French authorities seem to be dating the bust to the foundation in 46 B.C., making it the oldest known bust of Caesar, but it is unclear on what material evidence they are dating it to the foundation. Hopefully, future publication will clarify the context of the bust and provide more detailed information.
According to the CNN report, the bust was recovered along with a life-size statue of Neptune from the third century AD and two small bronze statues, one of which is described as "a satyr with his hands tied behind his back, 'doubtless' originat[ing] in Hellenic Greece," according the French Culture Ministry. Unfortunately, there are no photographs of the other finds at this time. I wonder if the bound satyr statue is a variant of one of many representations of the "Punishment of Marsyas" [link to Wikipedia article - 'buyer beware']. Sculptural representations of these mythological scenes were widely produced by Hellenistic sculptors, but also copied by the Romans. It is unclear to me whether the statement, attributed to the French Culture Ministry, implies the sculpture itself came from Greece or if the type simply comes from Hellenistic Greece.
Hopefully future publication will provide more information on these important finds. One important question archaeologists are trying to determine is the context in which the group of statues were put into the river.
(AP Photo from CNN article)