I just filmed 3 year old nephew playing...
After about 4 goes he could do wheelies, boosting and accelerating.
JD definitely is too hard for very small children, but they can get some fun out of it. The ps3 controller would need to be smaller though, not that we should be encouraging them too early :)
Funnily enough, he was most vocal about the faces that appear when you do something good :)