Douglas Hahn, of Sink Review, found this interesting article on the perception variations of color between Babies and Adults.
While adults translate color information, for the most part, in the left hemisphere of the brain (right side vision), babies performed their tasks in their right hemisphere. The article notes:
Over the course of our lives, it appears that an unfiltered perception of color gives way to one mediated by the constructs of language.