You're putting the cart before the horse. The Vedas were written thousands of years ago, and that is when the word "Varna" was first used. Historical epics written by Kalidasa and even the Mahabharat and Ramayana generally depict the heroes as fair-skinned and the villains as dark-skinned (save for a few exceptions such as Krishna etc).
The word "varna" does mean "color", but it may or may not refer to actual skin pigmentation. The Mahabharata associates white with brahmins, red with kshatriyas, yellow with vaisyas, and black with shudras. Unless all the warriors of ancient India were Native Americans and the traders were East Asians, I don't see how such labels could refer to skin color. They seem instead to refer to some abstract, brahmanical notion of purity. It is important to remember that varnas are not ethnic or racial groups, or even endogamous occupational groups like jatis. They are simply labels given by brahamans to other people who they deem as ritually inferior. Several groups had their varna status raised or lowered through the course of history (for example, the Buddhists were often denigrated as Shudras, even though many Buddhists, including Siddhartha Gautama, were born as Kshatriyas or Brahmins). It is quite difficult to guess the ethnic or "racial" origins of a person based on his/her varna status alone, especially given how the status itself has fluctuated.
As for the attitudes of ancient Indians towards skin color, it is difficult to say exactly what they believed, but based on the surviving paintings and artistic depictions from ancient India, it doesn't seem like they particularly cared about skin color. The paintings at Ajanta, for example, depict most people as dark-skinned, including royalty and the upper class. A lot of dancing girls, apsaras (nymphs), princesses/maidens, etc. are depicted as dark-skinned. Even if there was some preference for fairer skin, I have not seen any good evidence that ancient Indians were as racist or obsessed with skin color as modern Indians are. On the contrary, a lot of the evidence seems to suggest that they were pretty laid back and didn't really give a shit.
Just a few pictures to show what I am talking about: