Sri Lanka was never part of any Indian union to begin with. Cultural influence is different to Political influence. Do not get confused. British understood this reality better and that is why they kept the both countries apart.
And also, legitimacy of a nation does not only come from language...