Answer :

I think it was because we really don't know what exactly happened in the past.
Also, I'm pretty sure people did not keep track of colored people back then. 
As there were countless african people brought into slavery so we only have a rough estimate of the people.