In 1860 the US Census said that roughly 39% of the South consisted of blacks who were slaves - or if free - basically had no rights.
The Abolitionists of the day didn't think "niggers" were anywhere close to being equal to "whitey". There's some really racist "send them all back to Africa" from...