Setting aside legalistic considerations about federal vs. state power, the institution of slavery in the South itself had been drifting in an increasingly dystopian direction (even for slavery) throughout the 19th century. To my understanding, by 1860 most slaveowners' view of it had evolved from 'it's a necessary evil that should be abolished at some point in the future' to 'actually it's a positive good for the slaves and they should be grateful for it', mirroring the hardening in pro-slavery sentiment across the South just as abolitionism was becoming more popular in the North. Lighter-skinned ('high yellow') to outright seemingly white slaves were already being bought and sold,
per Harper's Weekly the Union freed several when they took New Orleans, and Southern thinkers of the late 1850s like James H. Hammond and George Fitzhugh were already calling the Northern lower class 'mudsills' (just as the slaves were the 'mudsill' on which their society & economy was built) & outright suggesting that perhaps poor Southern whites too should share in the 'benefits' of slavery.
Yes, that does sound completely insane and was hardly a mainstream opinion in 1859. But you could say the exact same thing about the idea of slavery-as-a-positive-good in 1800, which Thomas Jefferson (as a slaveholder who nevertheless was one of the subscribers to the 'slavery is a necessary evil' viewpoint, ended American participation in the overseas slave trade & backed the idea of gradual emancipation) would have found abhorrent, among others. Throw in the rising popularity of scientific racism and eugenics from the mid-19th century onward, as well as the aforementioned existence of obviously-white-skinned slaves, and is it really so inconceivable that slavocratic ideology (for that's what the defense of slavery had become by 1860, it wasn't just an economic argument but one about the defining cornerstones of Southern society and racial hierarchy) might trend in an 'expand the category of who's fit to be a slave' direction?
This isn't the Alternate History subforum so I'll avoid launching into a longwinded hypothetical (well, longer than this has already been). Suffice to say I think it's a good thing the Confederates lost in the end even if the Union weren't perfectly saintly themselves, better an expansion of federal power than the further entrenchment of the slavocratic ideology - and I certainly have little to no faith that the CS would have organically abolished slavery on its own had it managed to prevail in its mortal struggle over the Peculiar Institution (even if it 'only' maintains the status quo instead of jumping off the Fitzhughian deep end entirely), which doesn't seem any likelier to me than the USSR adopting anarcho-capitalism after winning the Russian Civil War.
Also it should also be noted that Southern aggression against the free states was a very real thing and much more concrete, not to mention typically much more violent, than anything coming from the Northern abolitionists toward the slave states in the lead-up to the civil war. The Fugitive Slave Act is the one that's most talked about, but how about Bleeding Kansas? I'm sure most if not all posters on this board have a dim view of Californians today who migrate to red states and bring their voting habits with them. Now imagine that, but the 'migrants' are basically bandits with Southern accents who don't even intend to live in the state they're moving to, just ride over the border every time there's an election to steal and crack the skulls of any abolitionist they can find while they're at it, and you'll have a good approximation of the 'Border Ruffians' from Missouri who made Kansas bleed. John Brown has a not-undeserved reputation as a radical and a murderer; but it should be noted that he didn't fire the first shots in Kansas, the slavers did, and he didn't start hacking pro-slavery settlers to bits until
after the side of slavery had sacked Lawrence, KS and specifically targeted abolitionist papers for destruction.