I can't imagine such usage causing any serious harm, the gun is still going to recoil up or to the left (assuming a right hander).
If it did any damage firing it sideways, it was probably going to hurt real bad firing it normally anyways.
Guessing .500SW or .50AE?