If the power armor's armor can absorb 2500-3000 joules, then it's going to be all but invulnerable to intermediate calibers that all run well below 2000 joules even with hot loads. However, full 7.62x51mm NATO runs right around 3500 joules and should punch through even without weak spots and/or specialized AP rounds. WWII-vintage .30-06 is a little more powerful at 3600 joules, but modern .30-06 loadings run in the 4000 joule range.
In other words -- power armor is very, very dangerous due to the combination of protection, mobility, and firepower, but the average civilian hunting rifle will punch holes in it. As such, power armor infantry *shouldn't* be standing in the open and intentionally soaking up hits, but moving and firing from cover like "regular" infantry, using their superior mobility and firepower to prevent enemies from engaging accurately.