Guitar Amp making humming/buzzing sounds with distortion turned up?

I just bought a les paul special edition II electric guitar and paired it with my small 10 watt or so guitar amp. Everything sounds nice but when I turn up my distortion even just a teeny bit theres this humming/buzzing sound coming from the amp. the more i turn up the distortion,the louder it gets.when i took the guitar out the box I noticed the cable i'm using that came with the guitar goes into the guitar all the way but its a little loose and it pops out maybe just a quarter inch maybe even less. I tried a different cable from my old guitar and it fits better but still pops out a little. the cable that came with the guitar and the cable from my old guitar fit in the amp perfectly. Is this my amp or is this my guitar?