This jacket is our TIGHTEST fitting jacket, so I'm not surprised. In addition, this jacket will feel tight across the chest while you are standing around, and will only feel "perfect" once in a riding position (once you bring your shoulders down and arms forward, the chest will fit right in place).
One thing you might double-check is the arm-length. The XL is about 34" and the XXL is about 35".
If anything, you could buy the XL and XXL and return the one that doesn't fit.
hope that helps,
Robert, Gore Bike Wear