Update: I bought Compressor just so I could see if it would help and what's happening.
The issue seems to be with exporting the AAC codec. If I change the audio codec from AAC to linear PCM, then it stays synced and the audio does not shift earlier by 2 frames. Even though the source is AAC.
Does that make sense to anyone? Exporting using AAC as the audio codec causes it to be out of sync a little. (Even though the source file is AAC)