That's the iffy part.
Japan was involved with Berlin well before 1941. If Japan doesn't attack the U.S., then Germany doesn't declare war (yet) and so does the U.S. continue to remain neutral?
It's been said that Japan turned southward because of easier access to oil, but oil had been procured from Eastern Soviet Union through a joint venture earlier.
So if Japan is just focusing on east Asia and their co-prosparity sphere and not fighting the U.S. and Commonwealth, than the IJN and IJA van bring it's full weight to bear in the Soviet Union in concert with the Axis invasion in the West.
I understand that the Soviet Union got the upper hand against Japanese forces, but that was just the Kwantung Army and not the bulk of Japan's forces.
Plus, the Soviet's I-16 was more than a match for their KI-10 and KI-27. Once the KI-43 shows up, things would change in a hurry.