A (very rough, but bear with me here..) rule of thumb I've seen is that for each step down in (the widely used) calibre, about 4-5x as many hits were needed to bring down a plane. For a single-engined plane, a single 30mm hit was usually enough to bring it down. Step down to 20mm, and you need about 4-5 hits. Step further down to HMG class guns, and you need a further 4-5x, that is, around 20 hits. Step down all the way to rifle caliber, and you need around a hundred hits. Bigger aircraft were obviously sturdier, I think the average was something like 4-5 30mm hits to bring down a 4 engine heavy bomber. Now these rules of thumb are AFAIU roughly based on averages of shot down planes, obviously you can find outliers in all directions.
With that in mind, for shooting down fighters anything above 30mm is probably a waste. Even 30mm is a bit on the heavy side, as a 30mm gun with decent muzzle velocity and rate of fire would be heavy enough to be a significant problem for installing on a WWII fighter. Stepping down to 20mm seems to have, historically, been a pretty good compromise, as can be seen by the late war fighters with 4x20mm armament. One problem with 20mm is that it's slightly marginal for shooting HE shells in that the ballistic coefficient tends to be slightly poor (particularly for mine shells), and an annoyingly large fraction of the internal volume will be taken up by the fuze rather than HE filler.
So perhaps an optimal choice would be something in the 25mm range? Such as the Madsen mentioned by marathag . Though muzzle velocity and RoF is somewhat worse than one would have liked.
(That is not to say the US choice to standardize on the Browning HMG was a poor one. There are certainly big logistical advantages to that, though the gun itself was maybe not that impressive in terms of punch per installed weight. But then again US fighters tended to be on the bigger side in general, so they had better ability to carry a heavy battery to compensate. And of course, the US had little need to shoot down heavy bombers, where a big autocannon would really shine.)
With that in mind, for shooting down fighters anything above 30mm is probably a waste. Even 30mm is a bit on the heavy side, as a 30mm gun with decent muzzle velocity and rate of fire would be heavy enough to be a significant problem for installing on a WWII fighter. Stepping down to 20mm seems to have, historically, been a pretty good compromise, as can be seen by the late war fighters with 4x20mm armament. One problem with 20mm is that it's slightly marginal for shooting HE shells in that the ballistic coefficient tends to be slightly poor (particularly for mine shells), and an annoyingly large fraction of the internal volume will be taken up by the fuze rather than HE filler.
So perhaps an optimal choice would be something in the 25mm range? Such as the Madsen mentioned by marathag . Though muzzle velocity and RoF is somewhat worse than one would have liked.
(That is not to say the US choice to standardize on the Browning HMG was a poor one. There are certainly big logistical advantages to that, though the gun itself was maybe not that impressive in terms of punch per installed weight. But then again US fighters tended to be on the bigger side in general, so they had better ability to carry a heavy battery to compensate. And of course, the US had little need to shoot down heavy bombers, where a big autocannon would really shine.)