Part of the problem was lack of numbers early in the war when a handful of Beauforts was it. Once the Beaufighter was available in quantity and the Strike Wings were formed it was a different matter. I recall an account of an attack on a couple of merchant ships with a large escort of other vessels. The attack force was some torpedo carriers plus twenty or more anti-flak aircraft. As the torpedo aircraft commenced their attack, the anti-flak aircraft, in line abreast, went for the escort ships and none of the torpedo droppers were even hit.
This shows part of an attack:
View attachment 830254