I think it's better to think of it not as "military flying is supposed to be dangerous", and more that "when someone is shooting at you, being able to run away before they can shoot you is armor in itself". Making an aircraft safer and easier to fly, even before considering structural strength often involves sacrifices being made to performance. For example, strongm dihedral makes a plane very resistant to rolling, which makes it very stable… and also makes it very resistant to the pilot trying to start a rolling maneuver.
The tradeoff when trying to shave a few pounds is "how many pilots we risk dying in an accident who wouldn't have before, vs how many pilots survive the mission who wouldn't have before"