A few uninformed guesses:
1. While parasite drag decreases with altitude, that assumes constant speed. If we assume that the racks represent the same proportion of parasite drag, we should expect the speed loss to be greater.
2. Level flight at higher altitude requires a higher angle of attack at a given speed, and going faster may not make up for this. The turbulence caused by the racks may change considerably with angle of attack (almost certainly), producing more drag. This could even reduce the lift under part of the wing enough to require a still higher angle of attack.