Not necessarily. A typical spitzer-type rifle/MG bullet has its C of G quite far back, so is inherently unstable when flying point-first. But it is relatively stable when flying base-first. So if it is fired absolutely vertically, at the moment that it stops and begins to fall, it may continue to fall base-first, achieving a higher terminal velocity than if it was tumbling.
There is a data table in the NRA Firearms Fact Book (a useful compendium of miscellaneous information) which gives the terminal velocity of a vertically-fired 180 grain PSP .30 cal bullet depending on whether it falls point first (450 fps), base first (320 fps), or is tumbling (180 fps).