I do not think that would solve the problem in a good way. I think it would just look ridiculous if the weapon suddenly appeared or if the swap happened at 200% speed.
The only way I see fixing this would be to add animations for weapon transitions while moving.