When swapping your loadout, you should hold down the crouch button. You'll be able to swap between the two even faster.