Yes, that is a drawback.
With loot threshold your options are Green, Blue or Purple. Common, Rare or Epic.

If you want cloth as well (white quality) then you need to manually check every corpse.
If you'll be vendoring crap anyway, then loot it all.

If you are picking and choosing what to loot, the auto loot within interface options is a good option.
You can set Alt (or Control or Shift) as the loot key.
You can auto loot everything only when the key is pressed or only when the key is not pressed.
Generally I have auto loot everything unless I hold Shift while looting a corpse.
If I run Deadmines for Linen/Wool, I reverse that to only auto loot with Shift, so I don't pick up the 2s selling gray quality items.