My opinion on this is to save ‘feeling fast’ for races or competitive situations. For you, that seems to be Saturdays. There’s nothing wrong with that, in my opinion. Use whatever advantage you can.
In general, I believe in not being kitted out for speed unless you want to be fast. Training should be made hard and not using the fast kit unless the goal is to get comfortable with it. On the bike that may mean, for example, using Gatorskins or wearing ‘normal’ kit to not decrease drag. In the pool, by the way of example, it means wearing a drag suit for workouts and saving the jammers & speedsuits for workouts you really want to go fast and/or you really want to feel like you’re going fast (these could be certain workouts and some races).
My two cents.