Yes, shifting that early puts you in a low powered car for a couple reasons; one, you're spending time in nuetral at a point when all the cars around you are staying in gear, and more importantly, you are spending lots of time at zero boost. This is great for gas mileage but terrible for power.
Each time you shift, you dump all your boost, so your engine goes back to being a 1.4 liter naturally aspirated anemic little thing with a max of about 69hp with the pedal to the floor. With time (and we are talking fractions of a second, but still noticeable ones) the boost builds back up. With air crammed into the cylinders at pressure, more fuel can also be sprayed in and still maintain the same fuel to air ratio. At, for example, 14.9 psi of boost which would mean two atmospheres, you are also running twice as much fuel, meaning you have the equivalent of a 2.8 liter engine, or about 138hp. (I fudged the numbers so they would match the specs for the car but be easy to understand)
Official specs show
138hp @ 4,900RPM
148 lb.-ft. @ 1,850RPM