A three-pound difference in "total bike weight" is not a big deal.
Taking "three pounds" out of the revolving wheel weight by reducing the rim/tire/tube weight is HUGE and will result in a measurable performance improvement. In the 1970's I rode a 23" Lime Green Varsity back and forth to work after I crashed my Paramount. I simply removed the Paramount wheels and placed them on the Varsity, no other changes, "It was a nice riding bike".
All weight on a bicycle is not the same. If you empty both of your water bottles, you will not be able to tell the difference in ride. But taking that same excess weight out by removing your thorn-resistant tubes and replacing them with thin performance inner tubes is very noticeable. Upgrading wheels and tires offers a good for the money spent.
Having sold, owned, and ridden Carbon Kestrels I believe the advantage to using carbon is that you (the engineer's) can tailor the ride to the rider. It can be made to ride soft and flexible for a century rider or ridged for the sprinter. I do not believe that steel bikes will ever be able to match the aero advantages of the new wind tunnel designed carbon bikes. Most of the advantages are for the hardcore competitors, not the weekend warriors that have the money to buy, and the "improved performance" dreams.
John