When you reduce the weight of the bike and rider (either one), you cause the force requirement to get them moving (object at rest tends to stay at rest) to drop. A lighter bike is not necessarily a "faster" bike because speed is largely the product of the force of the rider on the pedals. A strong rider on a 75 lb ballooner can outride a very poor rider on a 15 pound road bike. What reducing weight gives you is more excess force beyond what it takes to begin moving the bike and hence more speed for the same amount of force. If other factors are working against you (bad bearings, rubbing brake shoes, etc.) then whatever you've done to reduce weight can be offset by mechanical issues holding you back.
The first place to reduce weight is at the rims/wheels. The wheels have a huge effect on how a bike handles and feels. From there you might move on to the cranks, fenders, handlebars/stem, pedals. By the end you're down to just an old frame with a bunch of modern parts.
Remember too that having good, properly maintained bearings and properly adjusted brakes will help through reduction of friction. This is one place that I definitely would check - you can continue to use original parts but help your cause by getting rid of badly worn or etched bearings, cups, and cones. You can adjust the brakes so they properly release. Go with a modern, slick lithium grease rather than the thick, old-type brown axle grease. Do the occasional clean-out and re-grease. All basic stuff, but maintenance can be looked at as cumulative.