You can't really go past Zumbo's Patisserie - now in The Star as well as Balmain and Manly. World beating macarons, cakes that will impress anyone you're visiting with their delicate flavour and a lot of choice. For sweet breads, muffins, flourless chocolate cake and croissants, Bourke St Bakery is fantastic. Black Star Pastry is also good, and if you're in the city and want something French, you can't really go past the vanilla slices etc at Becasse Bakery in Westfield Pitt St Mall.
For chocolate brownies in the city - Sweet Infinity at the Strand Arcade. That fudge like centre. Yum, and so filling.
Laduree is great for their macarons. I like Cest Bon in Rozelle, and Pattison's in Mosman.