There are a few cool things. See, Sydney isn't an expensive city...
1. Vivid Sydney
2. Those free historical walking tours with Ross and Justine are awesome. Peek Tours also run similar free history walks with Gregg. Although free, they'll want a tip.
3. Darling Harbour's Month of Love (the free stuff there)
4. Walking in the city!
5. Street theatre (although they do want some dosh)
It's always free to take a walk around Centennial Park or Hyde Park, or simply take a drink and hang out and read a book. Otherwise if you have a weekly transport ticket to head to Leichhardt, Manly on the ferry, or to Balmain or Homebush Bay to see the sights in a more leisurely way. The walk all the way along the finger wharves, from Circular Qual to Woollomoloo is also stunning and many people don't do the whole think unless it's Vivid time.
I agree, surprisingly there's quite a bit that you can do for free! Try the Museum of Contemporary Art (some exhibits you have to pay for but there's lots to see for free) for some interesting and quirky exhibits. There's also the Art Gallery of New South Wales if you want to really be cultured!