I like the Manly area a lot. It's my favorite set of beaches, and is far more laid back than south of the bridge. There is a big national park at the southern tip for day hikes and photo opps over the harbour. The ferries run regularly and are a very pleasant way to commute
On the other side, at the far end of the eastern bays, there is a very good walk along the shore and cliffs starting (or ending) at Watson's Bay. Plus if you are into it there is the added bonus of nude sunbathing at Lady Bay (better leave the camera in the bag though!)
In the city, you can easily spend the better part of a day at the botanic gardens, which offer lots of different ecosystems and some gorgeous views over the harbour.
For a day out of the city, you could rent a car and drive up to the Blue Mountains, or southwest down the coast through Woollongong and over the hills all the way to Canberra, which is well worth a day trip if you don't mind 3+ hours one way through (mostly) lovely scenery. Or head north to the wine country of the Hunter Valley for a day. Just remember to drive on the left!
For more exotic pursuits, check out adrenalin.com.au. You can book a day driving European sportscars, head to the race track to drive V8 supercars, go hot air ballooning, etc.
I think of Sydney as primarily a night life destination, probably because I am there for business a lot and that's the only free time I have. If you are a jazz fan, The Basement often has top local and international players and an excellent dinner menu (which is the only way to get a table on the main floor). The Art House Hotel always has some interesting music going on if you are looking for something more adventurous. The State Theater usually has good touring acts too. Check www.jazzandbeyond.com.au/sydney_gig_guide to see what'e happening. If you are a foodie, Sydney restaurants can be world class. Just be prepared to pay world class prices too!
I avoid Potts Point, as there is very little of interest in the immediate vicinity (other than King's Cross, the red light district!) and it is a_long_ walk into the city. It's possible to find a few good values in the city center, and also around the Chinatown area. My go-to site for the best prices for accommodation in Sydney is wotif.com.au