There is a couple of nudist beaches near Middle Head that should suit your needs. There are some nice rocks there.
Now that the school hols are over, the beaches should have fewer people there.
Just drive along Military road to the end.
Directions
http://www.freebeach.com.au/NSW%20List.htm
Obelisk Beach - Legal
Obelisk Beach is on the southern side of the National Park area of Middle Head. To get to Obelisk Beach from the city cross over the Harbour Bridge and turn off toward Neutral Bay along Military Road (Route 14). Follow this road to Spit Junction. Continue straight ahead at Spit Junction through Mosman Junction and just after Raglan Street bear left into Middle Head Road. Follow this to the end where it turns into Chowder Bay Road. Take the first turn right. After about 100 metres park your car and follow the track in to this small beach.
Cobblers Beach - Legal
Cobblers Beach is located on the northern side of Middle Head. Follow the same directions as for Obelisk Beach but after parking, head north across Middle Head oval and follow the track to the beach. Cobblers is a lovely beach and attracts a large number of boats. It is a great place for snorkelling around the rocks.