I second the De Young museum and SFMOMA, although they're not entirely "dorky," per se. In a more scientific line would be the Academy of Sciences and the Exploratorium, though it's been a few years since I've been in the city so I can't vouch for what they're exhibiting these days.
If you go to Pier 39, it is indeed touristy, but enjoyable... If you like Clam Chowder, I'd recommend Chowder's, some of the best breadbowls to be had in the city. I don't know if the season's right for it, but there's a great little fresh-made donut stand down by the farmers market, too-- if you should happen to run across a randomly cold day (not uncommon in summer), a bag of fresh hot cinnamon donuts and a couple of hot chocolates while you walk the Pier looking out at the sea lions is always a perfect afternoon. If you happen to find yourself in the vicinity of the Powell Street Cable Car Turnaround, Blondie's Pizza and Amoeba Records are great if you like-- well, pizza and big indie record stores, respectively. If you're the combative/lulzy/anon sort, there are usually a few Scientologists hanging around there handing out dogma that you can try and troll.
(There's a bigger, better Amoeba location down by Haight/Ashbury if you happen to be down that way. Other highlights, aside from the numerous kitschy trinket shops lining the steets there, would be the Ben & Jerry's shop (just a rinky-dink little ice cream parlor, really, but c'mon-- Ben & Jerry's!) and the numerous used bookstores.)
Metreon is Sony's consumer complex in the city, and it basically amounts to a big touristy tech mall-- it used to be worth a look, but it may have gone downhill since they closed the PlayStation store. It's right up against Yerba Buena Gardens, though, a great spot to eat lunch and just take in the view.
There are plenty of historical spots like the old San Francisco Mint, Fort Point, and the Legion of Honor, if you're into that sort of geekery, and Chinatown and Japantown offer a lot to see in the same regard, as well as the finest in WTF imported Asian consumer culture the city has to offer.
And odd spot that I always loved, down Highway 1, was the ruins of the old Sutro Baths. It used to be a complex of indoor saltwater pools right down by the water on the coast highway, but they burned down a couple times over the years and around 40 years back they finally just said screw it and left them that way. It's an interesting walk, not grueling enough to be called a hike, and I always thought it was the closest thing the city had to offer to an ancient ruin. YMMV, of course. It works a little better if you want to make it part of an hours' hike up the coast trail or down the water's edge, but it's cool enough, and easy enough, to pop in and see if you happen to be down that way anyway-- eating at The Cliff House, perhaps.
Keep in mind that I was a local and a very weird kid... some of these spots may not strike your fancy, but I thought I'd mention them all anyway.