Question:

Any particularly good but authentic ones in central Tokyo? Must be gajjin friendly too.

Answer:

There are a lot of onsen in Tokyo.
These are famous 3 onsen from Edo period.

Azabu juban onsen 麻布十番温泉
http://www.tokyoessentials.com/azabu-jub…

Asakusa kannon onsen 浅草観音温泉
http://www.onsenjapan.net/onsenreview.ph…

Shinjuku junisou onsen 新宿十二社温泉
http://onsensoaker.blogspot.com/2009/03/…

If you can read Japanese, you'll find other onsen in Tokyo.
http://www21.ocn.ne.jp/~spa-mich/todofuk…

However I recommend you to visit a onsengai, like Hakone, Atami or Kusatsu.

And read this book.
http://www.enterbrain.co.jp/comic/TR/