And if you look back at Utopia, Sound of Drums, and TLOTL and compare them to what you see in Fragments, no wonder Jack didn't want to stick around. The Doctor had violently rejected him in just about every way possible...as a friend, as living being, and as a victim, and degrading his life's work? I'd have left as soon as I could too.