Durant had his differences with each of his first two NBA organizations