That's a pretty plausible explanation. Ill-matched though they are, the others are all he has. And despite everything, I think he did like the fact that they accepted him; he just couldn't say it.

I hope it turns out to be "our" Rimmer in Series X. :)