I only had two conversations with him. The first only lasted about five minutes, as he had to look for what it was I wanted to talk to him about. The second convo was probably about twenty minutes. He seemed pleasant to me. He's living in a small neighborhood on a little island in Florida with a boat lift in his backyard that can get him out on the Gulf in just a few minutes. He said he spends quite a lot of time ministering to prisoners.
I hadn't realized how young he was when he retired. He was in his early 50s when he left Toledo and never got back into it. The guy who worked with him for a few years had nothing but really positive things to say about working for him. From my experiences talking to FBS head coaches, most want to control conversations (especially on the phone), dominate the discussion, and talk mostly about themselves (there are a few I have noticed aren't like that). He wasn't like that at all.
Based on his appearance and leading Marshall's rival at the time, he was an easy target to dislike and get the wrong idea about. From very limited time talking to him, he seemed the complete opposite of what you may think.