My boyfriend sleep mostly all day?

So I've been seeing this guy for maybe a month now. but we have been talking a few months before we starting going out. We have hung out a few times time we have been talking/Dating. but I just don't seen like it's enough. he tells me he is going to come to my house and usually ends up sleeping most of the day until the late evening. Also he tells be he stays up really late into the morning. so I can see why he sleeps so much now. I feel like I don't even get to talk to him at all now because of this. It is totally effecting our relationship. and he doesn't even see it! he tells me he is sorry that he couldn't make it every time, and that he slept in again... and he keeps asking me if I'm mad at him or if I want to break up with him. I say no because if I said that I would be lying. but I really don't enjoy have a boyfriend that I can't see hardly at all. also I am very shy and the only way I can get used to him more is to be around him and he doesn't get that either. I feel like I'm stuck because he says he doesn't want to loose me. I do really like him a lot and I don't want to break up with him over his sleeping issues, but I am really getting upset about this. do any of you guys know what I can do? or any advice at all ?