? Ok its a little long to explain the story but hopefully you'll bare with me

This guy friend of mine (we has a bit of a fling kinda before we were really friends) and well he wasn't looking for a relationship but he never really asked me what I was looking for he sort of made assumptions, (because I'm a virgin and we nearly went all the way but he didn't feel comfortable with being my first, he said he wasn't looking for a relationship, though he did ask me if I wanted him continue at one point I said I wasn't sure.) anyways we decided to stay friends, but it wasn't working too well, there was a lot of miss communication, he was very flaky, but often turned it around and said I was being paranoid and would re-arrange things for times that suited him, instead of admitting he had been unreliable.
Just before we stopped talking he was trying to suggest I go see a doctor because he thought I was depressed, and that he had recently gone on anti-depressants that had really helped him. I told him I appreciated his advice but I didn't think it was a good choice for me.

I felt confused about his motives. I decided it would be better for us to not hang out anymore because I felt rejected and I would just end up feeling more jealous and resenting him (which I think is worse) However when I messaged him on fb to tell him I didn't think we should hang because I felt like he was grossed out by me after finding out I was a virgin, and that being around him wasn't good for my confidence, I said I'd like to remain civil and talk online still etc-
He replied saying that I was the one who was acting weird, and said I was the one shying away when he tried to make conversation, and said whatever i wished and that he didn't mind.
I messaged him trying to explain a little better and asking if we could be friends in future if it was cool with him, but no reply, 3 days later, I felt guilty so I thought I should apologise, I sent him a fb message saying sorry for blaming him and that I didn't want to lose a friend completely and that I still consider him a friend even if he doesn't care.
Still no reply after about 5 days so I deleted him off FB (mainly because I didn't want to bother him anymore)
Hadn't heard from him at all text or fb in a month, until I saw him at work a few weeks ago (he popped in to get something) When he walked in he didn't say anything but as he left I was talking to our mutual friend and serving a customer he just said "Hi " and i said hello then he said "you ok?" I was like "yeah" then he just left.
Didn't see him again until Saturday last week, again he just popped into work, He said hi to me again, and the Saturday just gone he popped in again (he doesn't work weekends and I havent seen him pop into work before which is annoying when you re trying to avoid someone) he said hi to me this time too. I'm just confused/upset/annoyed about how he just ignored me all that time, without even giving me much closure about being ok about everything, but sometimes I feel guilty about it, is it just me who is in the wrong here?

Its hard because we have a mutual friend, hes invited to the same birthday and another event where he will probs befriend all my closest friends, it means I feel like I can never go out with them if he is there.

I am basically just looking for some help/suggestions on what I should do, and what could help make me feel better, thanks.
I guess I'll have to lose my friends then