About a month ago, a little bump appeared between the toes of my foot. It was kinda itchy, and being who I am, I couldn't leave it alone. i popped it, and it was filled with a clear fluid. More appeared, on and off in the last month or so. They appear mostly either between my toes, or on the bottom of the toes. Also, the skin right between my toes looks like it split, and seeps the same kind of clear, oily fluid. It itches terribly sometimes, and other times, I don't even think about it.
I don't think it's athlete's foot. The bumps aren't red, and to be completely honest, I'm not wearing my shoes very often, and when I do, I'm wearing sandals. Maybe it's because my feet get dirty a lot?

Does anyone know what this might be, and if I can treat it at home?