Lincoln really didn't have a religion. He was replacing religion with a moral stance. Even then, it was hard to get elected without having some sort of religious belief. I guess it worked.
I believe he lived his life by this principle, and that is why he called it his 'religion.' To me what it means is that he gets a sense of worth out of doing good things, but when he does something bad, perhaps a sinful act, he feels bad and doesn't like it. Makes him feel ashamed.. So he bases his life on this and lives by them religiously. If he didn't he would consider himself immoral; perhaps a hypocrite.