Gravitational potential energy - finite amount of work to get to infinity?

Formula for gravitational potential energy:
-GMm/r (defining 0 gravitational potential energy to be at infinity)

So say we subbed in some value of r, m and M to get the gravitational potential energy of an object and the Earth

Say this gave gravitational energy of -100 joules

Now we want to know how much work needs to be done to get to infinity

By definition, at infinity gravitional potential energy = 0, so then the work is 0 - (-100) = 100 joules

So this means if we do a finite amount of work (100 joules) on the object we'll get to infinity

I don't see any reason why we wouldn't be able to do this amount of work

So how is this possible? How can we actually get to infinity?