Well, first of all there's the price. The F/4 will run you 2-3 times more than the f/4-7.1!
The f/4 is a Canon L series lens which means better build, better glass, a lens hood included, some level of weather sealing. It is also almost twice as heavy and obviously has a brighter aperture when zoomed out to 105mm.
The F/4-7.1 has a smaller filter size so if you use filters they will cost you less.
But overall, there will be a difference in image quality; Like the saying goes, you get what you pay for. And keep in mind you have a super duper camera body. All the more reason to put quality glass on it. Don't get me wrong, Canon makes a lot of good non-L lenses. I own several of their prime ones that are non-L and overall they produce good pictures. But I own the 24-105mm f/4 and love it. It's my general walking around lens, good for a variety of things but also for portraits to landcapes.