Is your Jeep lifted?
Don't know if the 04 WJ's have independent or solid front axles.
If an independent front end the i'd first check is the CV joint where it connects to the driver's side front wheel.
Check if the rubber boot is split. If so that CV joint could be history needing replacement.
If it a solid front axle with no lift, its usually the U-joint where it connects to the driver's side front wheel.
I would bet the U-joint(or CV joint) at the drivers wheel is bad. I'm not familiar if the WJ comes with a CV or U joint at that location. Should be easy to replace, just be careful inserting the shaft back into the diff as it is easy to damage the seal.