A clock comparison experiment, analyzing the ratio of spin precession frequencies of stored ultracold neutrons and 199Hg atoms, is reported. No daily variation of this ratio could be found, from which is set an upper limit on the Lorentz invariance violating cosmic anisotropy field b perpendicular < 2 x 10(-20) eV (95% C.L.). This is the first limit for the free neutron. This result is also interpreted as a direct limit on the gravitational dipole moment of the neutron |gn| < 0.3 eV/c2 m from a spin-dependent interaction with the Sun. Analyzing the gravitational interaction with the Earth, based on previous data, yields a more stringent limit |gn| < 3 x 10(-4) eV/c2 m.