Quantum information processing using atomic qubits requires narrow linewidth lasers with long-term stability for high fidelity coherent manipulation of Rydberg states. In this paper, we report on the construction and characterization of three continuous-wave (CW) narrow linewidth lasers stabilized simultaneously to an ultra- high finesse Fabry-Perot cavity made of ultra-low expansion (ULE) glass, with a tunable offset-lock frequency. One laser operates at 852 nm while the two locked lasers at 1018 nm are frequency doubled to 509 nm for excitation of 133Cs atoms to Rydberg states. The optical beatnote at 509 nm is measured to be 260(5) Hz. We present measurements of the offset between the atomic and cavity resonant frequencies using electromagnetically induced transparency (EIT) for high-resolution spectroscopy on a cold atom cloud. The long-term stability is determined from repeated spectra over a period of 20 days yielding a linear frequency drift of ∼ 1 Hz/s.