I made a simple height map generator which basically just sums a set of sine curves with different directions, amplitudes, and wavelengths, creating something that looks like water. The amplitude of the waves are also modulated by the water depth by looking at the height of the terrain when computing the water height map, so that the waves will get more shallow closer to shore. Combining the water height map with Fresnel based reflection and refraction produces the nice water effect shown in the image below:

The height map for the water now takes up the same amount of memory as the terrain height map, so I might try to avoid storing it explicitly, instead computing the sum of sines on demand. This also requires some changes to the kd-tree I use for ray tracing a height map, storing the tree nodes there implicitly as well. The upshot is that the highest wave frequency would no longer be limited by the resolution of the height map.
References:
Reflections & Refractions in Raytracing