It depends on your community. Lots of pretty little towns have had homes go in that were built to be affordable housing for locals, but the locals got outbid by the folks doing short term vacation rentals pretty quickly.

so build more.