Ha, it's not like CYS points have much in terms of value (apart from, I believe, adding some weight to ratings you give to stories), but the points are a nice thing to have. Besides, even the stupid 'gameification' of CYS by awarding points for visiting might make people visit more often.
Also, in relation to the dwindling user numbers in the OP, I think the frequent issues with viewing story categories if a story has fewer than ten ratings, might scare off quite a lot of people as well.