The golden rule of list management is "benevolent dictatorship". You, as a list owner, can do whatever the hell you like with your list, including closing it down when you get sick of it. You're offering it as a service to the community and because you're personally interested in it, not because you're somehow obliged to.
Don't try to use democracy on a mailing list. There'll always be at least n+1 opinions, where n is the number of subscribers, and in fact there are usually more than that. If you want to do something to or with the list, post a message telling them your plans, allow a couple of days for any major issues to arise, then do what you feel is right. Any other course of action leads to insanity.