Discord servers are private, but they don't necessarily have to be. As soon as a non-expiring invite link makes its way out into the world, the server is much more public! However, the server administrator can always deactivate an active invite link. If you set up a server and disable invite link creation at all levels except the administrator level, you can tighten down your server pretty effectively! Invite links are a secure way to add new people to your server. No secret passwords required!
Go to the roles tab of your Server Settings, select that @everyone role, and uncheck all permissions boxes. Then you can create a new role for your members, name it whatever you like, and customize the permissions to your needs. Then assign this role to anyone you like. This server will only be accessible by the people you want in it. Even if someone gets invited somehow, they still won't be able to see or send messages unless given the new role you created!