Pursuant to Section 106, each appointing authority has the power to remove from office, at any time, any member of the board appointed by that authority under Section 2703 for continued neglect of duties required by law, or for incompetence, or unprofessional or dishonorable conduct.