When the defendant is so delivered into custody he must be held by the Sheriff, unless admitted to bail on examination upon a writ of habeas corpus.