Every marriage solemnized by any person who has received episcopal ordination, but who is not a Clergyman of the Church of England, or of the Church of Rome, or by any Minister of Religion licensed under this Act to solemnize marriages, shall immediately after the solemnization thereof, be registered in duplicate by the person solemnizing the same; (that is to say) in a marriage-register book to be kept by him for that purpose, according to the form contained in the Fourth Schedule hereto annexed, and also in a certificate attached to the marriage-register-book as a counterfoil.