Is it a different Elizabeth? Perhaps his first wife died and he married another Elizabeth - stranger things have happened! The Elizabeth on the 1861 census is 10 years younger than the Elizabeth on the 1851 census. Just a suggestion! Jess