1870 U.S. census

I've had a request for clarification of column 20 on the 1870 U.S.census.  The heading reads "Male citizen of the United States of 21 years of age and upwards whose right to vote is denied or abridged on other grounds than rebellion or other crime."  In one small town in Ontario County, about two dozen names had check marks in this column.  All foreign-born, four men were born in England, the rest in Ireland.  Can anyone tell me what these "other grounds" might be?

Mary Jo Barone
Ontario County Dept. of Records, Archives, and Information Management Services