This study explores socio-structural factors that influence uptake of antiretroviral treatment (ART) in Zambia and assess differences between men and women. We conducted a case-control study nested in a community- and health facility-based survey, between September 2010 and February 2011. Cases were defined as HIV-positive individuals who, while eligible, never started ART and controls were HIV-positive individuals who were on ART. Cases and controls were matched by place of residence. We performed a conditional logistic regression analysis using a discrete logistic model stratified by sex. Overall, a significantly larger proportion of men (32.7%) than women (25.6%) did not uptake ART (Pearson χ(2) = 5.9135; p = 0.015). In the crude analysis, poor health status and low self-efficacy were common factors associated with non-uptake in both sexes. After adjusting for covariates, men were more likely than women to refuse ART even though men's self-rated health was lower than women's. In general, the adjusted analysis suggests that HIV status disclosure affects uptake in both sexes but women's uptake of ART is largely hampered by poverty-related factors while for men, side effects and social pressure, probably associated with masculinity, are more important barriers. Alarmingly men's health seems to deteriorate until they start treatment, in contrast to women. Understanding gender differences in uptake and attitudes to ART is a crucial component to providing effective and appropriate health care to both men and women living with HIV/AIDS in Zambia.