The relationship between diet and colorectal cancer has been previously demonstrated and supported with strong epidemiological evidence. The role of genetic polymorphisms has, however, been less well elaborated upon. We conducted a hospital-based case–control study including 727 cases and 736 healthy controls to evaluate the associations of the polymorphic phase-I and -II biotransformations (CYP1A1, CYP1A2, GSTM1, GSTT1, GSTP1, NAT1 and NAT2) and DNA-repair enzymes (XRCC1, XRCC3 and XPD) with the risk of contracting colorectal cancer. We found that men featuring the CYP1A1*2C G/G genotype, the GSTT1 null genotype and XPD 751 with the Gln allele were associated with an elevated risk of colorectal cancer than were men who did not exhibit such genetic features. Multivariate logistic regression analysis revealed that individuals featuring more than two high-risk genotypes increased the colorectal-cancer risk 3.1-fold (OR = 3.1, 95% CI = 1.8–5.2). For women, subjects featuring the CYP1A1*2C G/G genotype and the XRCC3 Thr/Thr genotype faced a 3.1-fold greater risk (95% CI = 1.3–7.0) of colorectal cancer when compared to those featuring the CYP1A1*2C A allele and the XRCC3 Met allele. Taken together, this study suggests that polymorphisms of genes involved in biotransformation and DNA repair could modulate colorectal-cancer risk in Taiwan.