Effective immunity requires a complex network of cellular and humoral components that interact with each other and are influenced by different environmental and host factors. We used a systems biology approach to comprehensively assess the impact of environmental and genetic factors on immune cell populations in peripheral blood, including associations with immunoglobulin concentrations, from ∼500 healthy volunteers from the Human Functional Genomics Project. Genetic heritability estimation showed that variations in T cell numbers are more strongly driven by genetic factors, while B cell counts are more environmentally influenced. Quantitative trait loci (QTL) mapping identified eight independent genomic loci associated with leukocyte count variation, including four associations with T and B cell subtypes. The QTLs identified were enriched among genome-wide association study (GWAS) SNPs reported to increase susceptibility to immune-mediated diseases. Our systems approach provides insights into cellular and humoral immune trait variability in humans.