Phytophthora infestans is the causal agent of late blight disease of potatoes and tomatoes. This pathogen is highly dependent on environmental factors, such as temperature, to fully develop its life and disease cycle. Due to climate variations, changes in the patterns of plant diseases' development are expected to happen. Previous studies have evaluated the effect of temperature on the rate of sporangial germination, the development of the infection, and mycelia growth of P. infestans. These studies have shown that the response to temperature changes may vary depending on the genotypic diversity. However, the genetic basis associated to the phenotypic response for temperature changes has been scarcely studied. Through this study, we investigated the effect of temperature on the growth of Colombian isolates of P. infestans and we identified SNPs associated to relative growth assessed at different temperatures (5, 10, 15, 20, and 25°C). A high variation on the growth of the isolates at each evaluated temperature was found. However, for most isolates, the ideal growth temperature was 20°C. The population structure showed that two populations group most of the genotypes used in this study, and there are intermediate genotypes in this population. The association analysis between high-quality SNPs and the relative growth for temperature assessed was performed using a model GLM corrected with the Matrix Q. A total of 17 SNPs associated with relative growth at different temperatures were found and 13 SNPs were associated with growth at 5°C. Among these, four were related to genes, codifying for proteins with functions at the cellular level such as transport of amino acids, ATP binding, and hydrolysis of peptide bonds. This study provides further insights on the genetic basis of the response to temperature changes of isolates of P. infestans collected in Colombia.