The objective of this study was to determine the fundamental resilient moduli (Mr) parameters of four asphalt-rubber gap-graded (AR-gap), one asphalt-rubber open-graded (AR-open), four polymer-modified gap-graded (P-gap), and two conventional dense-graded asphalt concrete (DGAC) mixes at various temperatures and frequencies using ASTM D7369-11-based standard resilient modulus test [Standard Test Method for Determining the Resilient Modulus of Bituminous Mixtures by Indirect Tension Test, ASTM International, West Conshohocken, PA, 2011, www.astm.org]. Mr tests were conducted at 15°C, 25°C, and 35°C and at 0.5, 1, 1.5, and 2 Hz on a total of 33 samples with three samples per mix. DGAC mixes had the highest Mr followed by P-gap and AR-gap, and then followed by the AR-open mixes. Mr master curves were constructed for the mixes with 25°C as a reference. Furthermore, the Mr model was developed based on the material properties of 11 mixes totaling 121 data points provided by R2adj = 0.9436 (adjusted coefficient of estimation), and Se/Sy = 0.1579 (ratio of standard error to standard deviation indicative of relative accuracy of the predictive model), depicting excellent correlation between the measured and predicted Mr. Fatigue lives of each mix type was predicted using the obtained Mr and estimated tensile strains. The fatigue lives of the modified gap-graded mixes were found to be seven times higher than the conventional mixes. A novel approach was also devised to obtain fatigue lives of conventional and modified mixtures with a reduced thickness design concept based criterion with Mr being the major input parameter. Overall, it is envisioned that the Mr parameters obtained in this study will be helpful to understand the performance characteristics of the different mixes through future laboratory-field correlations.