Fracture energy density (FED), which is defined as the energy per unit volume required to initiate fracture, is a key property governing the resistance to fracture of asphalt binders. This study evaluated the effect of reclaimed asphalt pavement (RAP) and recycled asphalt shingles (RAS) on virgin binder FED using the binder fracture energy (BFE) test. The objective was to determine whether RAP and RAS can be used with soft virgin binders to achieve satisfactory fracture tolerance. Experimental factors included two RAP sources, two RAS sources (manufacture waste (MW) shingles and tear-off (TO) shingles), and four virgin binders. BFE tests were conducted on blends of virgin and recovered RAP/RAS binders at two binder replacement rates of 15 % and 30 %. Moreover, the Superpave true grade of RAP/RAS binder blends was determined using the dynamic shear rheometer and bending beam rheometer. Results showed that the use of soft virgin binders effectively compensated for the stiffening effect of RAP/RAS in terms of true grade. The addition of RAP binder and MW shingle binder increased the FED of unmodified binders, whereas the opposite trend was observed for TO shingle binder. Furthermore, great caution should be exercised when using virgin polymer-modified binders because significant reductions in FED were observed when RAP and RAS binder was introduced, possibly because of the dilution of polymer modification in addition to stiffening and embrittlement effects. This study indicated that both RAP and RAS are recyclable, according to the Superpave true grade requirements. However, further research is needed to evaluate the effect of the reduction in binder FED caused by TO shingles on mixture fracture performance before its acceptance in asphalt mixtures.