The various possibilities for the origin ("progenitors") of gamma-ray bursts (GRBs) manifest in differing observable properties. Through deep spectroscopic and high-resolution imaging observations of some GRB hosts, I demonstrate that well-localized long-duration GRBs are connected with otherwise normal star-forming galaxies at moderate redshifts of order unity. Using high-mass binary stellar population synthesis models, I quantify the expected spatial extent around galaxies of coalescing neutron stars, one of the leading contenders for GRB progenitors. I then test this scenario by examining the offset distribution of GRBs about their apparent hosts making extensive use of ground-based optical data from Keck and Palomar and space-based imaging from the Hubble Space Telescope. The offset distribution appears to be inconsistent with the coalescing neutron star binary hypothesis (and, similarly, black-hole--neutron star coalescences); instead, the distribution is statistically consistent with a population of progenitors that closely traces the ultra-violet light of galaxies. This is naturally explained by bursts which originate from the collapse of massive stars ``collapsars''). This claim is further supported by the unambiguous detections of intermediate-time (approximately three weeks after the bursts) emission ``bumps'' which appear substantially more red than the afterglows themselves. I claim that these bumps could originate from supernovae that occur at approximately the same time as the associated GRB; if true, GRB 980326 and GRB 011121 provide strong observational evidence connecting cosmological GRBs to high-redshift supernovae and implicate massive stars as the progenitors of at least some long-duration GRBs.