With imaging at 3.6 and 4.5 microns where the light in nearby galaxies is dominated by old stars, the Spitzer Survey of Nearby Galaxies (S4G) stands poised for an optimal view of stellar mass and structure in the local Universe.  I will describe an effort to construct accurate 2D stellar mass maps from S4G images, starting with a correction for non-stellar (e.g. PAH and hot dust) contaminant emission using only the two S4G images as inputs; contaminant emission is isolated from the old stellar light using an Independent Component Analysis (ICA) technique designed to separate statistically independent source distributions.  An inventory of recovered contaminants is established via comparison to the non-stellar emission in archival 8 micron images.  Once these contaminants are removed, maps of the underlying distribution of old stars are revealed that retain a high degree of structural information and exhibit [3.6]-[4.5] colors consistent with those of K and M giants.  Contaminant-free S4G maps constructed with this approach should be ideally suited for tracing the stellar mass in galaxies spanning a range of morphological properties, dust contents and star formation histories.