With the aim of constructing accurate two-dimensional maps of the stellar mass distribution in nearby galaxies from Spitzer Survey of Stellar Structure in Galaxies 3.6 and 4.5 μm images, we report on the separation of the light from old stars from the emission contributed by contaminants. Results for a small sample of six disk galaxies (NGC 1566, NGC 2976, NGC 3031, NGC 3184, NGC 4321, and NGC 5194) with a range of morphological properties, dust content, and star formation histories are presented to demonstrate our approach. To isolate the old stellar light from contaminant emission (e.g., hot dust and the 3.3 μm polycyclic aromatic hydrocarbon (PAH) feature) in the IRAC 3.6 and 4.5 μm bands we use an independent component analysis (ICA) technique designed to separate statistically independent source distributions, maximizing the distinction in the [3.6]-[4.5] colors of the sources. The technique also removes emission from evolved red objects with a low mass-to-light ratio, such as asymptotic giant branch (AGB) and red supergiant (RSG) stars, revealing maps of the underlying old distribution of light with [3.6]-[4.5] colors consistent with the colors of K and M giants. The contaminants are studied by comparison with the non-stellar emission imaged at 8 μm, which is dominated by the broad PAH feature. Using the measured 3.6 μm/8 μm ratio to select individual contaminants, we find that hot dust and PAHs together contribute between ~5% and 15% to the integrated light at 3.6 μm, while light from regions dominated by intermediate-age (AGB and RSG) stars accounts for only 1%-5%. Locally, however, the contribution from either contaminant can reach much higher levels; dust contributes on average 22% to the emission in star-forming regions throughout the sample, while intermediate-age stars contribute upward of 50% in localized knots. The removal of these contaminants with ICA leaves maps of the old stellar disk that retain a high degree of structural information and are ideally suited for tracing stellar mass, as will be the focus in a companion paper.