We propose a wide-area, high latitude, imaging survey to trace the evolution of dusty, star-forming galaxies, evolved stellar populations, and AGN, as a function of environment from z~2.5 to the current epoch. Building on ISO's heritage, SWIRE complements smaller, deeper GTO (Guaranteed Time Observer) surveys, and paves the way for FIRST. With MIPS 5 sigma sensitivities of 0.45/2.75/17.5 mJy at 24/70/160 microns over 100 square degrees (424 hrs), and 7.3/9.7/27.5/32.5 microJy at 3.6/4.5/5.8/8.0 microns for 55 square degrees (IRAC: 427 hrs), we will deliver highly uniform source catalogs and high-resolution, calibrated images, providing an unprecedented view of the evolution of galaxies, structure, and AGN on co-moving scales up to several hundred Mpc. SWIRE will, for the first time, directly address the clustering of evolved stellar systems (IRAC) vs active star-forming systems and AGN (MIPS) in the same volume. Extensive modeling suggests that the Legacy Extragalactic Catalog may contain in excess of 2 million IR-selected galaxies dominated by (1) luminous infrared galaxies, Lfir>10^11 L_sun, up to 40,000 with z > 2; (2) ~10^6 early-type galaxies (~4x10^5 with z>2); (3)~30,000 classical AGN and as many as 250,000 dust-obscured QSO/AGN. Pixon image reconstruction will optimize spatial resolution, reduce confusion noise and improve sensitivity, and will be delivered to the SSC. These fields will have extensive data at other wavebands, particularly in the optical, near-IR and X-ray; further ground-based imaging will be undertaken at NOAO and other observatories. SWIRE Legacy data will be will be combined with a wide range of X-ray, optical, infrared, submm and radio data, largely available through IPAC's Infrared Science Archive (IRSA), as part of this legacy.