The true importance of the warm, AGN-driven outflows for the evolution of galaxies remains uncertain. Measurements of the radial extents of the outflows are key for quantifying their masses and kinetic powers, and also establishing whether the AGN outflows are galaxy-wide. Therefore, as part of a larger project to investigate the significance of warm, AGN-driven outflows in the most rapidly evolving galaxies in the local universe, here we present deep Hubble Space Telescope (HST) narrow-band [O III] λ5007 observations of a complete sample of eight nearby ULIRGs with optical AGN nuclei. Combined with the complementary information provided by our ground-based spectroscopy, the HST images show that the warm gas outflows are relatively compact for most of the objects in the sample: in three objects, the outflow regions are barely resolved at the resolution of HST (0.065 < R_[O III] < 0.12 kpc); in a further four cases, the outflows are spatially resolved but with flux-weighted mean radii in the range 0.65 < R_[O III] < 1.2 kpc; and in only one object (Mrk273) is there clear evidence for a more extended outflow, with a maximum extent of R_[O III]˜ 5 kpc. Overall, our observations show little evidence for the galaxy-wide outflows predicted by some models of AGN feedback.