Improving methods for determining the subsurface structure of sunspots from their seismic signature requires a better understanding of the interaction of waves with magnetic field concentrations. We aim to quantify the impact of changes in the internal structure of sunspots on local helioseismic signals. We have numerically simulated the propagation of a stochastic wave field through sunspot models with different properties, accounting for changes in the Wilson depression between 250 and 550 km and in the photospheric umbral magnetic field between 1500 and 3500 G. The results show that travel-time shifts at frequencies above approximately 3.50 mHz (depending on the phase-speed filter) are insensitive to the magnetic field strength. The travel time of these waves is determined exclusively by the Wilson depression and sound-speed perturbation. The travel time of waves with lower frequencies is affected by the direct effect of the magnetic field, although photospheric field strengths below 1500 G do not leave a significant trace on the travel-time measurements. These results could potentially be used to develop simplified travel-time inversion methods.