In the light of more and more new instrumentation to get a deeper insight into the universe, tons of data are collected. While traditional machine-learning methods have been used in processing stellar spectral data, such large new datasets are better handled with Deep Learning (DL) techniques. In this work, we present a Deep Convolutional Neural Network (CNN) approach to derive fundamental stellar parameters (effective temperature, surface gravity, metallicity and rotational velocity) from high-resolution high signal-to-noise ratio spectra. We construct an individual CNN architecture for each of the four parameters and train them on synthetic PHOENIX-ACES spectra. After that, we apply the trained networks to the observed spectra of 50 M dwarfs observed with CARMENES. The CARMENES spectrograph, installed on the 3.5 m telescope at the Calar Alto Observatory (Spain) has two channels, covering the visible (0.52 to 0.96 µm, R = 96,400) and near-infrared (0.96 to 1.71 µm, R = 80,600) spectral ranges. We compare our results to literature values, and demonstrate that our method can be used for stellar parameter determination without the need of having a huge sample of stellar spectra with known parameters, because our networks can be trained on synthetic spectra. Introducing Deep Transfer Learning (DTL) in our approach allows us to transfer external knowledge about the stellar parameters (e.g., from interferometry) to our training set and therefore improve our results compared to literature.