Predicted by General Relativity, black holes pose challenging and thorny questions to our knowledge of the Universe and cosmic structure formation. Just less than 1 billion years from the Big Bang, black holes whose mass exceeds 1 billion solar masses are now routinely discovered. Yet, their origin, growth, evolution and impact on the host galaxy and environment remain largely unknown. When and where did the first black holes form in the Universe? What physics regulated this event? These are just some of the questions that I will address during my talk, in the attempt to give a broad and simple overview of this exciting and blossoming field.