We present near infrared (J, H and Ks) observations of a 5'× 10' sample field in the Serpens Star Formation region obtained with SOFI at the NTT. These observations are sensitive enough to detect a 20 MJup brown dwarf through an extinction of AV˜ 16 and are used to build an infrared census of this field in the cluster. From photometry and mass-luminosity models, we have developed a detailed methodology to extract quantitative parameters (distance modulus, extinction, spectral type, masses) for objects observed towards and inside the Serpens molecular cloud. An extinction map of the region is derived allowing us to disentangle cloud members from background field objects. Luminosities and masses for 14 low-mass stars and substellar object candidate members of the cluster are derived. Three of these objects have masses compatible with the brown dwarf regime and one of them (BD-Ser 1) was observed spectroscopically with ISAAC at the VLT, confirming its substellar status. Long-term photometric variability of BD-Ser 1 could be consistent with signs of accretion. Based on observations collected with TBL/MOICAM instrument at the Pic du Midi Observatory, France, and NTT/SOFI & VLT/ISAAC instruments at the European Southern Observatories, La Silla and Paranal, Chile (ESO Programs P63.L-0227 and P65.L-0637). Photometric data are only available in electronic form at the CDS via anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or via http://cdsweb.u-strasbg.fr/cgi-bin/qcat?J/A+A/425/927