Observations and high-resolution hydrodynamical simulations indicate that massive star clusters form through a complex hierarchical assembly. We use simulations including post-Newtonian dynamics (the BIFROST code) and stellar evolution (the SEVN module) to investigate this collisional assembly. With a full initial stellar mass function, we study the effect of initial binary, triple, and massive single stars (450 $\,\mathrm{M}_\odot$) on the assembly, structure, and kinematics of massive ($M_\mathrm{cl}\sim 10^6 M_\odot$, $N=1.8 \times 10^6$) star clusters. Simultaneously, intermediate mass black holes (IMBHs), potential seeds for supermassive black holes, can form and grow in our models by stellar collisions, tidal disruption events (TDEs) and black hole (BH) mergers. At a fixed cluster mass, stellar multiplicity or a high mass limit increase the numbers (up to $\sim$10) and masses (up to $10^4 \,\mathrm{M}_\odot$) of the formed IMBHs within the first 10 Myr of cluster evolution. The TDE rates peak at $\Gamma _\mathrm{tde}\sim 5 \times 10^{-5}$ yr$^{-1}$ after IMBH formation at $\sim 2$ Myr. In all simulations, we find gravitational wave driven mergers involving stellar BHs and IMBHs. Initial multiplicity or a high mass limit also result in IMBH–IMBH mergers. The IMBH masses correlate with the initial cluster masses, surface densities, and velocity dispersions approximately as $M_\bullet \propto M_\mathrm{cl}$, $M_\bullet \propto \Sigma _\mathrm{h}^\mathrm{3/2}$, and $M_\bullet \propto \sigma ^\mathrm{3}$. Our results suggest the dense $z\sim 10$ star clusters recently observed by the JWST host IMBHs with masses above $M_\bullet \gtrsim {10^4}\:\mathrm{M_\odot }$.