We show that the barrier profile of in situ grown AlOx tunnel barriers strongly depends on the material choices of the oxide-metal interface. By doing transport measurements on Al and Nb-based metal-oxide-metal tunnel junctions in a wide temperature range and using the phenomenological Simmons' model, we obtain barrier parameters that are qualitatively consistent with the values obtained from the first-principles calculations. The latter suggest that the formation of metal-induced gap states originating from the hybridization between the metallic bands and Al2O3 conduction band is responsible for the tunnel barrier modification. These findings are important for nanoelectronic devices containing tunnel junctions with a thin insulating layer.