We conducted sub-millimeter observations with the Atacama Large Millimeter/sub-millimeter Array (ALMA) of star-forming galaxies at $z\sim3.3$, whose gas-phase metallicities have been previously measured. We investigate the dust and gas contents of the galaxies at $z\sim3.3$ and study how galaxies are interacting with their circumgalactic/intergalactic medium at this epoch by probing their gas mass fractions and gas-phase metallicities. Single-band dust continuum emission tracing dust mass and the relation between the gas-phase metallicity and gas-to-dust mass ratio are used to estimate the gas masses. The estimated gas mass fractions and depletion timescales are $f_{\rm gas}=$ 0.20-0.75 and $t_{\rm dep}=$ 0.09-1.55 Gyr, respectively. Although the galaxies appear to tightly distribute around the star-forming main sequence at $z\sim3.3$, both quantities show a wider spread at a fixed stellar mass than expected from the scaling relation, suggesting a large diversity of fundamental gas properties among star-forming galaxies apparently on the main sequence. Comparing gas mass fraction and gas-phase metallicity between the star-forming galaxies at $z\sim3.3$ and at lower redshifts, star-forming galaxies at $z\sim3.3$ appear to be more metal-poor than local galaxies with similar gas mass fractions. Using the gas regulator model to interpret this offset, we find that it can be explained by a higher mass-loading factor, suggesting that the mass-loading factor in outflows increases at earlier cosmic times.