A pathological signature of Alzheimer’s disease (AD) is the formation of neurofibrillary tangles comprising filamentous aggregates of the microtubule associated protein tau. Tau self-assembly is accelerated by polyanions including heparin, an analogue of heparan sulfate. Tau filaments colocalize with heparan sulfate proteoglycans (HSPGs) in vivo, and HSPGs may also assist the transcellular propagation of tau aggregates. Here, we investigate the role of the sulfate moieties of heparin in the aggregation of a recombinant tau fragment Δtau187, comprising residues 255–441 of the C-terminal microtubule-binding domain. The effects that the selective removal of the N-, 2-O-, and 6-O-sulfate groups from heparin have on the kinetics of tau aggregation, aggregate morphology, and protein structure and dynamics were examined. Aggregation kinetics monitored by thioflavin T (ThT) fluorescence revealed that aggregation is considerably slower in the presence of 2-O-desulfated heparin than with N- or 6-O-desulfated heparin. Transmission electron microscopy revealed that tau filaments induced by 2-O-desulfated heparin were more slender than filaments formed in the presence of intact heparin or 6-O-desulfated heparin. The 2-O-desulfated heparin-induced filaments had more extensive regions of flexibility than the other filaments, according to circular dichroism and solid-state NMR spectroscopy. These results indicate that the sulfation pattern of heparin regulates tau aggregation, not purely though electrostatic forces but also through conformational perturbations of heparin when the 2-O-sulfate is removed. These findings may have implications for the progression of AD, as the sulfation pattern of GAGs is known to change during the aging process, which is the main risk factor for the disease.