The Aβ peptide forms extracellular plaques associated with Alzheimer's disease. In addition to protein fibrils, amyloid plaques also contain non-proteinaceous components, including glycosaminoglycans (GAGs). We have shown previously that the GAG low molecular weight heparin (LMWH) binds to Aβ40 fibrils with a three-fold-symmetric (3Q) morphology with higher affinity than Aβ40 fibrils in alternative structures, Aβ42 fibrils, or amyloid fibrils formed from other sequences. Solid-state NMR (SSNMR) analysis of the GAG-3Q fibril complex revealed an interaction site at the corners of the 3Q fibril structure, but the origin of the binding specificity remained obscure. Here, employing a library of short heparin polysaccharides modified at specific sites, we show that the NS or 6-OS glucosamine sulfates, but not the 2-OS iduronate sulfate, of heparin is required for 3Q binding, indicating selectivity in the interactions of the GAG with the fibril that extends beyond general electrostatic complementarity. By creating 3Q fibrils containing point substitutions in the amino acid sequence, we also show that charged residues at the fibril three-fold apices provide the majority of the binding free energy, while charged residues elsewhere are less critical for binding. The results indicate, therefore, that LMWH binding to 3Q fibrils requires a precise molecular complementarity of the sulfate moieties on the GAG and charged residues displayed on the fibril surface. Differences in GAG binding to fibrils with distinct sequence and/or structure may thus contribute to the diverse etiology and progression of amyloid diseases.