Diabetes is a chronic metabolic disease with an increasing number of patients. As one of the leading causes of death worldwide, it is a significant public health problem. Prevention, early detection and appropriate therapy are key to prevent the progression of the disease and its complications. Therefore, bioactive plant compounds are increasingly being investigated for their potential antidiabetic properties, with fewer side effects compared to conventional antidiabetic drugs. The ability to inhibit α-glucosidase is one of these desirable properties, as it contributes to the control of postprandial hyperglycemia by reducing and slowing the absorption of glucose uptake. The aim of this study was to comparatively investigate the ability of selected polyphenols to inhibit the activity of the α-glucosidase enzyme and to establish a link between the effect and the structural properties of the compounds. Among flavonoids, flavonols with a C2=C3 double bond, a hydroxyl group at C-3 and a carbonyl group at C-4 in the central heterocyclic ring C and the hydroxylated aromatic ring B were found to be the most effective inhibitors. The flavonol myricetin (IC50 63.84 μg/mL), proved to be the only 3',4',5'-trihydroxylated compound tested, as a more potent inhibitor of the enzyme compared to di- and monohydroxylated flavonoids at ring B. Phenolic acids were much weaker enzyme inhibitors compared to flavonoids, and the effect was influenced by the degree of hydroxylation and methoxylation of the aromatic ring. Of all the polyphenols studied, stilbene resveratrol showed the greatest ability to inhibit α-glucosidase.