Background: Type 2 diabetes (T2DM) increases the risk of mild cognitive impairment (MCI), however, the mechanisms of this process have not yet been fully identified. The aim of the study was to determine the levels of vitamin B12 and homocysteine in elderly diabetic patients, with and without cognitive impairment, and identify the risk factors associated with MCI in this group. Methods: A total of 385 elderly diabetic patients were to screened to for MCI. Several clinical and biochemical data were recorded. Results: In the study group, MCI subjects had lower vitamin B12 levels and higher homocysteine concentrations compared to controls. In MCI subjects, vitamin B12 levels were negatively correlated with homocysteine or HbA1c levels and positively correlated with MoCA scores. The univariate logistic regression showed factors associated with MCI in elderly diabetic patients were the following: older age and fewer years of education, longer history of diabetes, a higher number of co-morbidities, higher levels of HbA1c, triglycerides and homocysteine, lower levels of vitamin B12, presence of cardiovascular disease, hypertension, hyperlipidemia, retinopathy, and nephropathy. Independent factors associated with MCI evaluated in multivariate model included lower levels of vitamin B12, higher levels of triglycerides, a higher number of co-morbidities and fewer years of formal education. Conclusions: The association between T2DM and higher risk of MCI is partly mediated by lower levels of vitamin B12, and higher levels of homocysteine. The findings of this study highlight the importance of regular vitamin B12 screening and routine cognitive function testing in older adults with diabetes.