Approximately 10% of the general population reports a penicillin allergy, making it one of the most commonly documented drug allergies in clinical practice. Yet formal evaluation confirms true hypersensitivity in fewer than 10% of these cases. This gap has practical consequences. Patients who carry an inaccurate allergy label are more likely to receive broader-spectrum alternative antibiotics, with downstream effects on cost, adverse drug events, and antimicrobial resistance. Although primary care physicians are often the first to record these labels and the ones who face their consequences most often in daily prescribing, they have remained peripheral to most systematic de-labeling efforts. In this narrative review, we examine how antibiotic allergy labels arise, why they persist, and what they cost—clinically, economically, and from a stewardship perspective. We also discuss emerging approaches to reassessment in primary care, along with the possible role of digital health tools and patient education in improving the accuracy of allergy documentation.