Tinnitus is an auditory phantom perception in the ears or head in the absence of a corresponding external stimulus. There is currently no effective treatment available that reliably reduces tinnitus. Educative counseling is a treatment approach that aims to educate patients and inform them about possible coping strategies. For this feasability study, we implemented educational material and self-help advice in a smartphone app. Participants used the educational smartphone unsupervised during their daily routine over a period of 4 months. Comparing the tinnitus outcome measures before and after smartphone-guided treatment, we measured changes in the tinnitus-related distress, but not in tinnitus loudness. Improvements on the Tinnitus Severity numeric rating scale reached an effect size of .408, while the improvements on the THI were much smaller with an effect size of .168. Analysis on the user behavior showed that frequent and intensive use of the app is a crucial factor for treatment success: participants that used the app more often and interacted with the app intensively, reported a stronger improvement of the tinnitus. Between study allocation and final assessment, 26 of 52 participants dropped out of the study. Reasons for the dropouts and lessons for future studies are discussed in this paper.