UIRate: A Personalized User Interface Aesthetic Scoring Tool Based on Individual Features

1. Introduction

User interface (UI) aesthetics play a crucial role in shaping users’ initial impressions of a website, mobile application, or digital product. However, aesthetics is not a universal concept, and users with different backgrounds may have different opinions about the same interface [1]. Consequently, a global UI aesthetics scoring system cannot show a preference for different people and may cause errors for designers’ decision-making, especially when designing for a Specific target population.

Previous studies in UI aesthetic evaluation have mainly focused on the interface of the visual features and learning from these patterns [2,3,4]. While these methods can be useful for general-purpose scoring, they are not able to directly model the individual features of a target group and effectively help the designers to estimate their design for their target users. On the other hand, while human evaluation by experts is valuable, it is time-consuming [5,6], and there may be significant variation between evaluators, especially when the target user’s perceived score should be predicted based on that user’s characteristics.

This work introduces UIRate, a personalized aesthetic scoring framework that uses the UI image and the user profile features to predict an aesthetic score specifically for each user. The method uses a pre-trained EfficientNet-based [7] backbone for extracting visual features and a Multilayer Perceptron (MLP) to encode individual features into an embedding. Then, a final prediction of the aesthetic score is produced by a fusion mechanism between these two representations. In addition, this method applies Grad-CAM [8] to address the need for transparency by highlighting the regions of an image that most influence the model’s predictions.

Figure 1. Overview of the UIRate tool interface. (a) Initial screen before providing any user information or UI input. (b) Result screen after submitting the required inputs, showing the generated aesthetic score and the tool’s output summary.

Figure 1. Overview of the UIRate tool interface. (a) Initial screen before providing any user information or UI input. (b) Result screen after submitting the required inputs, showing the generated aesthetic score and the tool’s output summary.

2. Framework

A multimodal system is implemented to predict the users’ aesthetic preferences about webpage user interfaces (Figure 2). Unlike most of the current methods, which focus on the visual features of a webpage, this approach considers users’ information to predict a personalized score.

The model was trained on a dataset including webpage user interfaces, user features, and the rates of each user for a webpage UI [9]. User features contain demographic and experiential information, such as web usage familiarity and background. These features are represented as a vector after the pre-processing step and used as an input for the model.

This method utilized a multimodal neural network architecture that includes two parallel branches: an EfficientNet-based encoder to extract visual features of a webpage user interface and a multilayer perceptron to encode user features. The extracted features are fused and used by a regression layer to predict the scores. This method enables the exploration of the difference between various groups of users instead of a global rating for a webpage UI.

Furthermore, UIRate utilized Grad-CAM for explainability of the model by generating a heatmap that highlights the important regions of an image for the model’s decision. In addition to the score predicted by the model, this option helps designers understand the visual components that have more effect on predictions. An example of this is provided in Figure 3, where Figure 3a is the webpage UI and Figure 3b is the highlighted version.

3. Preliminary Evaluation

3.1. Technical Validation

The model was trained on a dataset containing webpage UI images, user features, and their ratings of the webpage user interfaces. The proposed multimodal method can predict a personalized aesthetic score for webpage user interfaces. On a test set that included 15% of the dataset images that the model had not seen, UIRate achieved an MAE of 1.55 and an RMSE of 1.90.

3.2. User Feedback

For the initial evaluation of the tool, a small user study was conducted with 10 people with a background in designing, such as UI designers, user experience researchers, product designers, graphic designers, and students in this field.

This user study contains two steps. First, the designers were asked to estimate the user rate for five webpage UI, using the users’ information and the webpage UI images. The results show that the mean absolute error for this task is 1.8, indicating the complexity of personalized UI rating. To compare the model with the user study, the model was used to predict on the same webpages and user information. The results show that the model with the mean absolute error of 1.28 performs better in this small user study.

In the second step, participants use the UIRate tool to evaluate the usability and explainability of the system. The results show that most of the participants find the estimation of the personalized ratings of the model beneficial. Furthermore, the highlighted regions by Grad-CAM help understand the decision-making logic of the model. Participants found this tool valuable for early-stage design evaluation. The suggestions for the improvements include increasing the consistency of highlighted visual regions in some cases.

4. Discussion and Future Work

UIRate shows that the combination of the user features with UI images can propose a framework for personalized aesthetic prediction. The initial evaluations show that using user information in addition to webpage user interfaces can provide more insights than image-based methods. Furthermore, utilizing Grad-CAM allows a comprehensible explanation of the model’s decisions.

However, this research has limitations. The proposed model was trained on one dataset, and the user study is limited. In addition, in some cases, the participants mentioned that the highlighted regions are not understandable to them and can be improved.

In the future, this method can be trained and evaluated on various datasets, including mobile applications UI and datasets with modern UI designs. Also, a user study with more participants can provide deeper insight into the performance of the model in comparison with designers, as well as its evaluation in terms of usability.