Sustainability is pivotal in the urban transformation strategy in order to reach more resource-efficient, resilient and smarter cities. The goal of being a sustainable city should drive the decisions for city interventions. Nonetheless, impacts need to be quantified, lacking of standard and/or common methodologies that could be replicable across multiple cities. There exist many initiatives aiming at defining indicators and assessment procedures, but without convergence in the definition of terms and application methodologies, making complex its real implementation. Within mySMARTLife project (GA#731297), a KPI-driven evaluation framework is defined with the aim of covering the multiple pillars of a city (i.e. energy, mobility, citizens, economy) in a holistic way. This methodology also defines the concepts and terms to guide urban planners and/or experts at time of implementing the framework in a specific city. The evaluation framework has been deployed in the three cities of Nantes, Hamburg and Helsinki and some lessons learnt have been extracted, such as the necessity of providing a definition of measurement boundary to avoid interpretations. Thanks to a co-creation strategy, the main difficulties and issues from the cities have been taken into consideration for increasing the replicability.