NRP-Exp: An Integrated Framework for Reproducible Experiment Management in Multi-Objective Release Planning

1. Introduction

Multi-objective optimisation is used across software engineering and other decision-intensive domains in which several conflicting goals must be considered simultaneously [1]. The Next Release Problem (NRP) is a representative case: a release must be assembled from candidate requirements while balancing development effort, stakeholder satisfaction, risk, and structural requirement interactions [2,3]. Although many optimisation algorithms are available, practical adoption of Search-Based Software Engineering (SBSE) remains constrained by tool maturity, process integration, and usability [4,5].

A broad range of exact, mathematical-programming, constructive, evolutionary, and stochastic approaches has been applied to different NRP formulations [6,7,8,9,10,11]. However, executing an optimisation algorithm constitutes only one stage of an NRP experiment.

Three main challenges arise in experimental and practical workflows.First, instance preparation, preprocessing, optimisation, and decision analysis are frequently performed with separate scripts and tools. Manual transfer between JSON files, solver libraries, notebooks, and spreadsheets makes it difficult to preserve consistent objective directions, constraints, parameters, and transformations. Second, many research prototypes embed domain assumptions directly in their algorithms or interfaces, making reuse in another problem family expensive. Third, generating a Pareto front does not complete the decision process. Decision makers still need to interpret trade-offs, identify structurally different alternatives, articulate preferences, and reduce a potentially large front to a manageable set of candidates [12].

Existing tools usually address only part of this workflow. These activities are frequently distributed among optimisation libraries, scripts, notebooks, spreadsheets, and visualisation or decision-support tools [13,14,15,16].

This fragmentation can hinder the reproduction and interpretation of NRP experiments. Two studies may nominally use the same instance and optimisation algorithm while applying different preprocessing operations, objective directions, constraint interpretations, stopping conditions, or quality-measure configurations. In such cases, observed differences cannot be attributed exclusively to solver behaviour. Reproducible comparison therefore requires traceability between the original instance, the transformations applied to it, the resulting optimisation-ready model, the solver configuration, and the returned solution set.

To address this need, this paper presents NRP-Exp, an integrated open Python framework for reproducible multi-objective experimentation with the Next Release Problem. NRP-Exp coordinates instance provision, domain-specific preprocessing, problem formulation, solver execution, Pareto-front assessment, and result exploration through common model, problem, solution, and result representations and analysis. The framework defines a structured JSON representation for descriptive NRP instances and supports both externally supplied and synthetically generated instances through this common format. Within NRP-Exp, this representation provides an alternative to ad hoc tabular encodings that do not explicitly preserve attribute and interaction semantics. Previously computed Pareto fronts can also be imported for independent assessment and post-optimisation analysis. JSON and synthetic instances follow the same internal workflow, allowing their transformations and optimisation results to be examined through a consistent computational process. Both internally generated and imported Pareto fronts can subsequently be visualised and analysed using the framework’s post-optimisation facilities.

NRP-Exp is designed around a clear separation between domain-specific semantics, experimental orchestration, optimisation methods, and post-optimisation analysis. Common model, problem, solution, and result representations reduce coupling between these areas, allowing domain transformations and solver implementations to evolve independently of the presentation layer. This modular and method-agnostic design supports interactive and programmatic use, facilitates the integration of additional optimisation strategies, and preserves a consistent workflow across the different stages of an NRP experiment.

The objective of this study is to present and validate NRP-Exp as an integrated experimental infrastructure for the Next Release Problem. The evaluation focuses on the architectural and computational properties claimed for the framework rather than on establishing a general performance ranking of optimisation algorithms. Specifically, it examines the ability of NRP-Exp to represent and process different NRP configurations, transform descriptive instances into traceable optimisation-ready model variants, and exchange configured problems and results across heterogeneous solver families subject to their documented applicability conditions.

The principal contributions of this work are:

1.

An integrated and headless experimental workflow that connects NRP instance provision, evaluator-dependent attribute resolution, domain-specific preprocessing, configurable multi-objective problem construction, solver execution, Pareto-front assessment, and post-optimisation analysis. The framework introduces a structured JSON representation for descriptive NRP instances, including attribute definitions, evaluator groups, requirement data, interactions, and provenance metadata. The same computational components can be invoked from the graphical interface, scripts, notebooks, and automated tests.

2.

An explicit transformation model that separates descriptive NRP instances from optimisation-ready variants. The ordered pipeline resolves evaluator aggregation, constructs transitive coupling components, remaps affected relationships, distinguishes statically absorbed from dynamically evaluated value-based interactions, and represents mutual exclusions through traceable, implication-aware branches.

3.

A common solver-integration contract that separates NRP domain semantics from algorithm-specific search mechanisms. Heterogeneous exhaustive, constructive, evolutionary, stochastic, neighbourhood-based, and mathematical-programming implementations exchange common problem, solution, and Pareto-front representations while retaining their individual parameters and documented applicability conditions.

NRP-Exp also incorporates the post-optimisation analysis facilities introduced in our previous work [12]. These facilities complete the implemented workflow by supporting the inspection and reduction of an existing Pareto front, but they are not evaluated as a separate methodological contribution in the present study.

The remainder of this paper is organised as follows. Section 2 introduces the multi-objective NRP formulation considered by NRP-Exp. Section 3 positions the framework with respect to related optimisation, release-planning, and decision-support software. Section 4 presents the architecture, common representations, instance-provision mechanisms, preprocessing pipeline, and solver-integration contracts. Section 5 describes the validation methodology and experimental configurations. Section 6 reports the end-to-end reference workflow, formulation and transformation coverage, and solver-integration results. Section 7 discusses the findings and threats to validity, and Section 8 concludes the paper.

2. The Next Release Problem

Limited development resources require software organisations to select which candidate requirements should be included in a given release. Originally introduced by Bagnall et al. [2], the Next Release Problem (NRP) has since been generalized to flexible multi-objective formulations [3].

Let $\mathbf{R}=\{r_1,\dots ,r_n\}$ denote the set of atomic candidate requirements. A candidate release is represented by a subset $\mathbf{U}\subseteq \mathbf{R}$. Each requirement may be described by attributes such as stakeholder satisfaction, development effort, risk, cost, or dissatisfaction. These attributes can be used to define optimisation objectives or operational constraints. In addition to their individual attribute values, requirements may be connected through interactions that affect release feasibility, decision representation, or attribute evaluation.

NRP-Exp considers implication, coupling, exclusion, and value-based interactions between requirements. The first three are structural interactions because they impose structural conditions on the selected requirements, whereas value-based interactions modify attribute values when specific combinations of requirements are selected:

• $\mathbf{I}=\{(r_i,r_j)\mid r_i\mathrm{implies}{r}_j\}$ represents implication interactions. Under this convention, selecting $r_i$ requires $r_j$ to be selected. This is also called precedence, denoted as $r_j$ precedes $r_i$.
• $\mathbf{J}=\{(r_i,r_j)\mid r_i\mathrm{is}\mathrm{coupled}\mathrm{with}{r}_j\}$ represents coupling relationships. Coupled requirements must be selected together and are treated as a common decision unit during preprocessing.
• $\mathbf{X}=\{(r_i,r_j)\mid r_i\mathrm{excludes}{r}_j\}$ represents mutual exclusions. Requirements connected by an exclusion cannot be selected for the same release.
• ${\mathbf{Y}}_a=\{(\mathbf{S},t,q)\mid \mathbf{S}\subseteq \mathbf{R},t\in \{\mathrm{delta},\mathrm{multiplier}\},q\in \mathbb{R}\}$ represents the set of value-based interactions affecting an attribute $a\in A_{\mathrm{req}}$. Here, $\mathbf{S}$ is the activating set, t identifies the transformation strategy, and q specifies the adjustment. The interaction transforms $a\left(\mathbf{S}\right)$ into $a\left(\mathbf{S}\right)+q$ when $t=\mathrm{delta}$, or into $qa\left(\mathbf{S}\right)$ when $t=\mathrm{multiplier}$.
• $\mathbf{V}={\bigcup }_{a\in A_{\mathrm{req}}}{\mathbf{Y}}_a$, represents the complete set of value-based interactions.

Given the preceding definitions, the multi-objective NRP consists of selecting a candidate release $\mathbf{U}\subseteq \mathbf{R}$ according to

$$\begin{array}{cc}\mathrm{maximize}\hfill & F^1\left(\mathbf{U}\right)=\left(f_1^1\left(\mathbf{U}\right),f_2^1\left(\mathbf{U}\right),\dots ,f_{z^{\prime }}^1\left(\mathbf{U}\right)\right),\hfill \\ \mathrm{minimize}\hfill & F^2\left(\mathbf{U}\right)=\left(f_1^2\left(\mathbf{U}\right),f_2^2\left(\mathbf{U}\right),\dots ,f_{z^{\prime \prime }}^2\left(\mathbf{U}\right)\right),\hfill \\ \mathrm{subject}\mathrm{to}\hfill & \mathbf{U}\subseteq \mathbf{R},\hfill \\ & f_i^1\left(\mathbf{U}\right)\ge b_i^1,i=1,\dots ,z^{\prime },\hfill \\ & f_j^2\left(\mathbf{U}\right)\le b_j^2,j=1,\dots ,z^{\prime \prime },\hfill \\ & \mathbf{U}\mathrm{satisfies}\mathbf{I},\mathbf{J},\mathbf{X},\mathrm{with}\mathrm{attribute}\mathrm{values}\mathrm{adjusted}\mathrm{by}\mathbf{V}.\hfill \end{array}$$

(1)

The functions in $F^1\left(\mathbf{U}\right)$ represent objectives to be maximised, whereas those in $F^2\left(\mathbf{U}\right)$ represent objectives to be minimised. These functions may be defined from any requirement attributes elicited for the project. For example, an experiment may maximise stakeholder satisfaction while minimising development effort or risk.

The bounds $b_i^1$ and $b_j^2$ represent project, quality, or resource thresholds associated with the corresponding attribute-derived functions. In NRP-Exp, a threshold constraint may also be defined over an attribute that is not selected as an optimisation objective. For example, effort may be used only as an upper-bound constraint while satisfaction and risk are treated as the optimisation objectives.

A release is feasible when it satisfies the declared threshold constraints and all applicable structural interactions. Value-based interactions do not necessarily determine feasibility, but they can change the objective or constraint values associated with a release. Consequently, the effective optimisation problem depends not only on the selected objective functions and bounds, but also on how the source interactions are transformed or retained before solver execution.

3. Related Work

This section positions NRP-Exp with respect to NRP optimisation methods, general-purpose multi-objective optimisation platforms, release-planning environments, and post-optimisation decision-support tools.

Since its introduction by Bagnall et al. [2], research on the Next Release Problem (NRP) has evolved from single-objective requirement selection under resource constraints to bi-objective, multi-objective, and many-objective formulations. Early formulations primarily maximised stakeholder satisfaction subject to a development-effort bound. Subsequent extensions treated satisfaction and effort as conflicting objectives [17] and incorporated further dimensions such as dissatisfaction, risk, urgency, fairness, requirement coverage, instability, and stakeholder-specific objectives [3]. This diversity has driven the application of exact methods, integer programming, evolutionary algorithms, clustering, stochastic search, and hyper-heuristics. Exact and mathematical programming methods ensure complete or progressively improving Pareto-front recovery [7], whereas evolutionary approaches produce approximate fronts for larger or more complex formulations. [17,18,19].

Requirement interactions introduce an additional source of heterogeneity. Implication rules restrict feasibility, couplings modify effective decision units, exclusions define incompatible alternatives, and value-based interactions alter attribute-derived objective or constraint values. Previous studies have represented these relationships through graphs, matrices, mathematical constraints, metamodels, and probabilistic models  [3,18]. Their operational treatment also varies: interactions may be transformed before optimisation, embedded into a solver-specific formulation, or evaluated dynamically during search. Although valid, these divergent approaches make cross-study comparisons difficult when the transition from the descriptive instance to the effective solver input is not made explicit. NRP-Exp does not introduce a new interaction taxonomy; instead, it operationalises the generic formulation proposed in our previous work [3] through an ordered and inspectable transformation pipeline.

General-purpose optimisation libraries and platforms provide reusable algorithms, problem representations, operators, benchmark problems, quality indicators, and experimental utilities. Python-oriented libraries such as pymoo, DEAP, and jMetalPy support the implementation and execution of evolutionary and other metaheuristic methods [14,15,16]. Broader environments such as jMetal, the MOEA Framework, PlatEMO, and pagmo/pygmo additionally support systematic experimentation, algorithm comparison, parallel execution, or extensible problem and algorithm interfaces [20,21,22,23]. These platforms already provide substantial infrastructure for repeatable optimisation studies. Their abstractions are nevertheless intentionally domain independent: applying them to the NRP still requires the researcher to define evaluator aggregation, interaction processing, feasibility semantics, optimisation-ready model variants, and the mapping from solver variables to requirement-level release compositions.

Release-planning environments operate closer to software-product practice. EVOLVE II and ReleasePlanner support requirements prioritisation, stakeholder participation, resource-constrained release construction, and iterative plan refinement [24]. Recent extensions incorporate interactive optimization and preference learning to keep decision-makers active during search-driven planning [25]. Other research approaches support feature assembly, clustering-based requirements selection, continuous/agile release management, or the analysis of generated release alternatives  [8,26,27].Their main purpose is normally to assist the construction or refinement of a release plan within a particular planning process. NRP-Exp has a complementary focus on configurable experimentation across descriptive instances, preprocessing configurations, objective and constraint definitions, processed variants, and compatible solver families.

Other environments address the interpretation of multi-objective results. DESDEO supports interactive optimisation and preference articulation [28], while pymcdm provides methods for ranking an existing set of alternatives [29]. Visual analytics systems such as LineUp, SkyLens, and ParetoLens support multidimensional ranking, filtering, and exploration of solution sets [30,31,32]. Our previous work introduced a method-agnostic workflow for structuring NRP Pareto fronts through Solutions of Interest and Candidate Solution Sets [12]. That work begins after a Pareto front has been generated. NRP-Exp incorporates those facilities as an optional final stage, but the present contribution concerns their integration with the preceding instance-provision, domain-processing, problem-formulation, and solver-execution stages.

As summarised in Table 1, the gap addressed by NRP-Exp is therefore not the absence of optimisation algorithms, experimentation platforms, release-planning methods, or decision-support tools considered individually. It is the need for a coordinated workflow that makes the transition from descriptive NRP data to optimisation-ready models explicit and preserves consistent NRP semantics and traceability through problem construction, solver execution, and result assessment. NRP-Exp addresses this integration requirement through common model, problem, solution, and Pareto-front representations.

4. NRP-Exp Framework

NRP-Exp is a modular Python framework that coordinates the experimental workflow from NRP instance provision to preprocessing, optimisation, Pareto-front assessment, and post-optimisation analysis. Its architecture follows five design principles: separation of concerns, semantic traceability, extensibility, consistent solver execution, and reproducible experimentation. These principles support the independent evolution of domain plugins, preprocessing operations, solver implementations, and analytical facilities while preserving common model, problem, solution, and result representations throughout the workflow.

Figure 1 presents the principal architectural areas and execution flow. Interactive, programmatic, and automated clients invoke the same headless application operations for instance loading, preprocessing, solver execution, and result construction. The presentation layer therefore remains separate from the domain transformations, optimisation algorithms, and post-optimisation components.

The application layer exposes headless use cases for instance inspection and loading, preprocessing coordination, solver execution, and result construction. Interactive and programmatic executions can therefore use the same computational components, while automated tests can verify either complete workflows or individual steps.

NRP-specific interpretation is encapsulated by the NRP plugin, which provides the attribute schema, JSON extraction, synthetic generation, serialisation, and registered preprocessing operations. Imported and generated instances are represented as OptimizationModel objects. The preprocessing engine then transforms a descriptive model into one or more processed variants, from which an OptimizationProblem is constructed by selecting objectives, optimisation directions, and threshold constraints. This separation makes the transition from source data to solver input explicit and provides the basis for semantic traceability.

The optimisation area separates search strategies from common problem evaluation and plugins. Six algorithms implement the BaseSolver contract and operate on the same OptimizationProblem representation. The common representation standardises solver integration but does not imply that every algorithm is applicable to every problem formulation. In particular, the current MILP implementation requires linear objective and constraint expressions, whereas search-based solvers can evaluate dynamically activated value-based interactions through the common solution evaluator. MILP operates on a linearised subset of the common formulation and therefore preserves equivalence only for the supported formulation elements.

Candidate solutions are interpreted through a common evaluation component responsible for objective calculation, threshold and implication constraints, dynamic value-based interactions, and feasibility assessment. Solver outputs are converted into a common ParetoFront representation containing feasible non-dominated release plans according to OptimizationProblem constraints. Distinct release compositions are preserved even when they share the same objective vector, whereas geometric metrics operate on distinct objective vectors. The assessment and analysis area operates on the resulting Pareto fronts. Front assessment provides cardinality, hypervolume, spacing, and objective-space extent as Pareto front properties.

An AnalysisSession preserves the original Pareto front while supporting enriched, framed, and otherwise derived views. The analytical facilities provide diversity-, efficiency-, preference-, indicator-, and consensus-based analyses, together with manual exploration and the construction of Solutions of Interest for subsequent inspection. This subsystem incorporates and operationalises the post-optimisation decision-space structuring functionality introduced in our previous work [12], connecting it with the preceding instance, preprocessing, problem-formulation, and solver-execution stages. These facilities complete the implemented workflow but are not evaluated as a separate methodological contribution in the present study.

The architecture supports three persistent exchange routes. JSON documents provide descriptive NRP instances, synthetic instances can be generated through the NRP plugin and serialised using the same representation, and CSV files allow previously computed Pareto fronts or selected solution sets to enter the assessment and analysis stages directly. Experimental outputs can include source and processed-model summaries, generated variant identifiers, problem and solver configurations, random seeds, objective directions, metric settings and values, and solver results.

Together, these architectural decisions provide separation of concerns, semantic traceability, extensibility, consistent solver execution, and support for reproducible experimentation. Explicit boundaries separate execution clients, application use cases, domain-specific operations, optimisation methods, and analysis components. Common representations preserve the transition from a descriptive OptimizationModel to processed model variants, configured OptimizationProblem objects, and requirement-level compositions retained in the resulting ParetoFront. Reconstruction and repeatability are supported by retaining the source instance, preprocessing configuration, selected model variant, problem definition, solver parameters, random seed when explicitly supported by the selected implementation, metric configuration, returned results, and software environment.

The current release provides a complete implementation for the NRP domain and integrates six algorithms representing heterogeneous search and mathematical-programming strategies. Solver contracts and registries, registered preprocessing operations, and analytical lenses provide the main extension points of the framework. The domain-plugin contract also allows additional problem domains to reuse the generic orchestration and result-management components. However, such cross-domain reuse is an architectural capability and has not been evaluated in the present study.

4.1. Common Problem and Result Representation

NRP-Exp uses common internal representations to connect descriptive NRP instances, configured optimisation problems, solver outputs, and derived solution sets. A descriptive instance is represented by an OptimizationModel:

$$\mathcal{M}=\left(\mathcal{I},\mathcal{A},\mathcal{E},\mathcal{R},\mathcal{D},\mathcal{G}\right),$$

(2)

where $\mathcal{I}$ is the set of decision items, $\mathcal{A}$ contains the attribute definitions, $\mathcal{E}$ contains the evaluator groups, $\mathcal{R}$ represents the interactions among items, $\mathcal{D}$ stores source and descriptive information, and $\mathcal{G}$ identifies the active domain plugin.

In the NRP implementation, decision items initially represent candidate requirements. After preprocessing, however, one decision item may represent a group of coupled requirements that must be selected together. Solvers therefore operate on the effective decision units produced by preprocessing without embedding requirement-specific assumptions in their implementations.

Attribute semantics are defined separately from item values through AttributeDefinition objects. Each definition specifies whether an attribute is scalar or evaluator-dependent (i.e. multivalued) and declares the operations used to aggregate evaluator assessments, combine values when coupled requirements are merged, and evaluate the attribute over a candidate release. Evaluator groups represent stakeholders, developers, or other assessment sources, each identified by an identifier and an associated weight. For JSON instances, individual assessments are preserved in the source representation and converted into the scalar values required by preprocessing and optimisation during model loading.

The OptimizationModel describes the available instance and its domain semantics, but it does not prescribe a particular optimisation experiment. A configured experiment is represented by an OptimizationProblem:

$${\mathcal{P}}_o=\left({\mathcal{M}}^{\prime },\mathcal{O},\mathcal{B}\right),$$

(3)

where ${\mathcal{M}}^{\prime }$ is an optimisation-ready model variant, $\mathcal{O}$ defines the selected objectives and their optimisation directions, and $\mathcal{B}$ contains the threshold constraints. Several optimisation problems can therefore be formulated from the same processed model by changing the objectives, directions, or operational bounds without modifying the descriptive source instance.

Candidate results are represented by Solution objects. Each solution records an identifier, the selected decision-item identifiers, objective values, feasibility information, and optional auxiliary values. The selected identifiers connect an objective-space point with the corresponding release composition.

A set of feasible non-dominated release compositions is represented by a ParetoFront. Release compositions are considered distinct according to their selected decision-item identifiers, so different compositions may be retained even when they share the same objective vector. The front retains the release composition and objective values required for dominance evaluation, tabular conversion, visualisation, and serialisation. Quality-assessment functions additionally require the objective names and optimisation directions of the originating OptimizationProblem to be supplied explicitly, since a Solution’s objective mapping may contain descriptive attributes beyond the configured objectives.

These representations provide consistency at three levels. First, externally supplied and synthetic NRP instances converge on the same model abstraction. Second, all integrated solvers receive the same optimisation-problem representation and common evaluation semantics. Third, optimisation and post-optimisation components exchange compatible solution, Pareto-front, and solution-set objects. Experimental differences can consequently be analysed in relation to explicit instance characteristics, preprocessing outcomes, problem configurations, and solver behaviour rather than differences in exchange formats.

4.2. Instance and Result Provision

NRP-Exp supports three entry routes serving different purposes: externally supplied NRP instances represented using the JSON structure defined by the framework, synthetically generated NRP instances, and previously computed solution sets or reported Pareto fronts imported from CSV files. The first two routes provide descriptive problem instances that can be preprocessed, formulated, and solved within NRP-Exp. The third route provides an existing result set and enters directly into the assessment and post-optimisation workflow.

As part of NRP-Exp, we define a JSON representation for the persistent exchange of descriptive NRP instances. The representation provides a common structure for model metadata, attribute definitions, evaluator groups, candidate requirements, requirement-level attribute values, and interactions among requirements. It is not intended as a general interchange standard for the NRP, but as an explicit and extensible input format for preserving the information required by the framework’s loading, preprocessing, formulation, and traceability mechanisms.

JSON extraction is delegated to the active domain plugin. The generic loader reads the document and requests the NRP plugin to construct the corresponding attribute definitions, evaluator groups, decision items, and interactions. This separation keeps the loading infrastructure independent of NRP terminology, while the plugin interprets the fields defined by the NRP JSON representation.

Externally supplied instances are not required to contain a fixed set of attributes. An instance may use the classical satisfaction and effort attributes or include additional dimensions such as cost, dissatisfaction, risk, prevalence, time sensitivity, or instability. Objectives and threshold constraints are selected later from the attributes available in the loaded model. A single descriptive instance can therefore support several optimisation configurations.

The validation cases described in Section 5 use this representation to encode different combinations of evaluator- dependent and scalar attributes, objectives, bounds, and requirement interactions

NRP-Exp also includes a synthetic generator for constructing configurable NRP instances. A generation configuration specifies the number of requirements and the mechanisms used to generate their attributes. Each requirement receives a unique identifier and an attribute mapping compatible with the selected attribute definitions. Numerical values can be sampled from configurable distributions without changing the subsequent preprocessing or optimisation components.

The generator supports manual specification of value-based interactions and the generation of implication interactions with a declared density. Generated implications are validated to preserve a directed acyclic graph. Consequently, the realised number of implications may differ slightly from the requested density when adding an interaction would introduce a cycle. Synthetic generation supports controlled experimentation rather than attempting to reproduce the full organisational and semantic complexity of an industrial release-planning project.

The third entry route imports a previously computed solution set or reported Pareto front from a CSV file. This route does not construct an OptimizationModel and therefore does not support NRP preprocessing or integrated solver execution. Instead, it allows the assessment and post-optimisation components to operate on results produced by an external optimisation system.

An imported CSV solution set contains a solution identifier, objective or descriptive columns, and optional binary columns representing solution composition. Decision columns use the req_ prefix to identify the requirements selected by each solution. When these columns are available, NRP-Exp can relate objective-space observations to requirement-level composition. Without decision columns, objective-space assessment remains possible, but requirement-composition and stakeholder-coverage analyses are limited. Comparisons between imported fronts additionally require compatible objectives, directions, scaling conventions, and quality-measure configurations. Imported results use the same ParetoFront and tabular representations as internally generated fronts, although their preprocessing and execution provenance is limited to the information supplied with the external artefact.

4.3. Ordered Preprocessing Pipeline

The need to transform requirement interactions before or during optimisation has been addressed in previous NRP research. Earlier work described how coupling relationships can be resolved by constructing common decision units and how exclusions can give rise to alternative requirement-selection problems [18]. These contributions established relevant transformation mechanisms, but they did not define or implement a complete, solver-independent preprocessing pipeline integrating evaluator aggregation, transitive coupling resolution, relationship remapping, value-based interaction processing, exclusion branching, and provenance of the resulting model variants.

NRP-Exp operationalises and extends these earlier mechanisms as an explicit ordered pipeline over a common model representation. The contribution of the pipeline is therefore not a new taxonomy of requirement interactions, but their coordinated implementation as configurable transformations that can be executed independently of the selected solver and whose intermediate and final models remain inspectable.

Let ${\mathcal{M}}^{\left(0\right)}$ denote the descriptive model before its evaluator-dependent attributes and relationships have been resolved. The ordered transformation sequence is:

$${\mathcal{M}}^{\left(0\right)}\stackrel{T_{\mathrm{agg}}}{\to }{\mathcal{M}}^{\left(1\right)}\stackrel{T_{\mathrm{coup}}}{\to }{\mathcal{M}}^{\left(2\right)}\stackrel{T_{\mathrm{val}}}{\to }{\mathcal{M}}^{\left(3\right)}\stackrel{T_{\mathrm{excl}}}{\to }\left\{{\mathcal{M}}_1^{\left(4\right)},\dots ,{\mathcal{M}}_q^{\left(4\right)}\right\}.$$

(4)

Here, $T_{\mathrm{agg}}$ denotes evaluator-dependent attribute resolution, $T_{\mathrm{coup}}$ transitive coupling resolution and relationship remapping, $T_{\mathrm{val}}$ value-based interaction processing, and $T_{\mathrm{excl}}$ implication-aware exclusion branching. The first three stages produce a single transformed model, whereas exclusion processing may produce several model variants. Each transformation consumes the output of the preceding stage, and the pipeline records the mappings and branch information required to relate the resulting decision items to the descriptive source instance.

Evaluator-dependent attribute resolution is the first conceptual transformation in this sequence. In the current implementation, it is performed by the NRP plugin during model loading, before the remaining preprocessing operations are invoked. It is included in Equation (4) because its output provides the scalar attribute values consumed by coupling resolution and the subsequent transformations.

The transformation order is fixed because the operations are not generally commutative. Evaluator-dependent attributes must be converted into scalar values before coupled items can be aggregated. Coupling must be resolved before value-based interactions and exclusions are processed because item fusion changes the identifiers referenced by those interactions. Value-based interactions must then be remapped or absorbed before exclusion branching removes items and constructs alternative subproblems. Exclusion branching is therefore applied only after the preceding transformations have produced a consistent model. This explicit ordering is part of the NRP-Exp pipeline and was not formalised as a complete executable sequence in the preceding solver-specific treatments.

Some attributes are assessed by several evaluators rather than supplied as a single scalar value [3]. Let ${\mathcal{E}}_g=\{e_1,\dots ,e_m\}$ be an evaluator group, $w_k$ the weight of evaluator $e_k$, and $v_{ik}$ the value assigned by that evaluator to requirement $r_i$. For an attribute configured with weighted-mean aggregation, the resolved value is:

$$a_i={\displaystyle \frac{{\sum }_{k=1}^m{w}_k{v}_{ik}}{{\sum }_{k=1}^m{w}_k}}.$$

(5)

The framework also supports the arithmetic mean, sum, minimum, and maximum. The operation is specified by the corresponding attribute definition and may be overridden through the loading configuration. As noted above, the current implementation performs this transformation during model loading while preserving its position as the first conceptual step of the ordered workflow.

Coupling relationships identify requirements that must be selected together. The replacement of coupled requirements by common decision units follows the transformation considered in earlier NRP work [18], while NRP-Exp generalises its implementation to transitive coupling components and attribute-specific combination rules. Because coupling is transitive, pairwise relationships are converted into equivalence classes using a Disjoint-Set Union structure.

Let $G=\{r_{i_1},\dots ,r_{i_p}\}$ be a coupling class and let $a\left(r\right)$ denote the scalar value of attribute a for requirement r. The value assigned to the unified item is:

$$a\left(G\right)={combine}_a\left(a\left(r_{i_1}\right),\dots ,a\left(r_{i_p}\right)\right),$$

(6)

where ${combine}_a$ is the coupling rule declared for the attribute. Supported rules include sum, mean, minimum, and maximum, depending on the semantics of each attribute. After fusion, references in implication, exclusion, and value-based interactions are remapped to the identifiers of the unified items.

Value-based interactions represent non-additive effects activated when a specified group of requirements is selected. An interaction is represented as:

$$d=\left(a_d,{\mathcal{R}}_d,t_d,{\varphi }_d\right),$$

(7)

where $a_d$ is the affected attribute, ${\mathcal{R}}_d$ is the activating set of requirements, $t_d$ identifies the transformation strategy, and ${\varphi }_d$ is its factor. Let $\mu \left(r\right)$ denote the mapping from an original requirement to its processed decision item. The remapped activating set is:

$${\mathcal{R}}_d^{\prime }=\left\{\mu \left(r\right)\mid r\in {\mathcal{R}}_d\right\}.$$

(8)

If $|{\mathcal{R}}_d^{\prime }|=1$, all participants have been merged into the same decision item. The interaction is then absorbed statically into that item’s affected attribute. Otherwise, the remapped interaction is retained and applied dynamically by the common solution evaluator when all participating items are selected. This distinction prevents a statically absorbed value-based interaction from being applied again during solver execution.

A mutual exclusion prevents two incompatible decision items from appearing in the same release. The interpretation of an exclusion as alternative requirement-selection problems follows the preprocessing strategy introduced in earlier NRP work [18]. NRP-Exp incorporates this strategy into the common pipeline and implements recursive, implication-aware branching, relationship updating, early pruning of already satisfied exclusions, and variant provenance. Rather than requiring each solver to implement exclusion handling independently, the pipeline constructs alternative model variants before optimisation.

Let $Dep\left(i\right)$ denote the set containing item i and every remaining item whose implication condition requires i, directly or transitively. For an unresolved exclusion between items i and j, the two branches are constructed as

$${\mathcal{M}}^{-i}=\mathcal{M}\setminus Dep\left(i\right),{\mathcal{M}}^{-j}=\mathcal{M}\setminus Dep\left(j\right).$$

(9)

Removing an item also requires the removal of every dependent item whose implication condition can no longer be satisfied. Items required by the removed item are not eliminated solely for that reason, since they may still be selected independently or be required by other retained items. Because each mutual exclusion splits the feasible decision space, the number of generated variants q grows exponentially in the worst case. Specifically, the number of alternative subproblems is bounded by $\mathcal{O}\left(2^k\right)$, where k is the number of mutually exclusive requirement pairs.

The operation is applied recursively until no unresolved exclusions remain, producing $T_{\mathrm{excl}}\left(\mathcal{M}\right)=\{{\mathcal{M}}_1,\dots ,{\mathcal{M}}_q\}.$ Relationships involving removed items are updated or discarded, and equivalent branches are identified from their remaining decision-item sets. Each retained variant receives a branch identifier that relates subsequent optimisation results to the exclusion decision that produced the variant.

This strategy makes exclusion semantics independent of the selected solver. However, the number of variants may increase rapidly with the number and structure of exclusions. The number and size of the resulting variants are therefore recorded as part of the framework evaluation.

The pipeline returns a non-empty collection of optimisation-ready models,

$$\Pi \left({\mathcal{M}}^{\left(0\right)}\right)=\{{\mathcal{M}}_1^{*},\dots ,{\mathcal{M}}_q^{*}\}.$$

If no exclusion requires branching, then $q=1$; otherwise, each model represents an alternative subproblem.

In contrast with the earlier use of individual interaction transformations within particular NRP solution methods, NRP-Exp exposes the complete sequence as a reusable application-level operation. The same processed variants can therefore be inspected, serialised, compared, and supplied to any compatible solver without reimplementing their interaction semantics inside each algorithm.

4.4. Problem Formulation and Solver Integration

After preprocessing, each model variant can be used to formulate one or more multi-objective optimisation problems. Problem formulation is independent of the selected solver so that alternative algorithms operate on the same decision items, objective definitions, threshold constraints, interaction semantics, and feasibility rules.

Let ${\mathcal{M}}^{*}$ be a processed model variant with decision-item set ${\mathcal{I}}^{*}=\{i_1,\dots ,i_n\}$. Each processed decision item is associated with a binary variable:

$$x_j=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}\mathrm{item}{i}_j\mathrm{is}\mathrm{selected},\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(10)

A candidate release is represented by the vector $\mathbf{x}=(x_1,\dots ,x_n)$. The objective configuration is defined as:

$$\mathcal{O}=\left\{(a_1,d_1),\dots ,(a_m,d_m)\right\},d_k\in \{max,min\},$$

(11)

where each pair associates an available attribute with its optimisation direction. For an attribute evaluated additively, its release-level value is:

$$f_k\left(\mathbf{x}\right)={\displaystyle \sum _{j=1}^n}{a}_{kj}{x}_j+{\Delta }_k\left(\mathbf{x}\right),$$

(12)

where $a_{kj}$ is the scalar value of attribute $a_k$ for item $i_j$, and ${\Delta }_k\left(\mathbf{x}\right)$ represents the effects of dynamically retained value-based interactions. Value-based interactions absorbed during preprocessing are already reflected in the item attributes and are not applied again. For attributes configured with non-additive evaluation rules, such as the maximum or mean, the common evaluator applies the corresponding rule instead of the summation in Equation (12).

Threshold constraints define admissible bounds over attribute-derived functions. For example, a maximum-effort constraint is expressed as:

$$f_{\mathrm{effort}}\left(\mathbf{x}\right)\le B_{\mathrm{effort}}.$$

(13)

A constrained attribute need not also be an optimisation objective. An experiment may, for example, maximise satisfaction and minimise risk while using effort only as an upper-bound constraint.

Structural feasibility is evaluated separately from objective quality. A candidate release is feasible when it satisfies all threshold constraints and remaining implication interactions. If item $i_j$ requires item $i_k$, selecting $i_j$ is feasible only when $i_k$ is also selected. Coupling and exclusion interactions require no solver-specific treatment at this point because they have already been resolved through item fusion and model branching, respectively.

A common SolutionEvaluator reconstructs the selected decision-item identifiers, checks threshold constraints, applies dynamically retained value-based interactions, and calculates the configured objective values. Implication feasibility is enforced by the corresponding solver implementation through the common precedence semantics. Search-based solvers use this component directly, whereas solver-specific mathematical formulations encode their supported objective and constraint expressions internally. This separation allows new solvers to reuse the common problem, evaluation, solution, and result representations according to their formulation requirements.

This separation provides the principal extension boundary for optimisation methods. A new search-based solver can reuse the common candidate representation and evaluation component, whereas a solver requiring a specialised mathematical formulation can translate the supported elements of the same OptimizationProblem into its internal representation. In both cases, the implementation declares its parameters and and returns its results through the common solution and Pareto-front abstractions, .

Every optimisation algorithm implements the BaseSolver contract. A configured solver receives an OptimizationProblem and returns a ParetoFront:

$$solve({\mathcal{P}}_o,\theta )⟶\mathcal{F},$$

(14)

where $\theta$ contains the solver-specific parameters and $\mathcal{F}$ is the returned Pareto front. Solver-specific parameters may include a random seed when the corresponding implementation explicitly supports one. Parameters are declared through ParameterSpec objects containing their names, types, admissible ranges, and default values. The Streamlit interface uses these declarations to construct solver controls, while scripts and notebooks can provide the same configurations programmatically.

Solver discovery is managed through a registry. Consequently, incorporating an additional solver requires the implementation of the common contract, the declaration of its parameters and formulation requirements, and its registration with the framework. It does not require algorithm-specific execution or configuration logic to be added to the presentation layer. The solver remains responsible for determining whether the supplied problem falls within its declared scope and for translating the supported problem elements into any internal representation required by its search or mathematical-programming strategy.

The current NRP-Exp release integrates six optimisation algorithms through the common solver contract. These implementations cover exhaustive, constructive, mathematical-programming, evolutionary, random-sampling, and neighbourhood-based search strategies. They exchange the same problem and result representations, while retaining their individual search mechanisms, configuration parameters, stopping conditions, applicability requirements, and optimality guarantees. Their integration illustrates that the extension mechanism is not restricted to a single algorithm family. Table 2 summarises their roles, limitations, and representative parameters.

All returned candidates are converted into the common Solution representation, and feasible non-dominated solutions are retained in a ParetoFront. The front preserves the selected decision-item identifiers and objective values required to connect objective-space observations with release composition. Distinct release compositions are retained even when they share the same objective vector, whereas geometric quality measures operate on distinct objective vectors.

Solver results are comparable only when they originate from semantically equivalent experimental configurations. This equivalence includes the processed model variant, objective definitions and directions, threshold constraints, interaction semantics, and relevant solver budgets. Differences between fronts cannot be attributed solely to algorithm behaviour when any of these elements differ.

The framework computes front cardinality, hypervolume, spacing, and objective-space extent. Objective names and optimisation directions are obtained from the originating OptimizationProblem and supplied explicitly to the assessment functions. Hypervolume comparison additionally requires a common objective transformation and reference point; if normalisation is applied, the same bounds must be used for every compared front. The measures are interpreted jointly because they describe complementary properties of the size, coverage, extent, and distribution of the returned alternatives. Execution time is also recorded, but is treated as environment dependent and is not considered a reproducibility invariant.

The production of a ParetoFront does not terminate the implemented workflow. Internally generated fronts can be transferred directly to an AnalysisSession, while externally computed fronts can enter the same assessment and analysis stages through CSV import. The session preserves the original front and supports enrichment, contextual framing, analytical lenses, the construction and combination of Solutions of Interest, and the selection of Candidate Solution Sets for detailed comparison.

This post-optimisation subsystem operationalises the visual decision-space structuring functionality introduced in our previous work [12]. In NRP-Exp, that functionality is incorporated as a continuation of the instance-provision, preprocessing, problem-formulation, and solver-execution workflow. The analytical methodology itself is not re-evaluated or claimed as a new contribution in the present study. Its inclusion demonstrates that the common solution and Pareto-front representations can connect optimisation results with subsequent decision-support activities without discarding their objective values, requirement-level compositions, or available experimental provenance.

5. Validation Methodology

The validation examines whether NRP-Exp provides an integrated and traceable infrastructure for constructing, transforming, executing, and inspecting multi-objective NRP experiments. Its purpose is to assess the functional scope of the implemented framework rather than to compare optimisation algorithms or establish general scalability bounds.

Three complementary validation stages are considered. First, the Concise_Case_11 workflow demonstrates the complete transition from JSON ingestion to post-optimisation inspection. Second, a collection of NRP cases examines whether the common representations and preprocessing operations can accommodate different numbers of requirements, objective dimensionalities, resource bounds, and interaction structures. Third, selected formulations are supplied to compatible solver families to assess problem acceptance, feasibility, result construction, and formulation-specific applicability restrictions.

Table 3 summarises the formulation profiles included in the validation. Requirement counts refer to the granularity supplied to NRP-Exp before preprocessing. The selected cases cover scalar and evaluator-dependent attributes, two- and three-objective formulations, resource bounds, and different combinations of implications, coupling, exclusion, and value-based interactions. The controlled cases complement the externally established instances previously used [3]. Concise_Case_11 and Long_Case_30 exercise exclusion branching and value-based interaction processing, while Controlled_Linear_6 provides an additive formulation compatible with all six integrated solver implementations.

Table 4. Configurations used for the selected solver executions.

Case	Objectives	Bound	Solver	Parameters
Motorola	Sat. (max), effort (min)	Eff. $\le 1415$	NSGA-II	$N=80$, $G=150$
Classic	Sat. (max), effort (min)	Eff. $\le 43$	Pruned Brute Force	$E_{max}={10}^6$
Classic-100	Sat. (max), effort (min)	Eff. $\le 519$	NSGA-II	$N=100$, $G=150$
WordProc-50	Value (max), urgency (max), effort (min)	Eff. $\le 725$	NSGA-II	$N=80$, $G=150$
Concise_Case_11	Sat. (max), effort (min)	Eff. $\le 35$	Pruned Brute Force	$E_{max}=30000$
Long_Case_30	Sat. (max), effort (min)	Eff. $\le 85$	NSGA-II	$N=80$, $G=150$

For Concise_Case_11, the selected execution uses the Excl(r7) variant. For Long_Case_30, the selected execution uses variant 0, corresponding to Excl(r9) → Excl(r17). Parameters not shown in the table retain their solver defaults.

Table 4. Configurations used for the selected solver executions.

Case	Objectives	Bound	Solver	Parameters
Motorola	Sat. (max), effort (min)	Eff. $\le 1415$	NSGA-II	$N=80$, $G=150$
Classic	Sat. (max), effort (min)	Eff. $\le 43$	Pruned Brute Force	$E_{max}={10}^6$
Classic-100	Sat. (max), effort (min)	Eff. $\le 519$	NSGA-II	$N=100$, $G=150$
WordProc-50	Value (max), urgency (max), effort (min)	Eff. $\le 725$	NSGA-II	$N=80$, $G=150$
Concise_Case_11	Sat. (max), effort (min)	Eff. $\le 35$	Pruned Brute Force	$E_{max}=30000$
Long_Case_30	Sat. (max), effort (min)	Eff. $\le 85$	NSGA-II	$N=80$, $G=150$

For Concise_Case_11, the selected execution uses the Excl(r7) variant. For Long_Case_30, the selected execution uses variant 0, corresponding to Excl(r9) → Excl(r17). Parameters not shown in the table retain their solver defaults.

The cases were selected to exercise complementary framework capabilities and do not constitute a statistically representative NRP benchmark. Each case is subjected to the operations required by its validation role rather than being executed with every available solver. A formulation is considered operationally supported when NRP-Exp constructs its descriptive model, applies the required preprocessing operations, formulates an OptimizationProblem, executes at least one compatible solver, and preserves the decision-item composition in the returned results.

The assessment records original and processed decision-item counts, generated model variants, remaining interactions, absorbed and dynamically retained value-based interactions, nominal binary-space sizes, and result feasibility. Solver integration is assessed through registry discovery, parameter-schema exposure, problem acceptance, successful termination, preservation of decision-item composition, and conversion to the common ParetoFront representation.

Preprocessing and execution times, raw outputs, distinct release compositions, and distinct objective vectors are reported as descriptive observations rather than as comparative performance measures. Each execution retains its source instance, preprocessing configuration, selected variant, objectives and directions, threshold constraints, solver-specific parameters, any random seed explicitly supported by the selected implementation, software environment, and generated results.

The Controlled_Linear_6 case reproduces the six-item formulation used by the solver-integration script. It maximises value, minimises cost, and limits total cost to 24. The objectives and constraint are additive, and the case contains no dynamic value-based interactions, making it compatible with all six integrated solvers, including the current MILP implementation. Its complete space of $2^6=64$ selections is enumerated to obtain an exact reference front for checking the common problem and result contracts.

The stochastic implementations were executed once using the retained solver-specific configurations. The current validation does not control or repeat the stochastic executions across random seeds and therefore does not assess stochastic stability. Their returned fronts are treated only as functional outputs of the corresponding integration checks.

6. Results

The results follow the three complementary aspects of the validation. First, the illustrative case demonstrates the complete end-to-end workflow. Second, the selected NRP cases examine formulation and transformation coverage. Third, solver integration is assessed by executing each NRP formulation with a compatible solver and by exchanging a common controlled formulation among all six integrated implementations.

6.1. Illustrative End-to-End Example

The example uses the Concise_Case_11 JSON instance distributed with NRP-Exp. It contains 12 candidate requirements assessed by two stakeholders and two developers, together with 11 implication relationships, one coupling, one mutual exclusion, and three value-based interactions. The available attributes are satisfaction, dissatisfaction, risk, and effort. Satisfaction and dissatisfaction are assessed by stakeholders, risk is assessed by developers, and effort is represented as a scalar requirement attribute. Figure 2 shows the principal interface views involved in the workflow.

The JSON instance was loaded through the NRP plugin, which resolved its evaluator-dependent attributes and constructed the descriptive OptimizationModel. The resulting model was then processed through the registered NRP pipeline.

Preprocessing merged the coupled requirements $r_4$ and $r_5$ into the decision item r4_r5. Processing the mutual exclusion between $r_7$ and $r_9$ then produced two alternative model variants. The Excl(r7) variant retained six decision items and two implication relationships, whereas the Excl(r9) variant retained nine decision items and five implication relationships. Neither variant contained unresolved coupling or exclusion relationships. The former retained two value-based interactions for dynamic evaluation, while the latter retained three. These structural outcomes can be inspected in Figure 2(a).

For the reference execution, the Excl(r7) variant was selected. Stakeholder satisfaction was maximised, development effort was minimised, and total effort was constrained to 28. The processed problem contained six binary decision items and therefore had a nominal space of $2^6=64$ selections. This complete space was explored within the configured limit of 30,000 evaluations, so the returned front is exact for the selected variant and problem configuration.

Table 5. Configuration and outputs of the illustrative example.

Property	Value
Software version	v1.0.0
Instance	Concise_Case_11
Selected model variant	Excl(r7)
Active decision items	6
Nominal binary search space	$2^6=64$
Remaining implication relationships	2
Dynamic value-based interactions	2
Objectives	Satisfaction (max), effort (min)
Constraint	Effort $\le 28$
Solver	Pruned Brute Force
Solver parameters	max_evaluations = 30000
Feasible non-dominated release plans	25
Distinct objective vectors	18
Hypervolume reference point	$(1.04,30.80)$ in transformed minimisation space
Hypervolume	214.46
Spacing	0.47
Objective-space extent	29.88
Preprocessing time	$<0.01$ s
Solver execution time	$<0.01$ s
Lens	Diversity K-medioids
Attributes	productivity, squandering, dirtiness, annoyance, risk
Num. Cluster	4 (Silhouette: 0.496)

Table 5. Configuration and outputs of the illustrative example.

Property	Value
Software version	v1.0.0
Instance	Concise_Case_11
Selected model variant	Excl(r7)
Active decision items	6
Nominal binary search space	$2^6=64$
Remaining implication relationships	2
Dynamic value-based interactions	2
Objectives	Satisfaction (max), effort (min)
Constraint	Effort $\le 28$
Solver	Pruned Brute Force
Solver parameters	max_evaluations = 30000
Feasible non-dominated release plans	25
Distinct objective vectors	18
Hypervolume reference point	$(1.04,30.80)$ in transformed minimisation space
Hypervolume	214.46
Spacing	0.47
Objective-space extent	29.88
Preprocessing time	$<0.01$ s
Solver execution time	$<0.01$ s
Lens	Diversity K-medioids
Attributes	productivity, squandering, dirtiness, annoyance, risk
Num. Cluster	4 (Silhouette: 0.496)

The execution returned 25 distinct feasible non-dominated release compositions corresponding to 18 distinct objective vectors. Different release compositions can therefore share the same satisfaction and effort values. NRP-Exp retains these decision-space alternatives for requirement-level inspection, while geometric measures operate on distinct objective vectors. Figure 2(b) shows the configured problem, solver, and returned front in tabular form; the same front can also be inspected as a decision map. The assessment operations were applied to the 18 distinct objective vectors. Satisfaction was transformed to minimisation by changing its sign, while effort retained its original minimisation direction. Let the transformed nadir point be $z^{\mathrm{nadir}}$ and the observed objective range be R. The reference point $z^{\mathrm{ref}}$ is calculated as $z^{\mathrm{ref}}=z^{\mathrm{nadir}}+0.10R$. For this execution, the transformed nadir was $(0.94,28.00)$ and the range vector was $(1.0,28.0)$, yielding the explicit reference point $(1.04,30.80)$. Using this reference point, the front produced a hypervolume of 214.46, a spacing value of 0.47, and an objective-space extent of 29.88.

These values characterise the illustrative front and are not used for comparison with other algorithms or configurations. These measures are reported only to illustrate the assessment facilities integrated into the framework.

The front can subsequently be transferred to an AnalysisSession, which preserves the original result while allowing derived views to be enriched, framed, and processed through analytical lenses. In this example, the available attributes support indicators such as scope, productivity, squandering, dirtiness, and annoyance. Diversity and preference analyses can then produce complementary Solutions of Interest for requirement-level examination, as illustrated in Figure 2(c).

Figure 2(d) shows the application of an analytical lens to a decision map. The accompanying violin plots and box plots support inspection of the distributions, ranges, central tendencies, and variability of the attribute values associated with the analysed solutions.

As an integrity check, the reference workflow was executed three times using the same retained instance, configuration, and software environment. Every execution generated the same two processed variants, selected the Excl(r7) branch, and returned the same 25 release compositions and 18 objective vectors. The canonical representations of the loaded model, processed variants, selected branch, problem configuration, and Pareto-front contents were identical across executions. Preprocessing and solver times were below 0.01 s in every run and were excluded from the equality criterion. This check evaluates workflow reconstruction under the same software environment rather than cross-platform or stochastic reproducibility.

The instance, configuration, processed variants, results, and executable notebook required to reproduce the example are available in the NRP-Exp repository at https://github.com/imaguila/NRPExperimentFramework. An interactive deployment of the interface used to produce Figure 2 is available at https://nrpexperimentframework-nuwdjbrgrcwq3qzhbg2a6d.streamlit.app/. The notebook invokes the same headless loading, preprocessing, formulation, solver, and analysis components used by the application.

6.2. Formulation and Transformation Coverage

The second validation stage examines whether the same model, preprocessing, and problem-construction components can accommodate the selected NRP profiles. All six descriptive instances were loaded and processed successfully. Using the retained objective-and-bound configuration for each source case, an OptimizationProblem was constructed for every resulting model variant. The six instances produced nine configured optimisation problems in total: one each for Motorola, Classic, Classic-100, and WordProc-50, two for Concise_Case_11, and three for Long_Case_30.

Table 6 summarises the structural effects of preprocessing. Motorola required evaluator aggregation but no structural transformation, so its 35 candidate requirements remained independent decision items. Coupling resolution reduced Classic from 20 requirements to 18 decision items, Classic-100 from 100 to 96, and WordProc-50 from 50 to 42. For these instances, the coupling components induced by the source relationships reduced the effective decision spaces by two, four, and eight items, respectively. These reductions result from the connected components formed by the coupling relationships and should not be interpreted as a general one-to-one correspondence between relationship counts and removed decision items.

The affected implication relationships were remapped, leaving seven relationships in Classic, 37 in Classic-100, and 63 in WordProc-50. No processed model retained unresolved coupling or exclusion interactions.

Among the externally established cases, WordProc-50 contains the largest combination of implication, coupling, and value-based interactions. Its processed model retained seven value-based interactions for dynamic evaluation. The corresponding problem maximises value and urgency, minimises effort, and applies the configured effort constraint.

Exclusion processing generated two variants for Concise_Case_11 and three variants for Long_Case_30. The long-case variants contained between 20 and 22 active decision items, representing reductions between 26.67% and 33.33%. Two value-based interactions remained available for dynamic evaluation in the variants associated with the Excl(r9) branches, while three remained in the Excl(r10) variant.

The nominal binary spaces ranged from $2^6$ for the smallest Concise_Case_11 variant to $2^{96}$ for Classic-100. These values describe the binary representations of the processed problems, not their numbers of feasible releases. Implication relationships and threshold constraints further restrict the feasible spaces. Preprocessing completed for all cases. The largest observed preprocessing time remained below 0.01 s and corresponded to the three variants of Long_Case_30. This timing describes only the evaluated execution and does not support a general claim about preprocessing efficiency or scalability.

6.3. Solver Integration and Portability

The third validation stage examines solver integration at two complementary levels. The first check determines whether each selected NRP formulation can be executed by at least one solver whose declared scope is compatible with its processed structure. The second check supplies the same controlled formulation to all six implementations to isolate common problem and result exchange from formulation-specific applicability restrictions.

Table 7 reports the first check. Solver selection differs across cases because the processed formulations have different effective search-space sizes and interaction semantics. Consequently, the reported times and returned-set cardinalities are not comparable measures of algorithm performance. Controlled_Linear_6 was intentionally designed as a minimal formulation exercising the common contracts under conditions supported by all integrated solvers. It should therefore be interpreted as an integration benchmark rather than as a representative NRP instance.

All six executions accepted the configured problem, terminated normally, and returned a non-empty ParetoFront. Every retained solution was marked as feasible by the common evaluation process, and its selected decision-item identifiers were preserved in the result representation. These executions therefore establish operational applicability for the evaluated case-solver pairs, not compatibility between every formulation and every solver.

Motorola returned 35 distinct release compositions corresponding to 33 objective vectors, while Concise_Case_11 returned 25 compositions corresponding to 18 vectors. These results provide two examples in which different releases share the same objective-space position and confirm that the result representation does not remove compositionally distinct alternatives solely because their objective values coincide.

The portability check uses Controlled_Linear_6, a six-item additive formulation that maximises value, minimises cost, and constrains total cost to 24. Its complete space of $2^6=64$ selections can be enumerated, and its linear formulation lies within the declared scope of all six integrated implementations, including MILP. The case therefore provides a common problem for examining registry discovery, parameter exposure, execution, feasibility, composition preservation, and result construction.

For this controlled check, integration is considered successful when a solver is discovered through the registry, exposes its parameter schema, accepts the configured problem, terminates normally, returns a non-empty set of feasible solutions, preserves decision-item composition, and exposes its results through the common ParetoFront representation. The number of exact reference vectors recovered is reported as an additional observation and is not part of the integration criterion.

Complete enumeration produced 18 distinct release compositions corresponding to 17 objective vectors. Table 8 reports the results obtained when the same controlled formulation was supplied to the six integrated solvers.

All six implementations passed the defined integration checks. Each solver was discovered through the registry, exposed its declarative parameter schema, accepted the same configured OptimizationProblem, terminated normally, returned at least one feasible solution, preserved decision-item composition, and exposed its results through the common ParetoFront representation.

Complete enumeration identified 18 distinct release compositions and 17 objective vectors. Reference-vector recovery varied under the retained solver-specific configurations, from one vector for the Greedy and Simulated Annealing executions to 16 vectors for NSGA-II and 17 for complete enumeration. These cardinalities characterise only the reported configurations. The stochastic implementations were not repeated under controlled seeds, and the methods used different search mechanisms, parameters, and effective budgets. The observed recovery values therefore provide no basis for a general comparison of solver quality or stability.

Across the three stages, the reference workflow was reconstructed without variation, the six NRP source cases produced nine optimisation-ready model variants and corresponding problem configurations, every selected NRP case was executed with a compatible solver, and all six integrated solver implementations passed the controlled portability check.

7. Discussion

The validation supports the central integration claim of NRP-Exp at three levels. The end-to-end execution shows that the descriptive instance, processed variants, configured problem, solver output, and post-optimisation views can be connected without introducing a separate exchange representation between stages. This connection is particularly relevant when different release compositions share an objective vector. By preserving the selected decision-item identifiers, NRP-Exp retains decision-space alternatives that would be lost through objective-space deduplication alone.

The case-based transformation results indicate that the common model and preprocessing pipeline are not specific to the illustrative instance. The six evaluated cases exercise evaluator-dependent and scalar attributes, two- and three-objective configurations, implication and coupling relationships, mutual exclusions, and value-based interactions. The resulting nine variants show that the same transformation sequence can produce either a single optimisation-ready model or several traceable subproblems. This provides functional coverage for the selected profiles, but does not establish completeness with respect to all possible NRP formulations.

The observed reductions also clarify the role of preprocessing. Coupling resolution changes the effective decision units rather than merely adding feasibility checks to the original binary variables. Exclusion processing changes the number of optimisation problems by constructing alternative variants. In the evaluated cases, the single exclusion in Concise_Case_11 produced two variants, while the three source exclusions in Long_Case_30 produced three distinct variants after implication-aware removal and recursive exclusion processing. Dynamically retained value-based interactions, in contrast, remain part of solution-time evaluation.

The solver results distinguish operational applicability from portability. Execution of each NRP case with one compatible solver shows that the processed formulations can be consumed through the common problem and result representations. It does not show that every solver supports every formulation. The controlled linear case isolates the narrower portability claim: when a formulation lies within the declared scope of all six implementations, each solver can be discovered, configured, executed, and inspected through the same contracts. Reference-vector recovery is separate from this integration criterion and depends on the algorithm, parameters, and computational budget.

The validation also exposes boundaries in the current implementation. The MILP solver supports the encoded linear formulation but does not evaluate dynamically activated value-based interactions. Exclusion processing can increase the number of variants rapidly, and the present cases do not characterise that growth systematically. In addition, the current threshold representation is scalar.

7.1. Threats to Validity

7.1.1. Construct and Internal Validity

Workflow integrity was assessed by comparing canonical representations of the descriptive model, processed variants, problem configuration, and Pareto-front contents across three executions. Equality of these artefacts demonstrates deterministic reconstruction within the evaluated environment, but does not by itself establish that the encoded instance or transformation semantics are correct. Semantically equivalent artefacts could also receive different hashes if their canonicalisation retained non-substantive differences.

The validation instances were reconstructed from JSON files, tabular data, spreadsheets, and previously published cases. Errors in attribute transcription, evaluator weighting, interaction direction, or resource-bound adaptation could affect the resulting formulations. To mitigate this risk, source counts and preprocessing outcomes were checked against the retained case descriptions and intermediate artefacts. Nevertheless, the validation does not constitute an independent revalidation of the original datasets.

Solver integration was assessed through registry discovery, parameter exposure, problem acceptance, normal termination, output feasibility, decision-composition preservation, and common result construction. These criteria assess software integration rather than optimisation quality. Likewise, equality with the enumerated reference front is informative only for the two small configurations whose complete spaces were explored.

The stochastic implementations were executed once with the retained solver-specific parameters. The current implementations do not expose a uniform random-seed mechanism across NSGA-II, Random Search, and Simulated Annealing. Their returned fronts may therefore change between executions, as well as under different parameter values or stopping conditions. Since the study was designed to assess framework integration rather than algorithm performance, no statistical comparison, stability assessment, or general performance conclusion is drawn.

7.1.2. External and Reproducibility Validity

The case portfolio contains six NRP profiles with between 12 and 100 source requirements, two or three objectives, and different combinations of evaluator-dependent attributes, implication and coupling relationships, mutual exclusions, and value-based interactions. This diversity provides case-based evidence that the common representations and preprocessing operations apply beyond the illustrative instance. The cases are deliberately selected, however, and do not constitute a statistically representative sample of NRP formulations.

Only one objective-and-bound configuration was retained for each source case, and the study does not vary instance size, interaction density, exclusion structure, evaluator distribution, or resource tightness systematically. Consequently, the returned-set cardinalities are descriptive results for the retained configurations and are not comparable measures of algorithm performance.

A compatible solver was executed for every selected NRP case, while Controlled_Linear_6 was used to examine all six implementations under a common supported formulation. The stochastic executions used one retained parameter configuration and algorithm-specific budgets, but were not repeated under controlled random seeds. Their times, returned-set cardinalities, and reference-vector recovery therefore describe only the reported executions.

The retained source instances, problem and solver configurations, intermediate artefacts, canonical hashes, and results support independent repetition, but equivalent behaviour across operating systems, Python versions, dependency versions, or hardware platforms was not tested. The evidence therefore establishes same-environment repeatability rather than cross-platform reproducibility.

8. Conclusions

This paper presented NRP-Exp, an integrated Python framework for traceable multi-objective experimentation with the Next Release Problem. The framework connects instance provision, evaluator aggregation, interaction-aware preprocessing, problem formulation, solver execution, Pareto-front assessment, and post-optimisation analysis through common representations and reusable headless components.

The objective of the validation is not to demonstrate optimisation improvements over existing frameworks, but to evaluate whether NRP-Exp preserves traceability across instance transformation, problem formulation, solver execution and result analysis. Consequently, the reported evidence should be interpreted as a framework-validation study rather than as an algorithm-comparison experiment.

The validation provided evidence for the evaluated configurations at three levels. The Concise_Case_11 workflow connected the complete experimental sequence and was reconstructed three times without variation in its descriptive model, processed variants, configured problem, or exact Pareto front for the selected variant. Six NRP profiles containing between 12 and 100 source requirements were loaded and transformed into nine traceable model variants, each of which supported construction of the retained problem configuration. Each selected NRP case was also executed with a compatible solver. Finally, all six integrated implementations accepted and returned results for the same controlled linear formulation through the common problem, solution, and Pareto-front contracts.

These findings support NRP-Exp as an integrated and traceable infrastructure for constructing, transforming, executing, and inspecting configured NRP experiments. The results demonstrate functional coverage of the selected formulation profiles and common solver exchange under compatible conditions. They do not establish exhaustive formulation coverage, general scalability, cross-platform reproducibility, stochastic stability, or a performance ranking of the integrated algorithms.

Future work will examine systematic instance families and interaction densities, comparable solver budgets, uniform random-seed support, and repeated stochastic executions, multidimensional resource constraints, cross-platform reconstruction, and the practical use of the integrated post-optimisation facilities.