We introduce and discuss a generalization of the classical multi-objective optimization to pairs of functions. This procedure is referred to as bi-multi-objective optimization. A justification of this general optimization procedure is presented, related both to multi-objective optimization under ambiguity concerning individual preferences and to Pareto optimality for a family of preferences with nontransitive indifference. Incidentally, the binary relation naturally associated to a bi-multi-objective optimization problem is represented by a finite bi-multi-utility, which generalizes to the nontransitive case the classical finite multi-utility representation. An important application is presented to Markowitz portfolio selection under ambiguity concerning both the vector of returns and the covariance matrix.