Applications of generating functions to stochastic processes and to the complexity of the knapsack problem

The paper describes a method of solving some stochastic processes using generating functions. A general theorem of generating functions of a particular type is derived. A generating function of this type is applied to a stochastic process yielding polynomial time algorithms for certain partitions. The method is generalized to a stochastic process describing a rather general linear transform. Finally, the main idea of the method is used in deriving a theoretical polynomial time algorithm to the knapsack problem.


Introduction
Generating functions are one of the main ways of solving recursion equations. In this article we start by looking at two simple examples of solving stochastic processes with the probability generating function. Then we derive a general theorem of the generating function method for solving stochastic processes of a certain type. Next, we introduce a generating function of an unfamiliar type and show that it has relevance in calculation of partitions in a polynomial time. This result is of interest in computational complexity of some problems e.g. in cryptography. As an example, we give a polynomial time algorithm to the knapsack problem. While there are several books on generating functions (see e.g. [1] and [2]), usefulness of generating functions in deriving polynomial time algorithms seems not to have been noticed before. Probability generating functions are also used to model Queuing System [3], redundant renewable system [4], waiting Time Problems [5]. A good review of application of Probability generating functions to Stochastic Polling systems is done by Vishnevsky [6].

1.2.
Simple examples using the probability generating function Let us apply the probability generating function (pgf) to a birth-death process of a M/M/1 queue and of a loss system in Examples 1 and 2. It will be seen that these simple examples can be solved with the generating function method where pgf is the generating function.
Example 1: A process with nonempty boundary.
Let us consider the following simple birth and death process of an M/M/1 queue: In (1)  is the birth rate (calls/second),  is the death rate (calls/second), and q n s , is the probability, that the system is at the state q (has q calls in the system) at the discrete time instance n . Let Inserting i a the familiar steady state solution for state probabilities is thus obtained ( The negative side was the need to guess 1 . 1   n s but we did not need to use balance equations requiring a steady state. In addition to the state probabilities, we may be interested in a situation where each state implies a cost. Let the cost for the state q n s , be q n r , . The equation for calculating the costs gives another problem that can be solved by a generating function in the same way: In this case it is useful that the generating function method does not assume a steady state since costs are often growing.

Example 2. A teletraffic loss model.
The difference between Example 1 and the basic loss model for telegraphic (see e.g. [7][8]) is that the death rate depends on the state: and let as again take the probability generating function This gives Let us look for a solution . We obtain a differential equation: The left side in (8)

A general result of the generating function method
Let us consider the recursion equation Here i c are integers and , ,..., , , and the recursion equation is presented in the form  n q r f j , it is not always possible, or at least easy, to find a generating function as in (10). However, once there is this kind of form, we can try various analytic methods. For instance, let us assume that ) , is a function (i.e., not an operator). Let N approach to infinity. By Weierstrass inequalities holds. Let us prove a result for certain time-independent Markov processes. Theorem 1. Let (9) be a time-independent Markov process and ) , function, then the steady state probabilities can be obtained from the generating function: , . The generating function satisfies the equation Proof: Let us write (11) as where we may simplify ) , since the process is a timeindependent Markov process. Thus Let . Then the generating function approaches a steady state function ) , Equation (17) holds also if we start from ) , Solving (18) yields .
is zero in (13), there is no stationary state because in this case and the process must be constant.  ) , , are time-dependent, also in a non-Markov way.
The generating function method is of course well known: find a generating function, solve the equation with it, expand the solution into a series and pick up the terms whose powers correspond to those of . The nice thing with the method is that it works with non-stationary processes which are not even Markovian. Cost distributions are one example we already mentioned. The interesting thing with generating functions is that there probably are undiscovered types of useful generating functions for many problems. Let us look at some non-stationary processes for calculating partitions. They lead to unfamiliar looking generating functions.

3.
A generating function for calculating of partitions in polynomial time  [13]. It is possible to divide the set of partitions into two subsets: the one where each element contains a unit k and the set where no element contains this unit. The first set has a one-to-one and onto mapping into partitions of k m  while in the second set all units larger than k can be decreased by one. The latter set can be mapped one-to-one and onto to a number of lower dimensional partitions. Estimating the corresponding partitions from above and below may give a recursion equation to the original partition, and it may result to a limit theorem. There is also an interesting connection between partitions and theta functions (see [14][15] [16]), and it also may lead to limit theorems. However, if the goal is not a limit theorem but the exact result, then often there is only an exponential time algorithm that counts the partitions. In this article we show that there can be a better way: there are polynomial time algorithms for calculating the exact value of some partitions, even though the partitions grow exponentially.
Let us consider the recursion equations for the 3-dimentional Heisenberg group in [21] p.26: n q r q n r q n r q n s s s The recursion equation is solved with the generating function 2 , so it is a graph of a free group on two generators and corresponds to the hyperbolic covering space. But there are relations and in the quotient space, the covering space of the Heisenberg manifold of dimension 3, the graph of (21) grows only as a third degree polynomial (see [21] for a detailed discussion). Thus, some of the In the paper we try to see to what extent the method can be generalized. If we can find a more general recursion equation which can be solved with a generating function that generalizes (22), then we have a way to calculate directly some other partitions. As earlier work on this issue we can mention the Jacobi's identities [15] [16]. They are sum=product equations connecting generating functions and partitions.
Let us notice that in the 3-dimensional Heisenberg group there are two generators . This gives (21), see p.26 in [7]. Let us now consider a space with i generators which operate by a linear transform, not by simple multiplication from left, and we try to put many parameters that can be chosen.
are also integers). This linear transform gives a recursion equation The numbers r q n s , , satisfy the sum=product equation where b can be freely selected to get a more convenient result.

Proof: The linear transform is
i.e., the ith generator maps Let the generating function be (32) Multiplying both sides of (28) by bq aq r n q v u   2 2 1  , summing over q and r , and using the initial values (let us assume the initial values make the boundary disappear) gives The claim of the theorem follows.  Let us look at the special case, where 1   and there are only two alternative outcomes for an event: q can either increase by 1 c or decrease by 2 c  . Then the generating function can be expressed as (here This function can be easily expanded into a power series and the result gives partitions satisfying Let us note that in a general linear transform we should have where all parameters are real numbers in (28), but instead we have and all parameters are integers. These limitations were necessary in order to generalize the method (21)-(22).
As an example of the ideas in Theorem 2, we will derive a polynomial time algorithm for the knapsack problem. It has a very simple generating function. Let us notice that the recursion equation for 0  n n d j n j n j n s s s Gives the equation for the generating function (39) Consequently, we have a polynomial time algorithm for this knapsack problem with a polynomial limit on the knapsack. Clearly, the algorithm is not very useful as it is usually much worse than going through all n 2 combinations of the knapsack's numbers except for very huge numbers n . We are only interested in the existence of the algorithm.
Let us consider, what should be acceptable sequences of knapsacks for evaluating the running time of an algorithm to solve them. The argument which is used as the measure of complexity should grow in a linear manner and the running time of the algorithm is a function of this argument. The number n of elements in the knapsack is one measure of complexity. It is not a sufficient measure alone: if we can select knapsacks freely, then we can easily find a set of knapsacks growing so fast that there cannot be a polynomial time algorithm solving the knapsacks. We must include some measure of the size of the knapsack to the argument in addition to n . It is natural to use n and one of the following: We should require that the argument grows linearly. One natural interpretation of linearity is that n grows linearly and there is a polynomial bound on the other complexity measure. A polynomial bound on any of the mentioned measures gives a polynomial upper bound to the computation of the numbers j k s , as in (39). We can also obtain a polynomial algorithm for knapsacks where the numbers grow exponentially. This corresponds to taking a binary representation of one of the entities (40) and using the length in bits as a measure of complexity. This kind of measure gives another natural interpretation of linearity of the argument and it is similar to the one used in evaluating the speed of an algorithm for basic arithmetic operations, like multiplication.
Let us consider knapsacks where the numbers in the knapsacks grow faster than any polynomial, but their lengths in bits grow at most with a polynomial speed. We derive a polynomial The knapsack problem is NP complete but it is usually not clearly stated how knapsacks can be selected, one usually only talks of n as the measure of complexity [17] [18][19] [20]. If knapsacks can be completely freely selected, then taking a sequence of knapsacks where the length of the numbers grows faster than a polynomial, gives a sequence that cannot be calculated in polynomial time. Indeed, all such computations on the numbers which require all bits of the number will take a time that grows faster than any polynomial. This includes basic arithmetic operations like addition, subtraction, multiplication, division and taking modulus. In the other cases we find a polynomial time algorithm. Whether this polynomial time algorithm to the knapsack problem should be considered as a solution to the problem if P equals NP, we will leave to the reader to judge. In this paper we will not verify what kind of knapsack sequences are needed to NP completeness, nor if an acceptable algorithm should be some kind of an abstract algorithm that does not mention the sizes of the numbers in knapsacks at all. Most probably, for NP completeness we need the knapsacks that can grow exponentially but whose logarithms grow as polynomials. If this is the correct interpretation of the complexity of the knapsack then the answer probably is: P=NP, but the known polynomial time algorithm is totally inefficient.

Conclusions
The article derives a general theorem for the generating function method of solving stochastic processes. As an application of the method, a new type of a generating function for calculating a class of stochastic processes was presented. The generating function is interesting because it gives a link by which some exponentially growing partitions can be calculated exactly in polynomial time. The last example of a polynomial time algorithm to the knapsack problem may also be of some interest in the P=NP problem context. It is a purely theoretical algorithm without practical applications, but shows the idea in Theorem 2 and motivates why developing a method to derive polynomial time algorithms for partitions can have some relevance.