Sampled based estimation of network traffic flow characteristics

In this paper, we consider the problem of non-parametric estimation of network flow characteristics, namely packet lengths and byte sizes, based on sampled flow data. We propose two different approaches to deal with the problem at hand. The first one is based on single stage Bernoulli sampling of packets and their corresponding byte sizes. Subsequently, the flow length distribution is estimated by an adaptive expectation-maximization (EM) algorithm that in addition provides an estimate for the number of active flows. The estimation of the flow sizes (in bytes) is accomplished through a random effects regression model that utilizes the flow length information previously obtained. A variation of this approach, particularly suited for mixture distributions that appear in real network traces, is also considered. The second approach relies on a two-stage sampling procedure, which in the first stage samples flows amongst the active ones, while in the second stage samples packets from the sampled flows. Subsequently, the flow length distribution is estimated using another EM algorithm and the flow byte sizes based on a regression model. The proposed approaches are illustrated and compared on a number of synthetic and real data sets.