A new flocculation model using variable fractal dimension is proposed and validated with several experimental data and an existing model. The proposed model consists of two processes: aggregation and breakup due to flow turbulence. For aggregation process, the aggregate structure is considered to have the characteristic of self-similarity, the main concept of fractal theory. Under this assumption, a variable fractal dimension instead of a fixed one adopted by previous studies is utilized here for general cohesive sediment transport. For breakup, similar concept is adopted in a more empirical manner because breakup is too abrupt to entirely apply the concept of variable fractal dimension. By a linear combination of the formulations for aggregation and breakup processes, a flocculation model which can describe the temporal evolution of floc size is obtained. Flocculation model using variable fractal dimension is capable of predicting equilibrium floc size when compared with several experimental data sets using different types of mud provided that empirical coefficients are calibrated. Through model-data comparison with Manning and Dyer (Marine Geology 160:147–170, 1999), it is also clear that some of the empirical coefficients may depend on sediment concentration. Model results for the temporal evolution of floc size are less satisfactory, despite model results shows a more smooth “S-curve” for the temporal evolution of floc size as compared with the previous model using fixed fractal dimension. The proposed model is limited to mono-size of primary particle and dilute flow condition. These other features shall be investigated as future work.