Repeated-measures models with constrained parameters for incomplete data in tumour xenograft experiments

In cancer drug development, xenograft experiments (models) where mice are grafted with human cancer cells are used to elucidate the mechanism of action and/or to assess efficacy of a promising compound. Demonstrated activity in this model is an important step to bring a promising compound to humans. A key outcome variable in these experiments is tumour volumes measured over a period of time, while mice are treated with an anticancer agent following certain schedules. However, a mouse may die during the experiment or may be sacrificed when its tumour volume quadruples and then incomplete repeated measurements arise. The incompleteness or missingness is also caused by drastic tumour shrinkage (<0.01 cm(3)) or random truncation. In addition, if no treatment were given to the tumour-bearing mice, the tumours would keep growing until the mice die or are sacrificed. This intrinsic growth of tumour in the absence of treatment constrains the parameters in the regression and causes further difficulties in statistical analysis. We develop a maximum likelihood method based on the expectation/conditional maximization (ECM) algorithm to estimate the dose-response relationship while accounting for the informative censoring and the constraints of model parameters. A real xenograft study on a new antitumour agent temozolomide combined with irinotecan is analysed using the proposed method. Copyright (C) 2004 John Wiley Sons, Ltd.