Given a rational homology sphere which bounds rational homology balls, we investigate the complexity of these balls as measured by the number of 1-handles in a handle decomposition. We use Casson-Gordon invariants to obtain lower bounds which also lead to lower bounds on the fusion number of ribbon knots. We use Levine-Tristram signatures to compute these bounds and produce explicit examples.