146 CHAPTER 5 Partially Distributed Learning Algorithms Proof. We write the two inequalities defined by the algorithm: f 1 (a 1,t+1 ) + (t)c(a 1,t+1 , a 2,t ) + 1,t h 1 (a 1,t , a 1,t+1 ) f 1 (a 1,t ) + (t)c(a 1,t , a 2,t ), f 2 (a 2,t+1 ) + (t)c(a 1,t+1 , a 2,t+1 ) + 2,t h 2 (a 2,t , a 2,t+1 ) f 2 (a 2,t ) + (t)c(a 1,t+1 , a 2,t ). By adding the two inequalities, one has, f 1 (a 1,t+1 ) + f 2 (a 2,t+1 ) + (t + 1)c(a 1,t+1 , a 2,t+1 ) + 1,t h 1 (a 1,t , a 1,t+1 ) + 2,t h 2 (a 2,t , a 2,t+1 ) f 1 (a 1,t+1 ) + f 2 (a 2,t+1 ) + (t)c(a 1,t+1 , a 2,t+1 ) + 1,t h 1 (a 1,t , a 1,t+1 ) + 2,t h 2 (a 2,t , a 2,t+1 ) f 1 (a 1,t ) + f 2 (a 2,t ) + (t)c(a 1,t , a 2,t ) This implies that the sequence t := f 1 (a 1,t ) + f 2 (a 2,t ) + (t)c(a 1,t , a 2,t ) is non- t increasing and has a limit. We now prove that the partial sum of the series defined by (5.18) (5.19)