I also wonder if Giles was Peter's favourite when he was younger, and he hasn't acknowledged that these days Giles is competition. The age gap might have been enough - and Giles away enough - that he was happy to look up to him before. (Later, in another book, he can watch Rowan and know he wishes she wasn't always more competent than he is.)