I think Patrick's apparent lack of fear rather encourages Peter's foolhardiness. In that he thinks well, if Patrick isn't afraid when he's actually fallen off, what sort of a coward am I to be afraid when I haven't? It's established in TMAAT that Peter's a fairly long-standing adherent of the exposure theory of overcoming fear: Nicola remembering his prep-school master's comment on him insisting diving from the high board even though it would be quite reasonable for him to be wary of that. There is the Walls of Troy history to consider too: Patrick's the last person, one would imagine, that he would feel comfortable confessing his fear of heights to.