The Fifth Hokage of Konoha and the woman Jiraiya pursued all his life. If Jiraiya were to turn evil, Tsunade would be the first to stand up and stop him. She might even defeat him without using any chakra, leveraging Jiraiya's love for her to convince him to surrender. 

The Third Hokage: Jiraiya respects his teacher immensely, so he would never accept a challenge from the Third Hokage. Even if he did, he would never harm his own master. 

Fugaku: The leader of the Uchiha clan who unlocked the Mangekyo Sharingan and possessed powerful Sharingan abilities. Although his specific abilities are not mentioned, it is clear that Jiraiya would have a hard time if he fell under Fugaku's genjutsu.

Jiraiya would need to be cautious.

Guy: Even Six Paths Madara had a hard time with Guy's Eight Inner Gates technique, so Jiraiya would be powerless against him. As long as Guy has enough time to activate the Eight Inner Gates, Jiraiya won't stand a chance. 

Orochimaru: Jiraiya has always wanted to bring back his good teammate and friend, so he wouldn't harm Orochimaru. Orochimaru can take advantage of this and defeat Jiraiya easily. One Edo Tensei from Orochimaru is enough for Jiraiya, as he is no match for him