Kazunari was a son of Saji Nobukata and a nephew of Oda Nobunaga, whose favor he enjoyed. He was a vassal of Oda Nobuo, and the lord of Ôno castle in Owari province. After Nobunaga's death in 1582, Kazunari attached himself to Hashiba Hideyoshi but then threw in his lot with Oda Nobuo and Tokugawa Ieyasu in the Komaki-Nagakute Campaign. Hideyoshi responded by stripping Kazunari of his holdings. He then served Oda Nobukane and was given a 5,000 koku fief in Ise province.