Soft spin modes and magnetic transitions in trilayered nanodisks in the vortex state