The cane sword was a weapon used by Caine in his assassin career. He also used it during his hunt for John Wick when the Marquis coerced him to return to the Criminal Underworld.
The cane was used by Caine to navigate his environment. Inside the cane is a sword that is drawn by Caine whenever he needs it.