Leading events, I couldn't imagine playing with a controller while also trying to type to my party. I often throw out information between spells and abilities.
But that's the only serious disadvantage I can see from playing with a controller. If everyone in the party are using headsets it doesn't matter.
FFXI is the only game I play full-time with a keyboard because that's what I started with twenty years ago and got used to it. I don't use gearswap or anything like that, and can use commands via macros, typing it manually, or by doing it straight from the menu. And that's all it boils down to. Adapting and improving on what you use.