If you want to get real deep into the lore about it, and I’m talking pre-Disney, this was the case with the Jensaarai, who were caught between the Jedi and the Sith and whose neutrality neither group recognized. We could also point to the madters of teras kazi who were similarly caught.

In both cases, the Jedi Order’s persecution of the group led to their antagonism. Arden Lyn is a great example as Jedi persecution led her to ally initially with Vader snd then strike out on her own shortly thereafter. This is what I mean by colonialism on behalf of the Jedi order.