Sorry for the late response (for some reason eternity took a bit to show me this).

I used matrix a long time ago so I don’t remember it too well but from what I recall my reasons for leaving are:

1. lack of proper apps There’s a few apps for matrix; but if you want to use matrix as a discord replacement you are locked into either the official element app or hope a 3rd party app supports it. Additionally most clients are written using electron; which lies at the core of a lot of issues discord has.
2. voice calling This is less of a concern about matrix itself; but if you want to use it as a discord replacement VOIP is a hard requirement, discord has it’s current market share because it had a really easy to set up and use VOIP service compared to skype (which ran itself into the ground to become teams) and teamspeak (which you need to host yourself or rent a server for). Matrix does 1:1 voice calls fine (it beats 2016 discord), but group and video calls are done over jitsi which takes the app from an annoying background electron hog to a devourer of frames when you’re trying to play a game on less than ideal hardware. also because you can host a jitsi conference basically anywhere it defeats the purpose of doing one over matrix.
3. self-hosting This is something I’ve heard from others as I never hosted any of these; but this is from more than just luke smith’s video. Matrix servers are resource hogs, especially compared to the xmpp/jabber servers which I’ve heard are pretty lightweight and have the ability to integrate accounts from mastodon, lemmy, pleroma, … Do note that I don’t have personal experience on this point, so take it with a grain of salt.
4. matrix is unintuitive This is coming from somebody who has braved the discord UI for ages which is far from intuitive either; but matrix takes a special medal in my book. It’s like it’s trying to mimic slack (which discord also does); but channels and servers are mixed? The UI for element (although nice looking) is straight up terrible, settings were all over the place, and when I finally thought I’d figured something out there’s 3 more things I’d have to configure which are in totally different menus, friend and server channels are mixed with no way of separating them (unless there’s an option in a settings panel somewhere; but even I who figured out discord’s community onboarding didn’t find it) The encryption and approval process for new apps is nice, on paper… in reality it means that if you get logged out on your main session (which I found constantly happened on element) you’d be unable to read any messages before and you now had to resecure your account through one of the settings panels which I will tell you right now that no sane person will ever figure out so now all the messages they send come with a warning attached. lastly there’s the same issues you have with trying to onboard people onto mastodon or lemmy where they need to find an instance and deal with defederation; but turned up to 11 with nobody really explaining it. they also tell (suggest strongly so nobody really chooses anything else) you to make your account on the primary matrix server anyways which defeats the point of a decentralized protocol as nearly everyone is on the same instance.

1, 2, and 4 were by far my big gripes; and I probably could overcome 4 today now I’m familiar with the fediverse (which I wasn’t even a year ago) and I bet the UI has improved at least a little since my last endeavor years ago; but 1 and 2 are dealbreakers if it ever wants to pull anyone from discord, either make the official app good, or get decent 3rd party ones; discord is surviving on linux because it’s still the best option and it’s not even a decent one, voice calling also needs to be improved if it wants me; because it’s just easier to set up and host your own mumble server than get any shred of performance in matrix group calls and mumble’s VOIP implementation is nothing short of excellent.