I also didn't have in-depth prior experiences to non-ALGOL languages all that substantively. I had experimented with Scheme, OCaml and REBOL (and read documents on Prolog, but barely typed much of it) before, but nothing fancy.
It took me about four months of hard studying for erlang to click. I think it was due to learning some CS fundamentals (e.g. had never heard of a tuple before, never seen multiple dispatch), learning functional programming fundamentals (had never used a purely functional language before - unless you count bash), and had never looked at prolog. Finally, I was simultaneously exposed to new concepts of concurrency, pattern matching, etc.
I think what happened was that the unusual syntax is the easiest thing to blame for the resulting confusion, because it's the surface layer you engage with.
You never encountered tuples in python?
I suspect there are people for whom syntax has a large impact on productivity. Especially when first getting familiar with a language. I don't think downplaying that is wise.
So, in short: syntax does matter, although in a different way than most people think.
On the other hand, people who are fixated on some syntax should just stop. Writing a couple of lexers/parsers helps here. Knowing different kinds - of flavours - of syntax helps even more. Syntax does matter, but not nearly as much as such people seem to think.
Honestly I see it as just slightly less juvenile than "Lost In Stupid Parentheses".
Whether those reasons are significant enough for you to use Elixir over Erlang are another matter, but it wouldn't be classified as just another CoffeeScript when compared honestly.
What I find far more challenging is catching up with the myriads of libraries & frameworks that are used to build real applications. Oh, and finding compatible versions of these can also be an unwanted adventure at times.