Over on the RPG.net forums, there's a thread asking why wizards are so rare in D&D. Of course, as pointed out in the OP, the real reason is extradiegetic (out-of-character in plain English): to keep the setting from turning from faux-medieval into something quite different. But it's fun to consider in-world explanations as well, and plenty of people have pointed out some obvious, plausible reasons.
But my favorite one is by analogy with the real world, and it's so simple: why isn't everyone a programmer on Earth, in 2021? We all use computers all the time by necessity. The required knowledge and tools are free and easy to get by. It doesn't take riches, or a lot of time (though kids get started more easily), and even someone without much inclination can reach surprising levels of skill quickly enough. Yet programmers are very rare, and always in high demand. Why?
Whatever the reason, that's likely why wizards are rare in the Forgotten Realms, too. Or in my pet setting, for that matter.
(Note: we're not talking those settings where wizards are the special envoys of divinity, like Middle Earth, or need a genetic mutation like Harry Potter.)
Conversely, that begs the question of why wizards are so damn common in the Earthsea cycle, where most people know the basics well enough for practical use.
Sadly, the answer is education. We could be living in a world where writing simple scripts is a matter of basic (ha!) literacy. We actually started building that world back in the 80s. Then computing was hijacked by people who wanted these wondrous machines to be appliances, to make most of us dependent on them for software. And so they made programming into this mysterious dark art that only a few "chosen" can master, and even then only at the cost of their sanity.
Spoiler: that was all propaganda. Magic was in us all along.