The answer to this is a bit complicated. The Earth rotates daily on its axis, and it also circles the Sun, once per year. The position of the Sun at solar noon, the moment of highest elevation in the sky for that location, moves around the globe on these two axes and in relation to the Earth’s rotational axis.
The Sun rises due east on only two days of the year, on the spring and fall equinoxes (around March 21 and September 22). On these days, the Sun is exactly above the celestial equator. At other times of the year, the Sun rises north or south of due east.
At the Tropic of Cancer and the Tropic of Capricorn, it never rises more than 23 degrees above or below the horizon at solar noon. Those are the only latitudes where it can do that.
It’s a bit easier to understand this in the tropics, where the Sun is always above the horizon at solar noon. This is why it’s sometimes possible to see the Sun’s rays falling on a person without a shadow cast at all. This is called the sub-solar point and it varies through the tropics throughout the year.
There are, of course, many other places on the planet where it’s impossible to have the Sun directly overhead at noon. This is because the Sun can only be above or below your location’s meridian at solar noon. To figure out what your location’s meridian is, you can use an almanac or a GPS and find the longitude of your locale.