When I failed the Microsoft interview

if you look at interviews as a Test to check how good you are at studying interview questions. then things will make a lot more sense.

it also makes sense from interviewer point of view as you are filtering out anyone who could not study these things by themselves.

no one is expected to invent dijkstra's algo on the fly from first principles. but you are expected to be aware of it and competent enough to code it if the interviewer asks for it.