What's so special about the Lok Sabha? Why is the Budget presented there?

The Budget is always presented by the Union Finance Minister before the Lok Sabha, since the power to pass legislation on financial matters lies primarily with the lower house of Parliament, which is directly elected by the people of India.

Therefore, all financial matters, that is put before Parliament manifests itself through a Money Bill or Financial Bill, which can only originate in the Lok Sabha. The Rajya Sabha can only propose amendments to Money Bills and Financial Bills, which the Lok Sabha may or may not accept.