Whether the electoral college is "fair" or not depends on whether one accepts the premise of the system that the founders established.
My understanding is that the founders tried to balance population based representation and state based representation in our system of governance. Hence, the House is population based and the Senate is state based. Then the question became how to determine the Presidency? They opted to construct an electoral college that reflected both population and state based aspects by linking the number of electors to the number of House and Senate members per state. The net effect of this imperfect compromise is that the electoral college mostly mirrors the population but diminishes the impact of larger states and magnifies the impact of smaller states. Taking 2012 election data: California cast ~ 13 million votes while Wyoming cast ~ 250,000. Thus, from a population perspective California weighs 52 times more than Wyoming. But in the electoral college California gets 55 while Wyoming gets 3; thus, California's weight is 18 times larger. One can argue whether this is "fair" but that is the system we have and it was intended that way by the founders when, at that time, Virginia was the big state all the other states were worried would have too much influence. Additionally, should the electoral college fail to provide a winner, the Constitution calls for the decision to be made in the House which is the population based body yet the House members vote as states. Using the House to elect the president has very rarely happened (1800 and 1824) but the design again reflects an attempt to balance state and population based representation.
The Constitution did not set up a pure direct democracy for the national government but rather a representative democracy: as determined by population in the House, as determined by states in the Senate, as determined by a mixture of both in the electoral college mechanism for the Presidency.