Mercury
The Idea of colonizing Mercury is hard to achieve as it requires more fuel to go to mercury than to escape Solar system!
Mercury being the closest planet to sun, has a surface temperature about 450℃ during midday and -180℃ during night.The Probes that have gone there had elaborated system of protecting themselves from heat. Protecting humans from heat will be difficult.
Mercury doesn't have atmosphere and it is constantly under exposure to solar wind, which possess high radiation.
The another big problem of colonizing mars would be "Why anyone would want to?" There's no good reason in doing the same. Probes missions can be done better than any manned exploration.
So, it is easier to rather colonize mars than mercury which world is today looking forward to do so.
Aniket Kulkarni
Footnote :
Image source : Mercury image