Remember the context: Jesus has just finished His final public argument with the establishment. The tension is high, and in an effort to relieve it a bit, one of the disciples makes a comment about the beauty of the temple buildings. Yet Jesus sees no beauty in it; all He sees is the coming destruction for a world that has rejected its creator that will take place on the Day of the Lord.
Verses 12-13 are a good summary of this passage:
 Because of the increase of wickedness,
will grow cold,
 but the one who stands firm
to the end
will be saved.
We live in an age where "the increase of wickedness" is easily seen. This may be, or it may not be, a sign of Christ's impending return. In any case, it should not surprise us at all that love is decreasing. Both the love that people have for one another and the love that they have for Christ and His Church is getting colder. Not just the love of some people, but the love of most people will grow cold. This is exactly what Jesus promised would happen.