Although saying this can be correct in some contexts, it can be overly direct or even rude–especially in American English.