If a rock were big enough or moving fast enough it could impact and tear off a big section of crust and you'd be fucked.
However a smaller rock might be survivable in a shelter. <,,< Alzo! 2 seconds or 20 years. We have hundreds of telescopes around the planet looking for rocks and we've...