Yes. The chimney, flashing and surrounding roof should be assessed to identify the likely source of water ingress.