In Charwood mine there are two rooms you do not need to visit to complete any quest. On the first level, there is one on the left. On the second level, there is one on the right.
There is also a basement in the "rat poison settlement" (I don't know what else to call it), and IIRC you don't need to go down there for anything. Are you sure you went there?