Not everything is immediately accessible from the start. A couple of secret passages are locked until you complete certain events. Essentially, an earthquake occurs to open them after you meet the prerequisites. This is to make sure that you witness certain events in a certain order.
Unless you've covered the entire map already, don't worry about it yet.
Sorry I didn't explain that earlier.