Yeah that correlates with what other people are saying regarding this specific manipulator thread problem. One reason it might be fine now is that it has downloaded and cached the data for that area now, so when you go there again its fine. Once you go to enough other places it will push this one out of the cache and you will get bad fps again when you return.
If this is true you could reproduce it just by clearing your cache (or setting the size to zero).