Ran into a gotcha with PHP+MongoDb the other day.
Now, PHP ain't my favourite language, but its what Piwik is written in so I'm stuck with it for that. I've partially ported Piwik to use mongodb (vs mysql) as a storage system for scalability reasons.
The problem is that if you're connecting to a mongodb replicaset, you must ensure the hostnames you use are the same as the ones used in the replicaset configuration: if you refer to them as "node1, node2" in the replicaset, but connect to them as "node1.myprivate.network, node2.myprivate.network", it doesn't work well.
The PHP Mongo docs do actually warn about this:
"The host names that you specify here must match the names as given in the replica set configuration because the driver uses the name as specified in the replica set configuration as identifier in its internal mapping. Specifying (for example) the IP address instead of the configured host name makes that the driver keeps a connection for both the host name and its accompanying IP address."
I found it was worse than this: if you get it wrong, it will not cache connections at all; every db request will create a new connection to the database, which will cause unnecessary load. Its easy to miss where the load is coming from too; but as usual, good old tcpdump showed the problem almost instantly.
One simple reconfiguration of the db hostnames later and all was well.