Database Reference
In-Depth Information
It's unlikely that any of the data sets we've worked with up until now have been
large enough to generate queries lasting longer than 100 ms. So for the following
examples, we'll use a data set consisting of daily
NASDAQ
summaries. If you want to
follow along, you'll want to have this data locally. To import it, first download the
archive from
http://mng.bz/ii49
.
Then unzip the file to a temporary folder. You'll see
the following output:
$ unzip stocks.zip
Archive: stocks.zip
creating: dump/stocks/
inflating: dump/stocks/system.indexes.bson
inflating: dump/stocks/values.bson
Finally, restore the dump like so:
$ mongorestore -d stocks -c values dump/stocks
The stocks data set is large and easy to work with. For a certain subset of the
NASDAQ
stock exchange's symbols, there exists a document for each day's high, low, close, and
volume for a 25-year period beginning in 1983. Given the number and size of the doc-
uments in this collection, it's easy to generate one of the log warnings. Try querying
for the first occurrence of Google's stock price:
db.values.find({"stock_symbol": "GOOG"}).sort({date: -1}).limit(1)
You'll notice that this takes some time to run. And if you check the MongoDB log,
you'll see the expected slow query warning. Here's a sample of the output to expect:
Thu Nov 16 09:40:26 [conn1] query stocks.values
ntoreturn:1 scanAndOrder reslen:210 nscanned:4308303
{ query: { stock_symbol: "GOOG" }, orderby: { date: -1.0 } }
nreturned:1 4011ms
There's a lot of information here, and we'll go over the meaning of all of it when we
discuss
explain()
. For now, if you read the message carefully, you should be able to
extract the most important parts: that it's a query on
stocks.values
; that the query
selector consists of a match on
stock_symbol
and that a sort is being performed;
maybe most significantly, that the query takes a whopping 4 seconds (4011ms).
Warnings like this must be addressed. They're so critical that it's worth your while
to occasionally cull them from your MongoDB logs. This can be accomplished easily
with
grep
:
grep -E '([0-9])+ms' mongod.log
If 100 ms is too high a threshold, you can lower it with the
--slowms
server option. If
you define slow as taking longer than 50 ms, then start
mongod
with
--slowms 50
.
Of course, grepping logs isn't very systematic. You can use the MongoDB logs to
check for slow queries, but the procedure is rather coarse, and should be reserved as a
kind of sanity check in a staging or production environment. To identity slow queries
before they become a problem, you want a precision tool. MongoDB's built-in query
profiler is exactly that.
