7.2.1. Optimizing Queries with EXPLAIN
EXPLAIN tbl_name
Or:
EXPLAIN [EXTENDED | PARTITIONS] SELECT select_options
The EXPLAIN statement can be used either as a
synonym for DESCRIBE or as a way to obtain
information about how MySQL executes a SELECT
statement:
EXPLAIN
tbl_name is synonymous
with DESCRIBE
tbl_name or
SHOW COLUMNS FROM
tbl_name.
When you precede a SELECT statement with
the keyword EXPLAIN, MySQL displays
information from the optimizer about the query execution
plan. That is, MySQL explains how it would process the
SELECT, including information about how
tables are joined and in which order.
EXPLAIN PARTITIONS is available beginning
with MySQL 5.1.5. It is useful only when examining queries
involving partitioned tables. For details, see
Section 17.3.4, “Obtaining Information About Partitions”.
This section describes the second use of
EXPLAIN for obtaining query execution plan
information. For a description of the
DESCRIBE and SHOW COLUMNS
statements, see Section 13.3.1, “DESCRIBE Syntax”, and
Section 13.5.4.3, “SHOW COLUMNS Syntax”.
With the help of EXPLAIN, you can see where
you should add indexes to tables to get a faster
SELECT that uses indexes to find rows. You
can also use EXPLAIN to check whether the
optimizer joins the tables in an optimal order. To force the
optimizer to use a join order corresponding to the order in
which the tables are named in the SELECT
statement, begin the statement with SELECT
STRAIGHT_JOIN rather than just
SELECT.
If you have a problem with indexes not being used when you
believe that they should be, you should run ANALYZE
TABLE to update table statistics such as cardinality
of keys, that can affect the choices the optimizer makes. See
Section 13.5.2.1, “ANALYZE TABLE Syntax”.
EXPLAIN returns a row of information for each
table used in the SELECT statement. The
tables are listed in the output in the order that MySQL would
read them while processing the query. MySQL resolves all joins
using a single-sweep multi-join method.
This means that MySQL reads a row from the first table, and then
finds a matching row in the second table, the third table, and
so on. When all tables are processed, MySQL outputs the selected
columns and backtracks through the table list until a table is
found for which there are more matching rows. The next row is
read from this table and the process continues with the next
table.
When the EXTENDED keyword is used,
EXPLAIN produces extra information that can
be viewed by issuing a SHOW WARNINGS
statement following the EXPLAIN statement.
This information displays how the optimizer qualifies table and
column names in the SELECT statement, what
the SELECT looks like after the application
of rewriting and optimization rules, and possibly other notes
about the optimization process.
Note: You cannot use the
EXTENDED and PARTITIONS
keywords together in the same EXPLAIN
statement.
Each output row from EXPLAIN provides
information about one table, and each row contains the following
columns:
-
id
The SELECT identifier. This is the
sequential number of the SELECT within
the query.
-
select_type
The type of SELECT, which can be any of
those shown in the following table:
DEPENDENT typically signifies the use of
a correlated subquery. See
Section 13.2.8.7, “Correlated Subqueries”.
-
table
The table to which the row of output refers.
-
type
The join type. The different join types are listed here,
ordered from the best type to the worst:
-
system
The table has only one row (= system table). This is a
special case of the const join type.
-
const
The table has at most one matching row, which is read at
the start of the query. Because there is only one row,
values from the column in this row can be regarded as
constants by the rest of the optimizer.
const tables are very fast because
they are read only once.
const is used when you compare all
parts of a PRIMARY KEY or
UNIQUE index to constant values. In
the following queries,
tbl_name can be used as a
const table:
SELECT * FROM tbl_name WHERE primary_key=1;
SELECT * FROM tbl_name
WHERE primary_key_part1=1 AND primary_key_part2=2;
-
eq_ref
One row is read from this table for each combination of
rows from the previous tables. Other than the
system and const
types, this is the best possible join type. It is used
when all parts of an index are used by the join and the
index is a PRIMARY KEY or
UNIQUE index.
eq_ref can be used for indexed
columns that are compared using the =
operator. The comparison value can be a constant or an
expression that uses columns from tables that are read
before this table. In the following examples, MySQL can
use an eq_ref join to process
ref_table:
SELECT * FROM ref_table,other_table
WHERE ref_table.key_column=other_table.column;
SELECT * FROM ref_table,other_table
WHERE ref_table.key_column_part1=other_table.column
AND ref_table.key_column_part2=1;
-
ref
All rows with matching index values are read from this
table for each combination of rows from the previous
tables. ref is used if the join uses
only a leftmost prefix of the key or if the key is not a
PRIMARY KEY or
UNIQUE index (in other words, if the
join cannot select a single row based on the key value).
If the key that is used matches only a few rows, this is
a good join type.
ref can be used for indexed columns
that are compared using the = or
<=> operator. In the following
examples, MySQL can use a ref join to
process ref_table:
SELECT * FROM ref_table WHERE key_column=expr;
SELECT * FROM ref_table,other_table
WHERE ref_table.key_column=other_table.column;
SELECT * FROM ref_table,other_table
WHERE ref_table.key_column_part1=other_table.column
AND ref_table.key_column_part2=1;
-
ref_or_null
This join type is like ref, but with
the addition that MySQL does an extra search for rows
that contain NULL values. This join
type optimization is used most often in resolving
subqueries. In the following examples, MySQL can use a
ref_or_null join to process
ref_table:
SELECT * FROM ref_table
WHERE key_column=expr OR key_column IS NULL;
See Section 7.2.7, “IS NULL Optimization”.
-
index_merge
This join type indicates that the Index Merge
optimization is used. In this case, the
key column in the output row contains
a list of indexes used, and key_len
contains a list of the longest key parts for the indexes
used. For more information, see
Section 7.2.6, “Index Merge Optimization”.
-
unique_subquery
This type replaces ref for some
IN subqueries of the following form:
value IN (SELECT primary_key FROM single_table WHERE some_expr)
unique_subquery is just an index
lookup function that replaces the subquery completely
for better efficiency.
-
index_subquery
This join type is similar to
unique_subquery. It replaces
IN subqueries, but it works for
non-unique indexes in subqueries of the following form:
value IN (SELECT key_column FROM single_table WHERE some_expr)
-
range
Only rows that are in a given range are retrieved, using
an index to select the rows. The key
column in the output row indicates which index is used.
The key_len contains the longest key
part that was used. The ref column is
NULL for this type.
range can be used when a key column
is compared to a constant using any of the
=, <>,
>, >=,
<, <=,
IS NULL,
<=>,
BETWEEN, or IN
operators:
SELECT * FROM tbl_name
WHERE key_column = 10;
SELECT * FROM tbl_name
WHERE key_column BETWEEN 10 and 20;
SELECT * FROM tbl_name
WHERE key_column IN (10,20,30);
SELECT * FROM tbl_name
WHERE key_part1= 10 AND key_part2 IN (10,20,30);
-
index
This join type is the same as ALL,
except that only the index tree is scanned. This usually
is faster than ALL because the index
file usually is smaller than the data file.
MySQL can use this join type when the query uses only
columns that are part of a single index.
-
ALL
A full table scan is done for each combination of rows
from the previous tables. This is normally not good if
the table is the first table not marked
const, and usually
very bad in all other cases.
Normally, you can avoid ALL by adding
indexes that allow row retrieval from the table based on
constant values or column values from earlier tables.
-
possible_keys
The possible_keys column indicates which
indexes MySQL can choose from use to find the rows in this
table. Note that this column is totally independent of the
order of the tables as displayed in the output from
EXPLAIN. That means that some of the keys
in possible_keys might not be usable in
practice with the generated table order.
If this column is NULL, there are no
relevant indexes. In this case, you may be able to improve
the performance of your query by examining the
WHERE clause to check whether it refers
to some column or columns that would be suitable for
indexing. If so, create an appropriate index and check the
query with EXPLAIN again. See
Section 13.1.2, “ALTER TABLE Syntax”.
To see what indexes a table has, use SHOW INDEX
FROM tbl_name.
-
key
The key column indicates the key (index)
that MySQL actually decided to use. The key is
NULL if no index was chosen. To force
MySQL to use or ignore an index listed in the
possible_keys column, use FORCE
INDEX, USE INDEX, or
IGNORE INDEX in your query. See
Section 13.2.7, “SELECT Syntax”.
For MyISAM and BDB
tables, running ANALYZE TABLE helps the
optimizer choose better indexes. For
MyISAM tables, myisamchk
--analyze does the same. See
Section 13.5.2.1, “ANALYZE TABLE Syntax”, and
Section 5.9.4, “Table Maintenance and Crash Recovery”.
-
key_len
The key_len column indicates the length
of the key that MySQL decided to use. The length is
NULL if the key column
says NULL. Note that the value of
key_len enables you to determine how many
parts of a multiple-part key MySQL actually uses.
-
ref
The ref column shows which columns or
constants are compared to the index named in the
key column to select rows from the table.
-
rows
The rows column indicates the number of
rows MySQL believes it must examine to execute the query.
-
Extra
This column contains additional information about how MySQL
resolves the query. Here is an explanation of the values
that can appear in this column:
-
Distinct
MySQL is looking for distinct values, so it stops
searching for more rows for the current row combination
after it has found the first matching row.
-
Not exists
MySQL was able to do a LEFT JOIN
optimization on the query and does not examine more rows
in this table for the previous row combination after it
finds one row that matches the LEFT
JOIN criteria. Here is an example of the type
of query that can be optimized this way:
SELECT * FROM t1 LEFT JOIN t2 ON t1.id=t2.id
WHERE t2.id IS NULL;
Assume that t2.id is defined as
NOT NULL. In this case, MySQL scans
t1 and looks up the rows in
t2 using the values of
t1.id. If MySQL finds a matching row
in t2, it knows that
t2.id can never be
NULL, and does not scan through the
rest of the rows in t2 that have the
same id value. In other words, for
each row in t1, MySQL needs to do
only a single lookup in t2,
regardless of how many rows actually match in
t2.
-
range checked for each record (index map:
N)
MySQL found no good index to use, but found that some of
indexes might be used after column values from preceding
tables are known. For each row combination in the
preceding tables, MySQL checks whether it is possible to
use a range or
index_merge access method to retrieve
rows. This is not very fast, but is faster than
performing a join with no index at all. The
applicability criteria are as described in
Section 7.2.5, “Range Optimization”, and
Section 7.2.6, “Index Merge Optimization”, with the
exception that all column values for the preceding table
are known and considered to be constants.
-
Using filesort
MySQL must do an extra pass to find out how to retrieve
the rows in sorted order. The sort is done by going
through all rows according to the join type and storing
the sort key and pointer to the row for all rows that
match the WHERE clause. The keys then
are sorted and the rows are retrieved in sorted order.
See Section 7.2.12, “ORDER BY Optimization”.
-
Using index
The column information is retrieved from the table using
only information in the index tree without having to do
an additional seek to read the actual row. This strategy
can be used when the query uses only columns that are
part of a single index.
-
Using temporary
To resolve the query, MySQL needs to create a temporary
table to hold the result. This typically happens if the
query contains GROUP BY and
ORDER BY clauses that list columns
differently.
-
Using where
A WHERE clause is used to restrict
which rows to match against the next table or send to
the client. Unless you specifically intend to fetch or
examine all rows from the table, you may have something
wrong in your query if the Extra
value is not Using where and the
table join type is ALL or
index.
If you want to make your queries as fast as possible,
you should look out for Extra values
of Using filesort and Using
temporary.
-
Using sort_union(...), Using
union(...), Using
intersect(...)
These indicate how index scans are merged for the
index_merge join type. See
Section 7.2.6, “Index Merge Optimization”, for more
information.
-
Using index for group-by
Similar to the Using index way of
accessing a table, Using index for
group-by indicates that MySQL found an index
that can be used to retrieve all columns of a
GROUP BY or
DISTINCT query without any extra disk
access to the actual table. Additionally, the index is
used in the most efficient way so that for each group,
only a few index entries are read. For details, see
Section 7.2.13, “GROUP BY Optimization”.
-
Using where with pushed condition
This item applies to NDB Cluster
tables only. It means that MySQL
Cluster is using condition
pushdown to improve the efficiency of a
direct comparison (=) between a
non-indexed column and a constant. In such cases, the
condition is “pushed down” to the cluster's
data nodes where it is evaluated in all partitions
simultaneously. This eliminates the need to send
non-matching rows over the network, and can speed up
such queries by a factor of 5 to 10 times over cases
where condition pushdown could be but is not used.
Suppose that you have a Cluster table defined as
follows:
CREATE TABLE t1 (
a INT,
b INT,
KEY(a)
) ENGINE=NDBCLUSTER;
In this case, condition pushdown can be used with a
query such as this one:
SELECT a,b FROM t1 WHERE b = 10;
This can be seen in the output of EXPLAIN
SELECT, as shown here:
mysql> EXPLAIN SELECT a,b FROM t1 WHERE b = 10\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 10
Extra: Using where with pushed condition
Condition pushdown cannot be used
with either of these two queries:
SELECT a,b FROM t1 WHERE a = 10;
SELECT a,b FROM t1 WHERE b + 1 = 10;
With regard to the first of these two queries, condition
pushdown is not applicable because an index exists on
column a. In the case of the second
query, a condition pushdown cannot be employed because
the comparison involving the non-indexed column
b is an indirect one. (However, it
would apply if you were to reduce b + 1 =
10 to b = 9 in the
WHERE clause.)
However, a condition pushdown may also be employed when
an indexed column column is compared with a constant
using a > or
< operator:
mysql> EXPLAIN SELECT a,b FROM t1 WHERE a<2\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: range
possible_keys: a
key: a
key_len: 5
ref: NULL
rows: 2
Extra: Using where with pushed condition
With regard to condition pushdown, keep in mind that:
Condition pushdown is relevant to MySQL Cluster
only, and does not occur when
executing queries against tables using any other
storage engine.
-
Condition pushdown capability is not used by
default. To enable it, you can start
mysqld with the
--engine-condition-pushdown option,
or execute the following statement:
SET engine_condition_pushdown=On;
You can get a good indication of how good a join is by taking
the product of the values in the rows column
of the EXPLAIN output. This should tell you
roughly how many rows MySQL must examine to execute the query.
If you restrict queries with the
max_join_size system variable, this row
product also is used to determine which multiple-table
SELECT statements to execute and which to
abort. See Section 7.5.2, “Tuning Server Parameters”.
The following example shows how a multiple-table join can be
optimized progressively based on the information provided by
EXPLAIN.
Suppose that you have the SELECT statement
shown here and that you plan to examine it using
EXPLAIN:
EXPLAIN SELECT tt.TicketNumber, tt.TimeIn,
tt.ProjectReference, tt.EstimatedShipDate,
tt.ActualShipDate, tt.ClientID,
tt.ServiceCodes, tt.RepetitiveID,
tt.CurrentProcess, tt.CurrentDPPerson,
tt.RecordVolume, tt.DPPrinted, et.COUNTRY,
et_1.COUNTRY, do.CUSTNAME
FROM tt, et, et AS et_1, do
WHERE tt.SubmitTime IS NULL
AND tt.ActualPC = et.EMPLOYID
AND tt.AssignedPC = et_1.EMPLOYID
AND tt.ClientID = do.CUSTNMBR;
For this example, make the following assumptions:
-
The columns being compared have been declared as follows:
-
The tables have the following indexes:
The tt.ActualPC values are not evenly
distributed.
Initially, before any optimizations have been performed, the
EXPLAIN statement produces the following
information:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
do ALL PRIMARY NULL NULL NULL 2135
et_1 ALL PRIMARY NULL NULL NULL 74
tt ALL AssignedPC, NULL NULL NULL 3872
ClientID,
ActualPC
range checked for each record (key map: 35)
Because type is ALL for
each table, this output indicates that MySQL is generating a
Cartesian product of all the tables; that is, every combination
of rows. This takes quite a long time, because the product of
the number of rows in each table must be examined. For the case
at hand, this product is 74 × 2135 × 74 × 3872
= 45,268,558,720 rows. If the tables were bigger, you can only
imagine how long it would take.
One problem here is that MySQL can use indexes on columns more
efficiently if they are declared as the same type and size. In
this context, VARCHAR and
CHAR are considered the same if they are
declared as the same size. tt.ActualPC is
declared as CHAR(10) and
et.EMPLOYID is CHAR(15),
so there is a length mismatch.
To fix this disparity between column lengths, use ALTER
TABLE to lengthen ActualPC from 10
characters to 15 characters:
mysql> ALTER TABLE tt MODIFY ActualPC VARCHAR(15);
Now tt.ActualPC and
et.EMPLOYID are both
VARCHAR(15). Executing the
EXPLAIN statement again produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC, NULL NULL NULL 3872 Using
ClientID, where
ActualPC
do ALL PRIMARY NULL NULL NULL 2135
range checked for each record (key map: 1)
et_1 ALL PRIMARY NULL NULL NULL 74
range checked for each record (key map: 1)
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
This is not perfect, but is much better: The product of the
rows values is less by a factor of 74. This
version executes in a couple of seconds.
A second alteration can be made to eliminate the column length
mismatches for the tt.AssignedPC =
et_1.EMPLOYID and tt.ClientID =
do.CUSTNMBR comparisons:
mysql> ALTER TABLE tt MODIFY AssignedPC VARCHAR(15),
-> MODIFY ClientID VARCHAR(15);
After that modification, EXPLAIN produces the
output shown here:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
tt ref AssignedPC, ActualPC 15 et.EMPLOYID 52 Using
ClientID, where
ActualPC
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
At this point, the query is optimized almost as well as
possible. The remaining problem is that, by default, MySQL
assumes that values in the tt.ActualPC column
are evenly distributed, and that is not the case for the
tt table. Fortunately, it is easy to tell
MySQL to analyze the key distribution:
mysql> ANALYZE TABLE tt;
With the additional index information, the join is perfect and
EXPLAIN produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC NULL NULL NULL 3872 Using
ClientID, where
ActualPC
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
Note that the rows column in the output from
EXPLAIN is an educated guess from the MySQL
join optimizer. You should check whether the numbers are even
close to the truth by comparing the rows
product with the actual number of rows that the query returns.
If the numbers are quite different, you might get better
performance by using STRAIGHT_JOIN in your
SELECT statement and trying to list the
tables in a different order in the FROM
clause.