A set-oriented method for representing trees in SQL that runs orders of magnitude faster than the adjacency list (i.e. child-parent pairs) method. [Ed. note: A follow-up to this article is also available.]
The usual example of a tree structure in SQL books is called an adjacency list model and it looks like this:
Another way of representing trees is to show them as nested sets. Since SQL is a set oriented language, this is a better model than the usual adjacency list approach you see in most text books. Let us define a simple Personnel table like this, ignoring the left (lft) and right (rgt) columns for now. This problem is always given with a column for the employee and one for his boss in the textbooks. This table without the lft and rgt columns is called the adjacency list model, after the graph theory technique of the same name; the pairs of nodes are adjacent to each other.
The organizational chart would look like this as a directed graph:
The first table is denormalized in several ways. We are modeling both the personnel and the organizational chart in one table. But for the sake of saving space, pretend that the names are job titles and that we have another table which describes the personnel that hold those positions.
Another problem with the adjacency list model is that the boss and employee columns are the same kind of thing (i.e. names of personnel), and therefore should be shown in only one column in a normalized table. To prove that this is not normalized, assume that "Chuck" changes his name to "Charles"; you have to change his name in both columns and several places. The defining characteristic of a normalized table is that you have one fact, one place, one time.
The final problem is that the adjacency list model does not model subordination. Authority flows downhill in a hierarchy, b...
To continue reading for free, register below or login
To read more you must become a member of SearchOracle.com
ut If I fire Chuck, I disconnect all of his subordinates from Albert. There are situations (i.e. water pipes) where this is true, but that is not the expected situation in this case.
To show a tree as nested sets, replace the nodes with ovals, then nest subordinate ovals inside each other. The root will be the largest oval and will contain every other node. The leaf nodes will be the innermost ovals with nothing else inside them and the nesting will show the hierarchical relationship. The rgt and lft columns (I cannot use the reserved words LEFT and RIGHT in SQL) are what shows the nesting.
If that mental model does not work, then imagine a little worm crawling anti-clockwise along the tree. Every time he gets to the left or right side of a node, he numbers it. The worm stops when he gets all the way around the tree and back to the top.
This is a natural way to model a parts explosion, since a final assembly is made of physically nested assemblies that final break down into separate parts.
At this point, the boss column is both redundant and denormalized, so it can be dropped. Also, note that the tree structure can be kept in one table and all the information about a node can be put in a second table and they can be joined on employee number for queries.
To convert the graph into a nested sets model think of a little worm crawling along the tree. The worm starts at the top, the root, makes a complete trip around the tree. When he comes to a node, he puts a number in the cell on the side that he is visiting and increments his counter. Each node will get two numbers, one of the right side and one for the left. Computer Science majors will recognize this as a modified preorder tree traversal algorithm. Finally, drop the unneeded Personnel.boss column which used to represent the edges of a graph.
This has some predictable results that we can use for building queries. The root is always (left = 1, right = 2 * (SELECT COUNT(*) FROM TreeTable)); leaf nodes always have (left + 1 = right); subtrees are defined by the BETWEEN predicate; etc. Here are two common queries which can be used to build others:
1. An employee and all their Supervisors, no matter how deep the tree.
2. The employee and all subordinates. There is a nice symmetry here.
3. Add a GROUP BY and aggregate functions to these basic queries and you have hierarchical reports. For example, the total salaries which each employee controls:
4. To find the level of each node, so you can print the tree as an indented listing.
5. The nested set model has an implied ordering of siblings which the adjacency list model does not. To insert a new node as the rightmost sibling.
6. To convert an adjacency list model into a nested set model, use a push down stack algorithm. Assume that we have these tables:
This approach will be two to three orders of magnitude faster than the adjacency list model for subtree and aggregate operations.
Reader Feedback
Michael S. had some questions (in italics), to which Joe responded. Here's the exchange:
I was interested to read this article about implementing tree structures
in SQL. I work for Streamload, an online file storage website where I
helped developed the database technology that powers the site. We use a
SQL database to store information about hundreds of millions of files and
folders, basically a "tree in SQL" per registered user. We use the
adjacency list method, and we are forced to use recursive execution to get
all files under a folder, so this nested sets solution is of great interest
to us.
I have had other people with similar problems. They added another column
to the table for each tree within the forest:
Upon looking at the ramifications of the nested sets implementation,
however, it would seem to require too much database
thrashing for tree modifying operations. The code to do a tree insert was
provided, and it seemed at initial glance that a) the state of the entire
tree needed to remain fixed (locked) for the duration of the operation in
order to not risk leaving the tree in an invalid state if another insert
operations happened concurrently, and b) it seemed possible that much if
not most of the tree could be modified by a simple insertion, not just the
ancestor nodes as one might expect/hope, but possibly everything to the
left and/or right of a node. These seem to be major impediments to
implementing the nested sets solution in practice. I'd be interested in
getting some response from the author and/or others as to the validity of
these complications and any possible workarounds.
These limits are not a problem if your tree is relatively fixed -- most
companies do not do re-organizations on a weekly basis. Well, that might
not have been true for dot-coms...
But in your case, a rapidly changing tree structure, has been implemented
successful for a newsgroup project. The forest was the newsgroup itself,
each tree was a message thread, and each node was a posted message. There
was almost an order of magnitude improvement in performance (handling and
displaying text in and out of SQL is expensive, so most of the execution
time was spent there).
The gimmick? The nested set model has two properties which I call:
1) The Algebra Property: For each node (n), (rgt-lft+1)/2 = size of subtree
rooted at (n). This requires that the union of the rgt and lft numbers be
an unbroken sequence, no gaps.
2) The Between-ness Property that you can find superiors and subordinates
with a BETWEEN predicate:
The employee and all subordinates. There is a nice symmetry here.
This does NOT require that the union of the rgt and lft numbers be an
unbroken sequence. All you need is the condition that (lft < rgt),
uniqueness of lft and rgt, and that subordination is represented by
containment of one (lft, rgt) pair within the range another (lft, rgt)
pair.
Can you tell that I was a math major? In English: the trick is to use a
bigger step than 1, if you don't need the Algebra property. For example:
The organizational chart would look like this as a directed graph:
To insert someone under Bert, say Betty, you look at the size of Bert's
range and pack from the left:
To insert someone under Betty, you look at the size of Betty's range and
pack from the left:
Given a clustered index on (lft,rgt), a reasonable fill factor per data
page and reasonably smart concurrency control, only the datapage involved
will be locked for each insertion. Assuming you have a 32 bit integer, you
can have a depth of nine or ten levels before you have to re-organize this
particular tree within the forest. If you don't mind negstive numbers, you
can use the full range of the integers:
I am obviously skipping some of the algebra for computing the gap size, but
you get the basic idea. There are some other tricks that involve powers of
two and binary trees, but that is another topic. A consultant does not
give away all his tricks without getting paid!
About the Author
Joe Celko is author of Joe Celko's SQL for Smarties: Advanced SQL Programming (Morgan-Kaufmann, 1999, second edition).
For More Information
