Q
Problem solve Get help with specific problems with your technologies, process and projects.

# Optimize SQL query

## How can I rewrite the following SQL query in an efficient manner?

How can I rewrite the following in an efficient manner?

SELECT DISTINCT(store_num),
(SELECT COUNT(SUBSTR(tran_pd,7,1)) FROM store_sales s WHERE SUBSTR(tran_pd,7,1) = 1 AND store = s.store) AS WEEK1,
(SELECT COUNT(SUBSTR(tran_pd,7,1)) FROM store_sales s WHERE SUBSTR(tran_pd,7,1) = 2 AND store = s.store) AS WEEK2,
(SELECT COUNT(SUBSTR(tran_pd,7,1)) FROM store_sales s WHERE SUBSTR(tran_pd,7,1) = 3 AND store = s.store) AS WEEK3,
(SELECT COUNT(SUBSTR(tran_pd,7,1)) FROM store_sales s WHERE SUBSTR(tran_pd,7,1) = 4 AND store = s.store) AS WEEK4,
(SELECT COUNT(SUBSTR(tran_pd,7,1)) FROM store_sales s WHERE SUBSTR(tran_pd,7,1) = 5 AND store = s.store) AS WEEK5
FROM store_sales GROUP BY store_num

I was hoping there is a SQL function out there that could do this but could not find any.

To begin, we must remove the parentheses from around store_num in the SELECT list. Please see the article DISTINCT is not a function (26 September 2005). DISTINCT always applies to all columns selected.

Next, notice that there is a GROUP BY on store_num. This should prompt us to question whether the DISTINCT is really needed. We must therefore do a quick evaluation of all the columns in the SELECT list. Notice there are five almost identical subqueries in the SELECT. Now, a subquery is allowed to be written in the SELECT only if it returns a single (scalar) value (hence, this type of subquery is called a scalar subquery). But more importantly, these five subqueries are correlated subqueries, and each of them will return its value, a count, based on which store it is. Thus, for the same store, the subquery always returns the same count. Therefore, all the rows produced by the query will be unique, and thus either the DISTINCT or the GROUP BY can be removed.

Hey, wait a second! What are those subqueries actually counting? Answer: the number of transactions for each of five weeks. Does this mean that for every store, five different queries against the entire transaction table are executed? Answer: yes. Can this be simplified? Answer: yes.

```select store_num
, substr(tran_pd,7,1) as weekno
, count(*)            as trans
from store_sales
where substr(tran_pd,7,1)
between 1 and 5
group
by store_num
, substr(tran_pd,7,1)```

This query produces the same counts as your original query. The WHERE clause can be eliminated if 1 through 5 are the only possible weeks. Let's call this our grouping query. The layout of the result set will look like this:

``` store_num  weekno  trans
105        1       9
105        2      37
105        3       4
105        4      42
105        5      11
107        1      21
107        2       5
107        3      16
...```

But your original query produces output that looks like this:

``` store_num  week1 week2 week3 week4 week5
105        9    37     4    42    11
107       21     5    16   ...```

To produce the same layout, we query our grouping query like this:

```select store_num
, sum(case when weekno = 1
then trans end) as week1
, sum(case when weekno = 2
then trans end) as week2
, sum(case when weekno = 3
then trans end) as week3
, sum(case when weekno = 4
then trans end) as week4
, sum(case when weekno = 5
then trans end) as week5
from (
select store_num
, substr(tran_pd,7,1) as weekno
, count(*)            as trans
from store_sales
where substr(tran_pd,7,1)
between 1 and 5
group
by store_num
, substr(tran_pd,7,1)
) as d
group
by store_num```

Here the subquery in the FROM clause is our grouping query. When a subquery is used in the FROM clause, it returns a result set of possibly (usually) many columns, and possibly (almost always) many rows, unlike a scalar subquery. A subquery in the FROM clause is often called an inline view or derived table.

The key to the efficiency of this query is that the counting and grouping of transactions is done in the inner query, the derived table. It is done once, efficiently. Then the outer query simply summarizes the counts in a "crosstab" aggregating query.

This was last published in November 2006

#### Have a question for an expert?

Get answers from a TechTarget expert on whatever's puzzling you.

You will be able to add details on the next page.

#### Start the conversation

Send me notifications when other members comment.

## SearchDataManagement

• ### Focus, scope and spotting opportunity are key to role of CDO

Chief data officers and experts see the CDO role as changing to a more strategic orientation -- especially finding key ...

• ### Good data quality for machine learning is an analytics must

As companies add machine learning applications, they need to really understand -- and be able to improve -- their data. That's ...

• ### Chief data officer role: Searching for consensus

The chief data officer role is about many things -- regulations, innovation, AI and more. Consultant Randy Bean discussed the ...

• ### Data analytics in government efforts lack structure

Data analytics in government agencies lack organization, focusing on immediate problems instead of attacking underlying causes ...

• ### A complete guide to buying data discovery software

Data discovery software turns data into business insights, and it's vital for GDPR compliance. This guide outlines the products ...

• ### Rising demand for business analytics education programs

Colleges and universities are increasingly offering business analytics degrees. The graduates can help build IT and business ...

## SearchSAP

• ### On-premises, hosted most popular S/4HANA deployment options

The pure cloud -- SaaS -- version of SAP's newest ERP, S/4HANA Cloud, lacks some of the same features of the on-premises version....

• ### S/4HANA public cloud version can get lost in cloud confusion

The 'true' public cloud is the streamlined SaaS version of on-premises S/4. But private cloud options are often conflated with ...

• ### SAP S/4HANA migration: What you need to know

There's a lot to consider when contemplating a move to SAP S/4HANA, and this essential guide provides a starting point, including...

## SearchSQLServer

• ### Six sample databases for SQL Server and how to find them

SQL Server sample databases are useful for test and dev, but they can be difficult to parse. Use this SQL database sample ...

• ### A quick tutorial on SQL Server maintenance plans

SQL Server maintenance plans get a bad rap, but for DBAs who need a simple way to maintain databases, Microsoft's built-in tools ...

• ### Proposed Microsoft-GitHub buy confirms open source role in cloud

The looming Microsoft-GitHub pairing confirms the company's rebirth as an open source friend. Data tools on the Azure cloud are ...

## TheServerSide.com

• ### Jenkins Git integration: GitHub pull request via the Git plugin

This Jenkins Git integration tutorial demonstrates how to create a freestyle build job that performs a Jenkins GitHub pull ...

• ### Financial firms, vendors push self-service software delivery

Self-service DevOps automation appeals to enterprises that must push out new code as they adapt to changing requirements.

• ### IT projects and software teams need to include Agile people

Not every idea deserves equal weight in a software development project, but Agile people know that garnering input from a wide ...

## SearchDataCenter

• ### Four top open source SIEM tools you should know

Open source SIEM software offers organizations a way to test capabilities and augment existing functionalities for analytics and ...

• ### Rackspace colocation program hosts users' legacy servers

Rackspace now has a managed colocation program that it hopes to upsell its customers with additional services, once their servers...

Broadcom has acquired CA Technologies in a move some believe is largely financially motivated, while others see an opportunity ...

## SearchContentManagement

• ### At OpenText Enterprise World, security and AI take center stage

OpenText unveiled its new application, OT2, at OpenText Enterprise World 2018, while also touting the importance of security. ...

• ### Augmented reality devices speed van repairs at Volkswagen U.K.

Augmented reality headsets for garage mechanics speed collaboration between repair shops and experts in the home office to solve ...

• ### Endpoint security tool fueled OpenText's Guidance Software acquisition

Endpoint security was the primary draw for OpenText's Guidance Software acquisition. But plans to improve e-discovery and data ...

## SearchHRSoftware

• ### Cost, doubt about tech hold back AI for HR investment

AI technology will improve the productivity of HR departments by eliminating many routine and transactional processes. But what ...

• ### Health and wellness benefits are exploding; are you keeping up?

A WorldatWork survey shows companies offered a wide variety of health and wellness benefits at an expanded rate last year. Is ...

• ### Automated recruiting solves Groupon's sourcing talent woes

Building a talent pool through effective sourcing is a major effort by Groupon. It is using a recruiting automation tool to find ...

Close