Q
Problem solve Get help with specific problems with your technologies, process and projects.

# Normalize a column containing a list

I've just completed an assignment for my databases class in which we were given a database and told to put it in 1NF. Several of the columns contained multiple values (e.g. 1;2;3;4), and I wanted a new row for each value in the column. I ended up projecting out the columns with multiple values and then importing them back in split up by semicolons, copying them into the original database and then using a bunch of select statements to get it down to what I wanted. I'm dying to know what a better way to do this is. If you don't understand what I'm talking about, just try to explain how to split a column that contains multiple values separated by a semicolon into separate columns.

The obvious way to do this is with a scripting language. Both Microsoft SQL Server and Oracle have comprehensive languages (Transact-SQL and PL/SQL, respectively) in which this would be both easy and efficient.

The scripting approach involves a loop, in which the target column is searched for an occurrence of the list separator (a semicolon in your case, more often a comma). In pseudocode:

```variable tempstring varying character
set tempstring = columnvalue
while position(';' in tempstring) > 0
begin
insert
into normalizedtable
( newcolumn )
values
( left( tempstring
, position(';' in tempstring)-1) )
set tempstring =
substring(tempstring
from position(';' in tempstring)+1
for length(tempstring)
- position(';' in tempstring) )
end
insert
into normalizedtable
( newcolumn )
values
( tempstring )```

The POSITION function finds the separator, or returns zero if one is not found. If a separator is found, then the substring, up to the position before the separator, is extracted with the LEFT function, and used in the insert statement to insert a new row. Then the substring just used is chopped out of the string, by setting the string equal to the remainder of the string starting one position beyond the separator, and looping continues. After looping has finished, the last part of the string is used to generate the last row.

In a database that has a built-in scripting language like SQL Server or Oracle, this script would be saved as a stored procedure and invoked with one call to the database. With an external scripting language, like PHP with MySQL for example, control would be bouncing back and forth between the scripting engine and the database engine for each insert, so an external script, while perhaps just as easy to write, is substantially less efficient, although still better than doing it manually.

But is there a straight SQL solution, one that does not involve scripting? If there is, it would have to involve a cross join of some sort, since it must be able to generate multiple rows for each row that contains at least one separator. It will more than likely involve a cross join with an integers table, so that if, for example, there are four separators in a given column value, then five rows would be generated, probably by the cross join with integers 0 through 4.

That said, I have seen an SQL solution, one written by Joe Celko. It was truly complex, involving CAST, MAX, COUNT, SUBSTRING, and DATALENGTH functions, a double cross join with two copies of the integers table, and a GROUP BY clause. Not for the faint of heart. My advice is to use a script, or even do it by hand. After all, you don't normalize on the fly all the time, right?

#### Start the conversation

Send me notifications when other members comment.

## SearchDataManagement

• ### DataStax making Cassandra easier with Kubernetes Operators

New Kubernetes tool for the Apache Cassandra database is DataStax's attempt to create a broadly adopted approach for cloud native...

• ### Ascend boosts enterprise data governance with data lineage

The new capabilities of Ascend Govern bring data lineage and enterprise data governance as well as the ability to try and figure ...

• ### The business benefits of enterprise data governance and MDM

Data leaders from prominent large organizations provide insights into data governance best practices and benefits, at ...

• ### Many employees still resist data-driven culture

While most employees see the benefits of data to drive decision-making, many still resist analytics, according to a report from ...

• ### SAS analytics platform benefiting from AI investment

Fueled by SAS' investment in AI and augmented intelligence features, the long-established vendor strengthened its BI platform ...

• ### Trends and top use cases for streaming data analytics

As more enterprises adopt real-time analytics, new infrastructure and best practices are appearing. Here are some trending ...

## SearchSAP

• ### SAP S/4HANA migration: Critical advice for moving off ECC

With the end of SAP ECC support looming in 2027, organizations must make some tough decisions. Here's a look at your choices.

• ### How to pick a SuccessFactors implementation partner

Selecting the right SuccessFactors implementation partner is a critical step in the journey to deploying the system. Check out ...

• ### 6 critical steps of a successful SAP S/4HANA migration

A successful S/4HANA project starts with knowing why your organization should even make the move, then nailing down whether S/4 ...

## SearchSQLServer

• ### SQL Server database design best practices and tips for DBAs

Good database design is a must to meet processing needs in SQL Server systems. In a webinar, consultant Koen Verbeeck offered ...

• ### SQL Server in Azure database choices and what they offer users

SQL Server databases can be moved to the Azure cloud in several different ways. Here's what you'll get from each of the options ...

• ### Using a LEFT OUTER JOIN vs. RIGHT OUTER JOIN in SQL

In this book excerpt, you'll learn LEFT OUTER JOIN vs. RIGHT OUTER JOIN techniques and find various examples for creating SQL ...

## TheServerSide.com

• ### Developing a COVID-19 application? Design it the right way

Application developers with skills and an idea could design an app to combat the virus. Follow these insights to effectively ...

• ### How developers can volunteer during the COVID-19 pandemic

What resources can developers offer to go against COVID-19? Explore some promising developer volunteer and open source project ...

• ### 5 Jenkins alternatives for Java developers

In need of an alternative to Jenkins as part of a CI/CD environment? Consider options from Atlassian, GitLab, Spinnaker, ...

## SearchDataCenter

• ### Tech giants aim supercomputers, clouds at finding a COVID-19 cure

The new COVID-19 HPC consortium comprised of tech giants, national labs and academia are using supercomputers and clouds to speed...

• ### Top vendor SDDC certification programs for admins

If you decide to grow your knowledge of software-defined data center technologies, VMware, Nuage Networks and Microsoft offer ...

• ### Build up your knowledge of leaf-spine network technology

As organizations deal with challenges of single tree path networks, many admins are considering leaf-spine architectures. Learn ...

## SearchContentManagement

• ### Creating digital experiences key to driving exceptional CX

The way in which businesses build digital experiences is ever-evolving. It's a race to create easier-to-use customer interfaces ...

• ### Vendors offer free remote work technology for telecommuting

It's more than a marketing ploy: Cloud technology vendors, seeing their own issues enabling remote work, extend paid products to ...

• ### How businesses should deal with enterprise search issues

Enterprise search issues frequently complicate user experience with ECM systems. While users may face many problems, they also ...

## SearchHRSoftware

• ### Best-case, worst-case pandemic scenarios for India service providers

India's IT and business support services will be tested during that country's pandemic lockdown. Many workers are now operating ...

• ### Farmers Insurance piloting human-like VR training

VR training has become a best practice at Farmers Insurance Exchange. Its claims adjusters are being exposed to more scenarios ...

• ### Virtual hiring, avoiding layoffs part of HR's pandemic response

In the face of a pandemic, many firms are trying to avoid layoffs and even hire. They are using virtual tools, including testing ...

Close