Retrieving and Unpacking SDTS Data
Tutorial and Users Manual 
August 2003

Neither the U.S. Government nor any agency thereof nor any of their employees make any warranty, expressed or implied, or assume any 
legal responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed herein or 
represent that its use would not infringe privately owned rights. Reference to any specific commercial product, process, or service by trade 
name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the 
U.S. Government or any agency thereof.


1. Introduction	1

2. Overview of SDTS DLG and SDTS DEM Data	2
2.1  SDTS Overview	2
2.2  Mass Conversion of USGS Data to SDTS	2
2.3  Sales and Distribution Policies	3
2.4  SDTS data organization	3
2.4.1  SDTS DLG Data	3
2.4.2  SDTS DEM Data 	4

3.  Summary of Data Retrieval Procedures	4
3.1 SDTS DLG Data	4
3.2 SDTS DEM Data	5

4.  Detailed Instructions for Retrieving SDTS Data	5
4.1  Create Local Directory	6
4.2  Find and Retrieve Data	6
4.2.1 Find and Retrieve SDTS DLG Data  	6
4.2.1.1 SDTS DLG Transportation Overlay: A Special Case	7
4.2.1.2  Retrieve the SDTS DLG Master Data Dictionary	7
4.2.2 Find and Retrieve SDTS DEM Data	8
4.3  Unzip and Untar	8
4.3.1  WinZip	9
4.3.2  Other tar and zip utilities	10
4.3.2.1 DOS tar and zip utilities	10
4.3.2.2 UNIX tar and zip utilities	10
4.3.2.3 NT tar and zip utilities	10

5.  Problems Downloading Data	10
5.1 FTP	10

6.  Conclusions and Points of Contact	11


1. Introduction


Two digital cartographic products of the U.S. Geological Survey (USGS), Digital 
Line Graphs (DLG) and Digital Elevation Models (DEM), are available in the 
Spatial Data Transfer Standard (SDTS) format.  The USGS distributes these data 
sets over the Internet as a means of promoting the standard.  The SDTS is an 
ANSI approved data exchange standard that has been mandated for federal use. 
Numerous software packages from vendors such as ESRI, Integraph and 
ERDAS are able to accept data in this format.

This document is a detailed explanation of procedures for downloading and 
unpacking USGS SDTS data. DLG data sets were created by reformatting digital 
line graph optional (DLG-O) data as SDTS.  DEM data sets were created by 
reformatting existing data sets to the SDTS format.  These conversions include 
adding metadata to each transfer, but do not necessarily include any update or 
other content changes to the spatial data.

USGS SDTS data are a reflection of USGS native data.  For more information on 
USGS digital data, visit http://edc.usgs.gov. 

This document can be used as a companion to DLGv32 Pro, a free USGS Digital 
Data Viewer.  See http://mcmcweb.er.usgs.gov/drc/dlgv32pro/ for additional 
information about DLGv32 Pro. The procedures described here also may be 
beneficial to users of other software.  


2. Overview of SDTS DLG and SDTS DEM Data

2.1 SDTS Overview

The SDTS was designed by a group of people representing government 
agencies, universities, and private companies that saw a requirement for a robust 
way of transferring earth-referenced spatial data between dissimilar computer 
systems with the potential for no information loss.  In 1980, the U.S. Geological 
Survey (USGS) was designated the lead agency in developing earth science 
data standards for the federal government. The USGS worked with academic, 
industrial, and federal, state, and local government users of computer mapping 
and GIS to develop a standard for transfer and exchange of spatial data. In 1992, 
after 12 years of developing, reviewing, revising, and testing, the resulting 
standard--SDTS, was approved as Federal Information Processing Standard 
(FIPS) Publication 173, known as FIPSPUB 173-1, 1994. The FIPS version has 
been superseded by the current version, known as ANSI NCITS 320-1998 and 
was ratified by the American National Standards Institute (ANSI) June 9, 1998.  
Compliance with SDTS is now mandatory for all federal agencies.  SDTS is a 
transfer standard that embraces the idea of self-contained transfers; spatial data, 
attributes, georeferencing, a data-quality report, a data dictionary, and other 
supporting metadata are all included in the transfer.

The SDTS is a standard.  Standards are documents that specify rules; they are 
neither software nor databases.

The SDTS is a standard for data transfer, as opposed to a standard for data 
processing.  SDTS does not replace existing Geographic Information System 
(GIS) processing formats.

The SDTS is designed specifically for spatial data.  Other standards exist for 
other kinds of data (digital images, documents, electronic signals, etc.).
The National Mapping Division of the USGS is the maintenance agency for 
SDTS.  Please visit the SDTS web site at http://mcmcweb.er.usgs.gov/sdts for 
more information.

A single-document overview of SDTS is available at: 
ftp://sdts.er.usgs.gov/pub/sdts/articles/ascii/overview.doc or,
ftp://sdts.er.usgs.gov/pub/sdts/articles/ps/overview.ps  	

2.2 Mass Conversion of USGS Data to SDTS

In 1995 the USGS began converting digital cartographic data holdings to SDTS.  
1:24,000-scale (or 7.5 minute) DEM data were converted in early 1998.  

The 1:2,000,000-scale DLG data were revised in 1995 and are now available in 
SDTS. This includes coverage of the conterminous United States for the 
following categories of data: hydrography, transportation, boundaries (county 
names are included with the transfer), United States Public Lands Survey 
System, and manmade features.

All 1:100,000-scale DLG files have been converted to SDTS format. This 
includes national coverage of the hydrography and transportation overlays, with 
partial coverage of other layers.

All 1:24,000-scale DLG files have been converted to SDTS format.  Coverage 
varies between overlays at this scale, but no overlay has complete coverage.

"All files" means all files for which DLG data have been produced.  Coverage of 
1:24,000 and 1:100,000- scale DLG data is not complete.

2.3 Sales and Distribution Policies 

Most USGS data are neither online nor free.  SDTS data are a significant 
exception to this policy.  Other exceptions include 1:100,000-scale DLG-O and 
1:250,000-scale native format DEM data.  

Production, pricing, and distribution policies for native DLG-O and native DEM 
data have not changed because of the availability of SDTS data.  Data in native 
format are still available from the same sources at the same prices.  Because of 
data conversion problems, there are a small number of data sets that are 
available in native format but not available in SDTS.   All DEM data now offered 
in SDTS format are still available in native format.

2.4 SDTS data organization

2.4.1 SDTS DLG Data

DLG-O data are derived from standard USGS topographic quadrangles.  The 
packaging of DLG-O data reflects this: each physical file covers the geographic 
area of some part of a standard quadrangle, and contains a feature set that 
approximates the ink colors of a standard published map.

Similarly, SDTS DLG data are derived directly from DLG-O data, and reflect the 
DLG-O data packaging.  Each SDTS transfer has the same geographic coverage 
and same spatial data content as the DLG-O it was derived from.  This 
packaging is not dictated by the SDTS; the standard would allow (for example) all 
road and hydrography data for a county to be contained in one transfer, but 
reorganizing the data this way would have required a large and costly production 
operation, instead of a relatively simple data reformat.

A side effect is that the SDTS DLG data appear more fragmented and complex 
than necessary.  This is not a characteristic of the SDTS, but of policy decisions 
made when the USGS mass converted DLG-O data to SDTS format.

2.4.2 SDTS DEM Data 

DEM data are derived from standard USGS topographic quadrangles.  The 
packaging of DEM data reflects this: each physical file covers the geographic 
area of some part of a standard quadrangle,

Similarly, SDTS DEM data are derived directly from DEM data, and reflect the 
DEM data packaging.  Each SDTS Data transfer has the same geographic 
coverage and same spatial content as the original source DEM.


3.  Summary of Data Retrieval Procedures

SDTS DLG data can be found on the following ftp site:

ftp://edcftp.cr.usgs.gov/pub/data/DLG   

3.1 SDTS DLG Data

There are three sub-directories in the DLG FTP site; one for each of the three 
data scales, 100K, 2M, and LARGESCALE.

100K holds 1:100,000-scale data. The directory contains an SDTS master data 
dictionary for this scale, a readme file, and 26 subdirectories, one for each letter 
of the alphabet.  Each of these subdirectories holds the 1:100,000-scale 
quadrangles whose names start with that letter.

2M holds 1:2,000,000-scale data.  The directory contains an SDTS master data 
dictionary for this scale, a readme file, and 50 subdirectories, one for each state.

LARGE_SCALE holds 1:24,000-scale data. The directory contains an SDTS 
master data dictionary for this scale, a readme file, and 26 subdirectories, one for 
each letter of the alphabet.  Each of these subdirectories holds the 1:24,000-
scale quadrangles whose names start with that letter.

Each data sub-directory has several more levels of directories that reflect DLG 
overlays, data versions, and physical packaging.

The USGS GeoData web page, at http://edc.usgs.gov/geodata provides a user 
interface to the ftp site.  The web page provides several different "views" of the 
data, including lists by state, by name, and searching by graphic index.  All of 
these views eventually lead to the lowest level of the ftp directory structure. This 
web page also links to the SDTS Home Page.

An SDTS transfer is composed of multiple files.  SDTS DLG transfers have been 
combined into one physical file with the tar utility, and then compressed using 
gzip.  See section 4.3 for details.

3.2 SDTS DEM Data

USGS SDTS DEM files are available for download from any of several private 
venders.  Links to the venders can be found at

http://edc.usgs.gov/geodata 


4.  Detailed Instructions for Retrieving SDTS Data

This section contains step-by-step instructions for retrieving, unpacking, and 
organizing SDTS DLG and SDTS DEM data.  The San Francisco North, 
California 1:24,000-scale quadrangle is used as an example of DLG and DEM 
data.  
Procedures for the other scales are similar, but vary slightly because of different 
product characteristics.

These examples show how to find data using "FTP via graphics" and "FTP via 
state" capabilities on the GeoData web page.  Users who know what they are 
looking for and know how to use ftp client software may wish to skip these 
procedures and go directly to the ftp site described in section 3.  Reading section 

4.3 is strongly recommended before doing this.

4.1 Create Local Directory 

Create folders or directories on your local computer to hold downloaded files.  
For the purposes of these examples, we will assume you are using a computer 
with the Windows 2000 operating system installed (although these examples 
should also work with other versions of windows).  If you plan to use multiple 
quadrangles, it is important to organize the data in a file structure that will help 
you keep track of all data. Create an additional folder or subdirectory for each 
quadrangle.

An SDTS transfer typically contains many files.  Keeping transfers separate by 
putting each quadrangle in its own folder or directory is necessary because of the 
way files within a SDTS DLG or SDTS DEM transfer are named.  Storing multiple 
quadrangles in one folder or directory may result in file collisions, where one file 
may overwrite another with the same naming convention.

Create a folder/directory named san_francisco_north.  Within the 
san_francisco_north folder/directory create two more folders, sdts_dem_24k 
and sdts_dlg_24k. 

For the SDTS DLG example, the file path should be similar to 
c:\san_francisco_north\sdts_dlg_24k.

For the SDTS DEM example, the file path/name should be similar to 
c:\san_francisco_north\sdts_dem_24k.

4.2 Find and Retrieve Data

4.2.1 Find and Retrieve SDTS DLG Data 

A Master Data Dictionary (MDD) must be downloaded for each scale of DLG 
data (100K, 2M, and Large Scale) data you intend to use.  Detailed instructions 
are included in section 4.2.1.2, below.  To download a specific data set, follow 
these procedures:

1.  Go to the USGS GeoData page at http://edc.usgs.gov/geodata 

2.  Click on 1:24K DLG (in blue, near the top of the page).

3.  Since the name of the quadrangle is known, the data could be found by 
selecting either FTP via Graphics, FTP via Alphabetical List or FTP via State. 
However, it is common not to know exact quadrangle names:

Select FTP via Graphics.

4.  Select Conterminous 48 states.

5.  A simple map of the U.S. appears.  Click on the State of California.

6.  A map of California appears, with 1-degree lines of latitude and longitude.  
Click near San Francisco.  The next display will show approximately 30 7.5-
minute quadrangles, centered on where you clicked your mouse.

7.  Click on the 7.5-minute cell labeled San Francisco North, CA.

8.  A "list" of one quadrangle appears.  (Other paths to this page may produce 
lists of more quadrangles.) Select the link to San Francisco North, CA.

A list of nine directories will then appear.  Each directory corresponds to one 
DLG overlay.  San Francisco North is used in this example because all nine 
overlays are available.  This is unusual for 1:24,000-scale data; most 
quadrangles have fewer than nine overlays finished.

9.  Select the second overlay (in this example), hydrography.

10.  Select the file listed under (not necessarily inside) the folder 1/, this should 
be the most recent version.

The file 1525941.HY.sdts.tar.gz contains the SDTS transfer for the hydrography 
layer of the San Francisco North, CA quadrangle.  

11.  Select the file 1525941.HY.sdts.tar.gz for downloading 

12. Save it to the c:\san_francisco_north\sdts_dlg_24k folder. The file can be 
renamed (for example, sanfra.tar.gz).  Retaining the .tar.gz extensions is strongly 
recommended.  Some browsers will attempt to rename this file for you.  In some 
cases, this renaming causes problems for decompression software.  One 
common case is a rename from *.tar.gz to *_tar.gz, which confuses the WinZip 
decompression program.

Repeat steps 9 through 12 for all overlays of interest.  

4.2.1.1 SDTS DLG Transportation Overlay: A Special Case

While you are at the FTP site getting data for the San Francisco North, CA 
quadrangle, pull up the transportation subdirectory.  

You will see that it contains three SDTS DLG transfers: roads (RD), railroads 
(RR), and miscellaneous transportation (MT).   All DLG-O 1:24,000-scale sales 
units are packaged this way, and that packaging was retained in the SDTS 
version of the data: three separate SDTS transportation transfers were created 
for each quadrangle.  The data for these transfers must be kept in separate 
folders/directories.  Untaring two or more transportation transfers for the same 
quad in the same folder/directory will cause file collisions and loss of data.

If you intend to use more than one of these three transportation layers, create 
subdirectories for each of them.  For example, 
c:\san_francisco_north\sdts_dlg_24k\railroad.  This step need not be done before 
downloading the data, but must be done before unzipping and untaring the data 
(section 4.3).

In 1:100,000 and 1:2,000,000 DLG data, the three transportation overlays are 
combined into one file and, therefore, comprise only one SDTS transfer.  
Creation of additional transportation folders/subdirectories is not necessary for 
these scales.

4.2.1.2 Retrieve the SDTS DLG Master Data Dictionary (MDD)

Each SDTS transfer is required to have a data dictionary, specific to that 
particular transfer's scale.  Because SDTS DLG data were created by 
reformatting DLG-O data, the same data dictionary can be used for all data sets 
of the same scale.  Rather than include this dictionary with each transfer, one 
data dictionary is stored for all transfers at the top of the ftp directory tree.

To retrieve the MDD:

1.  Go to http://edc.usgs.gov/geodata.

2.  Click on 1:24K DLG.

3.  Click on "SDTS DLGs require Master Data Dictionary."

4.  Click on single tar file for 1:24K.

5.  Save it to the c:\san_francisco_north\sdts_dlg_24k folder.

This file must be unzipped and untared like other SDTS data. Placement of the 
data dictionary files may be software dependent.  If using DLGv32 Pro, the files 
may be placed in any directory on your hard drive, including the directory that 
holds the other SDTS data files. If using ESRI Arc/Info, place the data dictionary 
files in the MDD directory as specified by ESRI documentation.

4.2.2 Find and Retrieve SDTS DEM Data

1.  Go to the USGS GeoData page at http://edc.usgs.gov/geodata.  

2.  Click on 1:24K DEM (in blue, near the top of the page).

3.  Use links to navigate to the vender of your choice download free SDTS 
DEMs.

4.  Download San Francisco North (NOTE:  Both ten and 30 meter resolution 
SDTS DEMs are available for San Francisco North.  For a quicker download, 
choose the 30 meter resolution file).

5.  Select the file 1663964.dem.sdts.tar.gz for downloading.  

6.  Save it to the c:\san_francisco_north\sdts.dem.24k folder. The file can be 
renamed (for example, sanfran.tar.gz).  Retaining the.tar.gz extensions is 
strongly recommended.  Some browsers will attempt to rename this file for you.  
In some cases, this renaming causes problems for decompression software.  
One common case is a rename from *.tar.gz to *_tar.gz, which confuses the 
WinZip decompression program.

4.3 Unzip and Untar

The SDTS DLG and SDTS DEM data were designed in 1994, before the release 
of Windows 95 and before most PCs were powerful enough to use GIS data.  
The data were designed for the scientific computers of the time, which were 
mostly UNIX workstations.  The files in each SDTS transfer were combined into 
one file with the UNIX tar utility, and this combined file was then compressed 
using the gnu-zip (gzip) utility.  This technique for combining and compressing 
files is common on the UNIX operating system.

Tar and gzip are not as common in the PC world, but a variety of freeware, 
shareware, and commercial implementations of these utilities exist for all PC 
operating systems.

4.3.1 WinZip

One of the most common PC compression programs is the shareware program 
WinZip (http://www.winzip.com ).  WinZip will unzip and untar SDTS DLGand 
SDTS DEM files if proper procedures are followed.  These instructions are for 
WinZip 8.1 and Windows, 

Ensure that the file has a name of the form *.tar.gz   NOT  *_tar.gz

1.  Start WinZip

2.  From the toolbar, select Options

3.  Select Configuration. 

4.  Select Miscellaneous.

5.  There is a checkbox labeled "TAR file smart CR/LF conversion".  Ensure 
that this box is not checked.

6.  Select OK to exit configuration (this procedure should only have to be done 
the first time you use WinZip, although the option has been known to recheck 
itself).

7.  Navigate to the local directory you created earlier 
(c:\san_francisco_north\sdts_dlg_24k for SDTS DLG data or 
c:\san_francisco_north\sdts_dem_24k for SDTS DEM data).  A list of the files 
downloaded from the USGS GeoData site should be visible.  Select one of these 
files and double-click.

8.  A window appears with the message "Archive contains one file.  Decompress 
the file to a temporary folder and open it?"  Select Yes.

9.  A list of .ddf files will appear.  These are the files contained in the SDTS 
transfer

10.  Select Extract from the tool bar.  

12.  A popup window labeled "Extract" appears.  Enter, or navigate to, the path 
and directory that you want to place the data into.  The extracted files are stored 
in the same directory as the compressed files, so enter (or navigate to) the local 
directory you created earlier (c:\san_francisco_north\sdts_dlg_24k for SDTS 
DLG data or c:\san_francisco_north\sdts_dem_24k), then click the Extract 
button. This will unzip and untar the files.

NOTE: Do not rename files extracted from the tar file.  The files in an SDTS 
transfer are referenced by name in other files of the transfer.  Renaming .ddf files 
will break these relations.

4.3.2 Other tar and zip utilities

Tar and gzip for DOS, UNIX, and NT operating systems are available in the GNU 
utilities from the Free Software Foundation.  Software locations for each 
operating system listed below.

4.3.2.1 DOS tar and zip utilities

Copies of these can be retrieved from
ftp://ftpmcmc.er.usgs.gov/pub/sdts/software/tools/ 

These must be run in a DOS window, using the following syntax:
 	to Unzip:	<path>\gunzip <filename>
to Untar:		<path>\tar xvf <filename>

where <path> is the path to the directory that contains the gunzip and tar 
executables.

4.3.2.2 UNIX tar and zip utilities

Copies of these can be retrieved from
ftp://ftpmcmc.er.usgs.gov/pub/sdts/software/tools/ 

4.3.2.3 NT tar and zip utilities

Copies of these can be retrieved from
ftp://ftpmcmc.er.usgs.gov/pub/sdts/software/tools/ 

5.  Problems Downloading Data	

Unfortunately, the USGS does not have the staff to provide tutorial support for 
commercial software. If you cannot access these data through your browser, we 
recommend either: 

- studying your software documentation for anonymous ftp configuration 
procedures,  or
- using an ftp utility instead of a browser. 

5.1 FTP

Different brands and versions of web browsers implement anonymous ftp 
differently. The USGS has tested a number of browsers on the ftp site described 
in this paper and believes that the data can be accessed through most browsers. 
However, the default browser configurations are not always correct for 
anonymous ftp access, and in some cases changing those configurations is 
somewhat difficult.

Some sites where ftp utilities can be downloaded and purchased are:
http://www.cuteftp.com 
http://www.ftpx.com  

As an alternative, users of Win95 or WinNT can ftp data using the Run function 
on the Start Menu:

Type ftp://edcftp.cr.usgs.gov  and select OK.
Login:	anonymous
Password:  your e-mail address in the form  name@domain 

From there, make the following choices:  
pub/data/DLG/LARGE_SCALE/s/san_francisco_north_CA (this will get you to 
the SDTS DLG dataset we used in our example earlier)

At this point the user is able to navigate to the layer to be downloaded.

To change the data transfer type to binary, type binary.
To specify which directory the data are to be downloaded to, type lcd name of 
directory.
To download the data, type get name of file.
To exit, type quit.

6.  Conclusions and Points of Contact

SDTS DLG and SDTS DEM data are relatively new.  The design of these data 
could and will be improved for future data production. The data are usable in their 
current form if proper procedures are followed and appropriate software is 
available. 
  
For more information about SDTS, see http://mcmcweb.er.usgs.gov/sdts.  To 
make comments or ask specific questions about SDTS or the USGS 
implementation of SDTS DLG, send email to the SDTS Mailing List at 
sdts@usgs.gov.

For information about DLGv32 Pro, USGS freeware that will display USGS digital 
data see http://mcmcweb.er.usgs.gov/drc/dlgv32pro/.

For more information about USGS Digital Data products, visit 
http://edc.usgs.gov.
 



13