 |
 Global Surface ArchivesWORLDWIDE HOURLY WEATHER DATASET / 1930-2017 |
Global Surface Archives is the single largest collection of hourly and special
METAR and SYNOP weather observations in existence. Using any off-the-shelf viewer
like Digital Atmosphere, you can plot historical charts for anywhere in the world
for any hour, even with ship reports (after 1973).
You don't get complex XML or JSON records, or proprietary binary files.
All data is in plaintext (SYNOP, METAR, and CSV) and is designed for maximum usability.
These standard formats are familiar to almost all meteorologists and can be plotted with
a variety of off-the-shelf software tools or simply opened in Notepad.
For detailed historical maps, researchers traditionally had to rely on the NCEP Reanalysis series, which is a binary dataset requiring
specialized viewers and is mostly limited to 250 km resolution. In Global Surface Archives, you have all of the actual observations
and can construct even the most demanding mesoscale analyses for years gone by, whether you're looking at 2012's Hurricane Sandy
or the 1984 Ivanovo tornado in the former USSR.
Global Surface Archives represents so much processing work that it took 1 month of continuous processing time to complete on an Intel Core i7 machine.
DETAILS
■ Current period of record: January 1, 1930 to September 30, 2017
■ Available formats: METAR and SYNOP ordered by date-time. Volumes 2 and 3 contain data sorted by station in SYNOP, METAR, and tabular (CSV) format.
■ Displayable as maps?: Yes - Volume 1 can be plotted with Digital Atmosphere, GEMPAK, or other display software.
■ Displayable as text?: Yes - Volumes 1 and 2 are in text format.
■ Displayable as a spreadsheet?: Yes - Volume 3 is in CSV format (Excel, OpenOffice, etc). One file per station year.
■ Operating systems: All, including Windows and Linux. This is a data package.
■ Size: 127 GB compressed (apprx 1 terabyte if fully uncompressed)
■ Author: Developed and produced by Tim Vasquez / Weather Graphics
ORDERING
Global Surface Archives. This version includes all volumes: 1, 2, and 3 (see margin for details).
Please note that because of the massive increase in filesize, older portions of the dataset (typically before 1970)
will be provided on DVD-Rs bundled with the flash drive; Internet download of these old files is available; contact us.
Media: 128 GB flash drive (USB, V3.0 for fast transfer rates) / Allow 72 hours for processing
GSP001 |
Order Global Surface Archives Professional
$295
Lifetime Updates.
Lifetime Updates is a VIP service that entitles you to receive the latest Global Surface Archives data at any future date as long as
we're maintaining this product line. See the information below for more details. You may add this on later within 90 days of purchase.
GSXUPD |
Add Lifetime Updates
$49
Global Upper Archives.
Add the new Global Upper Archives dataset! Click here for more information.
INFORMATION ON LIFETIME UPDATES
If you have Lifetime Updates or are thinking about purchasing it, please read this important information:
When will there be an update? Updates are quarterly (every three months) at the beginning of January, April, July, and October.
Can I get a special update to make all the data current through today? Yes. If you want the data brought current through
the time of purchase, this is
available for a $75 service charge either at the time of purchase or any date after your purchase.
You do not have to have a Lifetime Updates package to request this.
Click here
to place this in your cart. Allow about 2-3 days for us to build and update the new data.
What's the procedure?
Updates are placed on our website for download. The link will be sent to you when you place the order.
You can check back on it at any future date, and if it stops working just contact us for a new link.
What if I don't want to download the updates? What if I want it on a USB drive?
If you prefer physical media or can't download the updates, we will be glad to provide a fully updated Global Archives
package on a USB drive. Either mail us your 128 GB USB drive or send $35 for a replacement drive.
Then add $5.95 for US postage, $22.75 for Canada, or $31.30 for all other countries.
Use this link to send payment.
Terms and conditions. This program is intended to allow customers to receive updates to Global Archives, providing we are continuing to maintain it, at no cost except for the actual
costs incurred by Weather Graphics, typically raw materials and postage. The guarantee provided to you is to
provide you a method of getting updates as we continue to sell that product without having to re-purchase it.
Terms and conditions are subject to change.
VERSION HISTORY
|
MAPS PLOTTED WITH GLOBAL SURFACE ARCHIVES
 |
An incredible cold wave in South America (17 July 2010, 2000 Brasilia Time -1) (click for large version). An amazing example of a cold outbreak making it to within 400 miles of the equator (see west side of Amazon basin). The town of Tarauaca reported 13°C (56°F) just after dusk. Rio Branco at 10°S reported a daytime high of 14°C (57°F), while Corumbá at 19°S never rose above 9°C (48°F). This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here. |
 |
The January 1987 cold wave hits Great Britain (12 January 1987, 1800 UTC) (click for large version). Winter sweeps westward across much of central and western Europe, producing what was arguably some of the coldest weather since the 19th century. Evening temperatures as seen here were well below freezing. The daytime maximum was -9.1°C (16°F) in the southern suburb of London of Warlingham, and the island of Jersey off England's southern coast recorded a high of -6°C (20°F)! This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here. |
 |
The Chernobyl disaster is depicted on this map valid 1200 UTC on 26 April 1986 (click for large version). The dataset shows that all of Ukraine under the influence of a 1034 mb anticyclone over the White Sea, which was producing easterly wind components across far western Russia, Ukraine, and Belarus. This pattern was partly responsible for advecting some of the contamination northwestward into the Baltic region, contributing to widespread alarm throughout western Europe. |
 |
A record heat wave in England (28 June 1976, 1800 UTC) (click for large version) which brought temperatures well above 30°C throughout much of England. Temperatures reached 35.6°C (96.1°F) in Southampton, the highest June temperature recorded in the UK. Unprecedented drought conditions led Parliament to pass the Drought Act, giving local authorities the power to control the use of water in their jurisdictions. The northeasterly flow into Great Britain is somewhat ironic since it is this pattern which, in the winter, is associated with cold waves. However, in this instance the trajectory off the European continent is bringing warm, dry air rather than cold, dry air. |
 |
The lowest temperature to ever occur on Earth (21 July 1983, 0000 UTC) (click for large version) occurred at Vostok, Antarctica. There's not much to see on this map because the station density in Antarctica has always been historically sparse, but there it is. Vostok had started out at -60°C (-76°F) on July 7th, then two weeks of clear nights in the "perpetual night" of the Antarctic winter allowed heat to radiate away before the site finally reached -89.2°C (-128.5°F) as shown here. That's a 16-knot wind. We tried to calculate wind chill with the NWS/SRH wind chill calculator, but at those readings it just says "Undefined". |
 |
One of the coldest days on record in India (9 January 2011, 1730 Indian Standard Time) (click for large version). Widespread, persistent fog in the northern valley regions of India led to record low temperatures. Delhi's maximum on this day was 11 degrees Celsius, the coldest on record since 1969. This map is in degrees Celsius; American readers can see temperatures in Fahrenheit by clicking here. |
| TECHNICAL ASPECTS |
Contents
Global Surface Archives is a ultra-massive collection of
worldwide "hourly" and "synoptic" weather observations
in two of the most universal formats for weather data distribution.
The size of this dataset is approximately 109.5 GB (~500 GB uncompressed).
Period of record
Datasets start in 1930, though a few records exist for Finland and France going back to 1920 and were included. The data is sufficiently
dense to allow synoptic maps of the US in the late 1930s and for parts of the US and Europe in the 1940s. Strong station density emerges worldwide in the 1950s.
Archive volumes
This dataset is made up of three different volumes.
Different volumes contain the exact same information as other volumes, but are sorted or formatted different ways.
- Volume 1. Observations are in SYNOP and METAR format. Each file contains data for a unique date/time. A filename might appear as 1995080214_metar.txt, containing all worldwide observations for August 2, 1995 at 1400 UTC. This allows easy plotting by weather mapping software like Digital Atmosphere. This volume is the one most requested by most of our customers and is very similar to the various Archives sets we have sold for the past 15 years. Click here for a sample of Volume 1 data.
- Volume 2. Observations are in SYNOP and METAR format. Each file contains data for a unique station. The station files are also split into years, since the row count would otherwise overload most text editors and word processors. A filename might appear as FAJS-1986.txt, which would contain all observations for Johannesburg, South Africa in 1986. Click here for a sample of Volume 2 data.
- Volume 3. Observations are in tabular (CSV) format. Each file contains data for a unique station. The CSV format is decoded and can be easily opened in Excel or OpenOffice. The station files are also split into years, since the row count would otherwise overload most spreadsheet programs. A filename might appear as RJTT-1997.csv, which would contain all observations for Haneda Airport in Tokyo for 1997. Volume 3 contains land stations only (no ships). Click here for a sample of Volume 3 data.
Documentation
Refer to the
documentation (590 KB, PDF)
for details about the Volume 3 comma-delimated data and for other information about this dataset.
Identifiers
METAR data is indexed according to standard ICAO codes. To enhance the usability of this dataset, we use a version of this called Epoch 2010,
in which only identifier assignments which were valid in the year 2010 are used. For example, a 1975 observation from
Hanoi Gia Lam would appear as VVGL, same as the current identifier. We wouldn't use VWHN, which is what the
pre-1976 North Vietnamese government used.
Doing so would require all our users to consult old ICAO documents to see what the identifier was used for.
We've taken care of all the hard work for you.
SYNOP data is indexed according
to standard 5-digit World Meteorological Organization station numbers; since these do not change they are provided
in their original assignment as received.
WMO Restrictions
SYNOP reports for about 40 countries at intermediate hours (outside of 00, 06, 12, and 18Z) falls under the purview of
WMO Resolution 40 and is omitted from this dataset. For more information about Resolution 40 see the
WMO page on the topic. Where data is omitted, METAR data tends to be sufficiently
abundant to cover the shortfall.
Known deficiencies
There are a number issues we are aware of and because of this, the dataset is sold as-is.
Worldwide data is very scarce during the 1930s and somewhat scarce during the 1940s since those were the primitive years of hourly observations.
It may be impractical to construct satisfactory maps until the 1950s.
Also, much of the data is reassembled from "packed" records; before the 1980s mass storage space was
extremely expensive and archiving data in original form was considered wasteful.
All METAR data is also standardized to the post-1996 standard, which is generally considered to be a bonus but
does cause some degradation of sky condition data, which was encoded in a different system before then.
There is a shortfall in observations between 1969 and 1972 due to data loss at the telecommunications centers and results in data not being available in some
countries; the data loss episode peaked at 1972 before recovering in 1973.
Inventory
A basic inventory of the dataset is provided here. These ZIPs contain a large text file that lists
every file in the set along with its size. THE FOLLOWING MAY NOT APPLY TO NEWER RELEASES; this update was prepared in January 2012 and has not been updated.
lib001.zip - Volume 1 (METAR/SYNOP by date/time)
lib002.zip - Volume 2 (METAR/SYNOP by station)
lib003.zip - Volume 3 (tables by station)