Public Access to Public Data

manifold.net is committed to supporting public access to public data. We also support unrestricted use of cartography, which we see as a form of free speech. We believe in modern times that it is not possible for free people to freely participate in democracies unless they may also freely utilize digital maps.

This essay is not a rant against maps that are sold by commercial entities. People and companies should be able to create new maps with an expectation that they will be able to sell those maps for a profit. However, we think government maps and GIS data created by the government are a special case. If they are placed by law into the public domain they should be made available to the public.

Civic Life Depends upon Free Access and Usage

Many aspects of civic life are defined by geographic considerations or are profoundly influenced by the geographic context in which they occur. In modern times, the use of digital data by governments and societies has become so interwoven into politics and ordinary civic processes that the maps used to drive decisions and debates are digital ones. GIS software running on computers is required to view and to manipulate such digital maps. To an increasing degree, the image of the world that is used to govern its citizens is a purely digital representation.

If you are forced to use only paper maps, you are barred from seeing or working directly with the data sets that are often used to make decisions. Instead of being a participant in democratic decision-making, you become the recipient of decisions already reached. Note that the popularity of web-based "map servers" does not remedy this situation, since such web servers provide a picture of the map and not the actual map itself. With the actual GIS data one may be a participant; when provided with a report, one is merely a recipient.

If you are charged a fee for access to the maps upon which government decisions are based, you are being charged a poll tax on your right to participate as a citizen. Free maps are essential to the democratic process. This does not mean that all maps should be free of charge, but it does mean that the maps created and used by government should be free or nearly so.

A corollary is that government maps should not only be free of charge, they should be free of usage restrictions. If the government tells you what you can do with the map, it is the same as if they told you what you could say with your words. Perhaps there is no charge for speaking, but you may not speak freely. Free usage of maps extends to the right to freely reproduce them and otherwise to use them and to provide them to others as you see fit.

In the United States, GIS users have become accustomed to free access via Internet to a very wide range of maps. Manifold imports so many formats that one could spend a lifetime downloading and viewing maps and still not see all that are freely available. However, despite the immense richness of the US GIS data wonderland, it still lacks many key GIS data sets that are required for informed citizen democracy:

It comes as a surprise to many technologically-literate people that even today, using the full resources of Internet, one cannot get a full set of digital maps of the world that are both usable by ordinary individuals and are free of usage restrictions. If NIMA's VMAP1 were made fully-available, this major restriction would end.
Until the publication of the first Free World Time Zones maps (by the Free World Maps Foundation), one could not even get free maps of time zones (!) for the United States and the world. There is still no official set of maps available to the public.
Telephone Area Codes affect life and commerce and are a key aspect of telephone communications. They are heavily regulated by government and the subject of numerous laws and regulations, but one cannot get a map of telephone area codes that is both free of cost and free of usage restrictions.

In the United States there seem to be two contradictory trends in public access to public data. On the one hand, more public data than ever before is being published on the Internet for free download. On the other hand, many public agencies ignore laws guaranteeing public access to public data, or they are providing the data in a form that renders it unusable by the public. In general, there are good news trends and bad news trends. We believe GIS users have a civic duty to educate the public and their representatives in government to encourage the good news trends and to reverse the bad news trends.

Good News and Bad News

Recent trends provide good news and bad news for free people who wish to use modern maps:

Good News:

Affordable GIS Hardware - Personal computers have become so powerful in recent years that anyone with an average PC can enjoy GIS capability with capacity and processor speed greater than the computers originally used to create most of the world's digital maps. As a practical matter, the cost of hardware is no longer a barrier for individuals in developed areas or for most organizations worldwide.
Affordable GIS Software - Recent advances in GIS software, driven by the PC technology revolution, have made sophisticated GIS capability (like that in Manifold System) available at prices affordable by ordinary individuals.
Expanding Public Domain data sets - Led by the United States, an increasing amount of government-originated data is emerging that by law is in the public domain and thus, in theory, is available for free usage.
Internet as the Great Equalizer - Internet makes it possible for people around the world to cooperate as equals to promote public access to public data. The low cost of communications and data interchange over Internet makes it practical for maps to be created and exchanged using teams of people in different countries. Because it is technically difficult for governments to censor Internet interactions, even people in countries where free cartography is illegal can benefit from freedoms in other countries.

Bad News:

State Cartographic Monopolies - Regrettably, the citizens of many so-called "free" societies must still purchase the landscape of their democratic discussions from a state-run monopoly and must utilize the maps so purchased only as allowed by the licenses granted by the state. Outside the United States, this is the rule, not the exception.
Attacks on Cartographers - Repressive governments often restrict cartography by private citizens. Such restrictions are prompted not by an uneducated fear of cartography but by a calculating, politically informed awareness of the utility of cartography. Citizen use of electronic maps supports undesired activities such as self-assembly, free speech or economic competition with the government. Mercator himself was imprisoned after his travels for cartographic purposes triggered government suspicion. Some repressive regimes ban citizen cartography as a means of maintaining dictatorships. Other countries, including some so-called "free" Western democracies, make it illegal for private citizens to engage in cartography. The ban helps maintain the state-run cartographic monopoly.
Repressive Influence on "Free" States - Some state cartographic monopolies have influenced agencies within free countries such as the United States in an attempt to prevent freely usable data from emerging that may threaten the state monopoly. In the US, for example, as of this writing the National Imagery and Mapping Agency (NIMA) has refused to release to the public many parts of "VMAP1", the public domain, non-classified, highly detailed world map it possesses, in part because of influences from state monopolies outside the US. The sharing of data between governments has now introduced the repressive policies of some state monopolies into formerly free areas. It also appears, in our opinion, to have become a handy excuse for rogue agencies to maintain an imperial disdain of statutory requirements to make public data available to the public.
Denial of Public Access - Some agencies simply refuse to provide public access to data or restrict access to approved users. If it is truly public data, you have the right to access it. Some agencies will not release public data to ordinary individuals, researchers or others not meeting their criteria for an approved user, or they will attempt to place a restrictive license on data that is in the public domain. Some agencies will make chilling statements about the law that are flatly untrue. For example, the Census Bureau at one point published instructions for TIGER/Line recipients that told them TIGER/Line was a trademark of the Bureau and that any usage of TIGER/Line data must be accompanied with a notice to that effect. That's simply wrong, since one can use this public domain data set however one wishes without any need to publicize the Bureau's trademarks. In another case, the EPA used restrictive licensing to deny access for years to the general public to important public domain GIS data and code regarding chemical emergencies.
Privatization by Public Agencies - Some agencies in the United States are privatizing public data and selling it back to the public as "products" using the lexicon and practices of private business. Although US law requires agencies to provide data for no more than the cost of copying and distribution, some agencies charge far more than allowed by law. The resources available to agencies are so great and the cost of utilizing the legal system in the US is so high that as a practical matter few people can afford to challenge even blatant violations of the law. Some agencies still charge thousands of dollars for data that costs tens of dollars to reproduce and distribute. In an even more disturbing trend, some agencies have registered trademarks on public domain data sets and have used their registration of the trademarks to attempt to prevent any "competition" from sources redistributing the public data for free.
Privatization of Public Data by Private Firms - One increasingly hears of key public data sets becoming privatized through opaque dealings with private firms. For example, by law telecommunications and telephones policy such as tariffs are a matter of public record in the United States; however, it is not currently possible to discover a map as fundamental to telecommunications debate as is the map of telephone area codes. A private contractor currently holds this map as a privatized data set. If you do not have free and open access to such maps as a private citizen you will forever be kept at a disadvantage in public hearings on telecommunications policy. In another example, the Census Bureau once released a CD that stated a commercial GIS vendor's license (who was involved in the production) controlled all data on the CD including the 1997 Statistical Abstract of the United States, an indisputably "public" data set. In 2001, the USGS effectively privatized the entire body of SDTS DEM files by delivering them in a de facto, exclusive deal to a small private company in Florida for distribution.
Privatization through Private Influence on Federal Agencies - A subtle, but very real, means of privatization of public data occurs when market concentrations in GIS software causes governmental agencies to publish public data in proprietary, private formats that require the permission of private companies for people to fully use the data. For example, much Federal data is published in ESRI's undocumented .e00 format. Although firms like manifold.net can discover the contents of such formats, there are very few other firms who can do so. In addition, there is no guarantee of transparency of data as would exist if the data were provided in an openly published format such as ESRI's .shp format. Although agencies could publish the data in an openly published format such as ESRI's .shp format, they will nonetheless at times publish data only in .e00. We feel this is a matter that education and awareness can help solve. However, it is a very important matter since to the degree any agency publishes data in a proprietary format it is not publishing data that is freely usable by the public.
Privatization by Researchers - As open as scientists normally are, there are few communities as secretive as "Big Science" can be at times. Many images of our world are in the form of public domain maps and data created by NASA and other government Big Science initiatives; however, these can be surprisingly hard to get when researchers feel a need to "protect" data from access by an unknown public. Numerous scientists are very open about sharing data, so we certainly don't wish to suggest that science overall has become "closed". However, it is not fair to criticize government and private firms for privatization of public data without acknowledging that the scientific community has been remiss in this area as well. Researchers withhold public data from the public for a variety of good and bad reasons, but if the result is to deny public access to public data the result is bad for democracy.
Privatization through "Webstacles" - A "webstacle" is the use of the Internet to avoid providing data to users. This is often the unintended result of good intentions, but at times webstacles are used to inappropriately privatize data. Here's how it works: an agency spends a few million dollars creating data. Rather than provide it for free download via FTP, they spend a few more million dollars creating a "map server" web site that spoon-feeds data to users in the form of dead images served over the web. The agency then says "we are fulfilling our duty of providing the data to the public through this web site." What's really going on is that the public cannot get free, public access to the data. The web server does not provide any data, only reports on the data in the form of images. If you don't like what the agency decided to show you in those images or if you wish to use the data in your own maps or analyses, tough luck. Further, webstacles traditionally only provide microscopic glimpses of the real data set. The agency has no risk that public users might ever actually get their hands on the entire data set. The result of the webstacle is that the data set remains inaccessible to the public. Map servers are a great idea if they are backed up with an FTP site or other means of making public data truly accessible to the public, but they cannot be considered a source for data if all they do is serve images.

We think on the balance it is possible to leverage the "good news" above to defeat the "bad news." When free citizens around the world join their GIS expertise via Internet, it only requires a single copy of a public domain data set to emerge in order to defeat the "bad news" reported above. Likewise, it is possible to engage in cartography in those countries where cartography is still legal to defeat government restrictions on cartography in repressed societies.

What You Can Do

Here are specific actions you can take to promote public access to public data:

Learn about copyright laws and Federal and State laws providing public access to public data. If you know the law, you'll know when someone is trying to restrict your rights under the law.
Learn about the GNU "Copyleft" license used to publish GNU and Linux and how it can be used to publish GIS data in a way that will prevent its subsequent privatization.
Learn how to use the FOIA and how to make and pursue FOIA requests for GIS data from Federal agencies.
Publish public domain maps and GIS data sets you obtain using the GNU license. Provide them to people who run servers allowing free download.
If you provide map images via Manifold's web server capabilities, consider providing access to the underlying data via FTP as well. This is not always appropriate for commercial data, of course, but all public agencies providing public access to public data should attempt to do so.
Encourage authors of public domain cartographic data sets to publish data under the GNU license. Do not patronize vendors who attempt to privatize public data or who reprint public data at unreasonably high fees. Support commercial vendors who provide real value added or original data by purchasing their products.
When you encounter a federal or state web site that features webstacles instead of real data, contact the administrator of the site and ask them to include links to real data in downloadable form. Some agencies mean well: because their agency uses overpriced software they think that GIS costs many thousands of dollars per license so they put effort into providing images in the belief that the public cannot afford to work with the real data. Set them straight: tell them that Manifold makes it possible to work with the most sophisticated data to as great a degree of sophistication as one chooses at prices no higher than ordinary Microsoft Office applications. Tell them the more sophisticated the data is, the more important it is that users be able to work with it as real data.
File Freedom of Information Act requests with US governmental agencies (or the equivalent with State agencies) to obtain GIS data that is not provided to the public. If after asking politely for the real data behind a "webstacle" web site the agency will not give it to you, make it a point to file an FOIA request for the data. Sooner or later the webmaster will realize that it is easier to provide a link on the site than to respond to a FOIA request. Very important: once you get data in response to a FOIA put that data into circulation so that other people can get it without repeating your FOIA request.
Educate federal, state, and local government worldwide on methods of publishing data to promote free map usage. This would include technical advice to avoid "privatization traps" such as publication in proprietary formats, as well as offers of free redistribution and access to Internet for organizations that do not have the technical resources or budget within their agencies to offer data that is being accumulated. Explain the difference between serving images and providing access to original data.
Complain to your Congressional representative and Senators about clearly illegal actions by federal agencies such as illegal FOIA responses and clearly illegal privatization of public data. Of course, unless you have given your Congressperson money in the form of political contributions you cannot expect any assistance but even in modern times there are surprisingly many Representatives and Senators who have directed their staffs to take an interest in legitimate complaints raised by "ordinary" people. Therefore, it is not totally an act of naivete to raise a complaint with one's Congressperson. If your Congressperson ignores you (as they almost always do on matters that require any thought) and you are among the one tenth of one percent who can afford to use the US justice system, litigate the denial of your FOIA request.
If you choose to take legal action over illegal agency actions, such as illegal responses or denials to a FOIA request, seek out allies. Many legal firms will have a "pro bono" (for free) program of contributing legal work for public causes. Seek out organizational allies and other people who may be affected by the illegal action. For example, NIMA's refusal to release the VMAP 1 coverages for Vietnam (clearly US origin data) is a slap in the face to the many US veterans who fought in Vietnam and who wish to use maps to preserve their history of the conflict, to note war graves and to seek closure in MIA/POW questions. Your Senator will likely ignore a single letter from you, but quite likely a series of letters arriving from Vietnam veterans and veterans' organizations will nudge the Senator's political instincts into action. Likewise, a law firm may be more likely to donate "pro bono" legal work if more than one person is involved.
If you are a government employee working within an agency that is deliberately restricting public access to public data (as, for example, NIMA does at this writing with VMAP1) make written notes of any such activities you observe. If you hear of any legal case filed, send your notes anonymously to the legal team filing the case. It will be a big help.
Encourage financial support from foundations, individuals, companies, and other organizations to promote public access to public data.
Tell your friends about manifold.net and encourage them to buy a copy. Manifold supports public access to public data (where else would you read a rant like this?) and provides a practical tool to get GIS data out of exotic Federal formats and into formats usable by many different GIS products. Manifold is also the only affordable way that ordinary people can work in a sophisticated way with sophisticated data.

The Freedom of Information Act (FOIA)

The Freedom of Information Act and its recent update, the Electronic Freedom of Information Act, guarantee access to virtually all GIS data created by the US government. There are only nine statutory exclusions to FOIA plus a pseudo-exclusion related to certain copyrighted map information provided to the Federal government. Use a good search engine to find sites devoted to FOIA and making FOIA requests. Numerous web sites will provide step-by-step instructions. These notes discuss issues especially relevant to GIS.

Many states have statutes equivalent to the FOIA for access to state information. Some, such as California's public records act, are considerably stronger than the FOIA because they award attorney's fees to the information requester if they have to go to court to enforce the act against an uncooperative agency.

Universal Access through FOIA

FOIA covers virtually everything created by the Federal government. If you see a printed map or a web image created by the Federal government, it's almost certain you have a right to the GIS data that was used to create that map or image. Ask politely to get the data. If you are ignored, send in a FOIA request letter. Send all letters by certified mail, return receipt requested so they know you know when they got the letter. The agency then has 10 or 20 days to respond.

If your FOIA request is denied, they have to tell you why, citing explicitly which exemption is being used. Don't accept any excuse that is not in the specific statutory exclusions. For example, NIMA stonewalled release of VMAP1 for years with excuses such as "our security office has not approved it" and so on. If it's not actually classified (a specific Federal designation that is the result of a specific process) it doesn't matter what their security office thinks of the matter. If it is not classified or not a copyrighted map provided by a third party under (rare) certain conditions you have a right to get it.

In recent years, some agencies have seeded public data with private, copyrighted material as a means of tainting all of the public data so that it cannot be forced into public hands via FOIA requests. We believe this is illegal and that that the agency would have to provide, at a minimum, all data it originated, "unwinding" the poisoning of the data at its own expense. A related tactic would be to file a FOIA request for all of the documents the agency has that prove the copyrighted material is, in fact, copyrighted and did not enter the public domain as a result of a dealing with the Federal agency.

In many circumstances material provided to the Federal government enters the public domain automatically unless certain procedures are followed to keep it out of the public domain. This is to prevent contractors from selling data to the government for public use and then seeking to deny the public use of the data. If the supposedly "copyrighted" material used to taint the public data is also in the public domain then this FOIA exemption no longer applies.

Costs

Agencies can charge the cost of locating and duplicating information in response to a FOIA request, although the first two hours are free. You can ask them to process your request only up to the two free hours allotted by law. If you ask for electronic transmission of GIS data via FTP, the marginal cost of sending GIS data via FTP is zero. You should be able to get almost any GIS data set within the statutory two free hours. Agencies may attempt to deny you access by threatening to levy very high costs for what really costs nothing to FTP. Hold your ground and appeal all such constructive denials of your rights under FOIA. If you seek a large data set and do not have a large FTP site, contact the webmaster@manifold.net to inquire about access to our mammoth FTP sites for use as incoming sites to receive FTP delivery of data pursuant to a FOIA request.

Be Fair, Be Firm

It is essential to know FOIA and your rights under it. It also is critically important from a moral and tactical perspective not to abuse FOIA by filing requests for data that has already been published, or to use FOIA to harass your fellow citizens in government who are doing their best to make data available. Please be flexible and be reasonable whenever using FOIA. Most government agencies are highly professional in responding to FOIA requests. Understand the process and do careful research on the data you seek so you can write a focussed, efficient FOIA request that does not require unnecessary work to process.

In many cases, making a few patient and responsible telephone calls or sending a few emails will eliminate the need to file a FOIA request. Many agencies and researchers are perfectly happy to provide a copy of a public data set in an informal manner. If someone does help you informally, do not abuse their goodwill by publishing their name on the Internet so they end up regretting their help to you. Do your best to help republish the information so that you too help disseminate the data.

Likewise, once a FOIA request is filed you should be firm in advancing the request. When asking for a politically sensitive data set you may be contacted by government representatives. They may try to convince you either not to pursue your FOIA request or to agree not to republish any data you get. Hold your ground. Be reasonable, but be firm. The requested material is either in the public domain or it is not. If it is not and the agency denies your request, they must cite the FOIA exemption used for denial. Get everything in writing and do not make verbal agreements authorizing delay, modification or abandonment of your request.

Most FOIA requests are handled in a highly professional (albeit slow) manner. If an agency has no political or internal "turf" issue with releasing data, you'll often get what you request very quickly. If the data requested is politically sensitive the situation is quite different. Our experience is that the art of using FOIA with unresponsive agencies is mainly the art of persevering in the face of various bureaucratic maneuvers intended to make you forget about your request.

It is important to persevere with FOIA even if you don't intend to (or cannot afford) the ultimate step of going to court to compel action on your FOIA request. Filing a FOIA request and then following through is a simple matter of sending a few letters. If enough people file FOIA requests for important data sets held hostage by recalcitrant agencies (as, for example, in the case of VMAP1 being withheld by NIMA), then pressure builds upon the agency to deal fairly. When many people file FOIA requests for key data sets, sooner or later the agency faces a real showdown in front of a judge where the judge will see a case record littered with numerous examples of underhanded agency maneuvers. In such cases judges have been known to award millions of dollars to FOIA requestors to compensate them for their legal costs incurred.

The GNU License

The Free Software Foundation originated the GNU ("GNU is Not UNIX") project to create a freely distributable version of UNIX. One of the results is Linux. To support this effort the Foundation created a license under which software could be freely distributed and freely used.

As used by GNU, the usage of the term "free" above is not focussed on a meaning of "free of cost". The key notion is freedom of usage. Many data are free of cost but suffer drastic restrictions on usage. For example, GIS companies often provide maps that are free of cost but which are subject to severe restrictions on utilization such as restriction to use only with the proprietary software packages of the originating GIS company. Such restrictions make it impossible for such maps to be used in open public policy discussions, because the restrictions require everyone participating in the discussion who wishes to see the full richness of the data in the maps to agree to the various proprietary licensing restrictions of a particular GIS company.

In other cases, non-profit organizations often publish maps that are provided free of cost but which may not be used for commercial purposes or by commercial organizations. We strongly believe that speech that is edited, even for a "good" purpose, is still not free speech. For someone to say you may use a map but not for commercial purposes as a practical matter is a very significant intrusion into your organizational life. It raises the question of when your behavior is "commercial" and when it is not.

As anyone knows who has done accounting for organizations that engage both in "commercial" and in "non-profit" activities, the distinction is often quite difficult to make and highly subjective in nature. Note also that blanket prohibitions on "commercial" usage would prevent the "chain reaction" mechanism of shareware publication that has enabled millions of CDs to be published at very low prices and near-zero cost to software authors. Commercial interests can often be harnessed to publish data and software at very low cost for the public good.

It has often been remarked that "public domain" data is already in the public domain and thus constitutes a freely distributable data set already. The problem with "public domain" data is that anyone may change such data slightly or convert it into a new format and then claim a copyright on the new form and use such a copyright claim to restrict further distribution. In fact, much of the GIS map data now sold by private companies under onerous restrictions originated in "public domain" data. Perversely, much data has emerged "in the public domain" only to be intercepted by a web of proprietary interests before it can reach users.

The GNU "copyleft" license resolves all the above problems by copyrighting the underlying programs or data and by licensing them under a specific license. The license (in informal summary, see the license itself for the precise terms and conditions) says you may freely use the data in whatever way you wish, so long as you pass on in any subsequent redistribution or usage the clearly stated right for any of your users to also do the same. This goes for any modifications of the data or programs covered by copyleft. You can change the data, edit it, improve it, simplify it, sell it, or give it away. However, you must clearly inform all of your users that they are free to do the same as well so long as they too, in turn, agree to the full provisions of the GNU copyleft license.

In this way, any maps that are distributed under the GNU license become seeds for yet further redistribution. Any improvements to the maps likewise become available to anyone else. Because of the GNU copyleft license, no one can take your maps and use them as a means of restricting usage, conversation, duplication, conversion to any other format, or any other restriction. This does not prevent anyone from selling programs or other data that may accompany GNU-licensed maps. It simply means one cannot restrict such maps in any way except by the "chain reaction" terms of the GNU license itself. It is this chain reaction effect that has been an important factor in the worldwide growth of software like Linux.

We don't suggest that the GNU license is the only way to license maps for distribution. We simply recommend it as one way of publishing GIS data in a way that will assure continued public access and free usage if that is the intent of the publisher. There are many other reasons for creating and publishing maps other than to provide public access to public data. In particular, private access to private data is an entirely different thing.

Please do not read this essay to infer in any way that manifold.net is against commercial map publication for profit. In our view, respect for private property rights is the flip side of the same coin as respect for public property rights. Just as public data should be protected against privatization, private data should be protected against forced transition to a de facto public domain status. Private vendors have every right to offer their intellectual property for sale (including maps) however they see fit. Private map creation and sale for profit are essential mechanisms for the creation of many important, value-added or original new maps.

Acknowledgement

Much of the material in this topic is used by permission and comes verbatim from the writings of the Free World Maps Foundation, which seeks to assure public access to public data in support of citizen democracy.

Back to Manifold Home Page