Home / Tutorials

Geospatial Data Services in ArcGIS Pro

If you are working alone on a simple research task, keeping everything in a project on your desktop computer may be adequate. However, if you are collaborating with other people in an organization or want to make your data available to the general public, you need a system architecture designed for serving data.

This tutorial will give a basic overview of how ArcGIS Pro is integrated with services from ArcGIS Online, ArcGIS Enterprise, and other online services.

• Services
  • The Client-Server Model
  • Servers
  • The Cloud
  • Local vs. Server
• Uniform Resource Locators
• Protocol Name
• Fully-Qualified Domain Name
• Subdomains
• Subfolders and Resource Names
• Query Strings
• ArcGIS Servers
• ArcGIS Online
• ArcGIS Enterprise
• Online vs. Enterprise
• ArcGIS Services
• Feature Services
• Image Services
• Vector Tile Services
• Map Services
• Using Services in ArcGIS Pro
• Feature Service Endpoints
• Feature Service Download
• ESRI's Living Atlas
• Image Service Download
• The Digital Dark Ages

Services

The Client-Server Model

The client-server model of system architecture is fundamental to contemporary internet-based communication.

• A server is a computer on a network that is dedicated to managing network resources like files or databases.
• Servers provide services that allow other computers on the network to access resources that are on (or controlled by) servers (Techopedia 2020).
• Clients are user devices like desktop computers and mobile devices that use network services.

A challenge with using services is that they require a reliable internet connect to access data on the servers. While this is generally not a major issue in urban areas in the developed world, this can necessitate designing systems that can work in a standalone mode when operating in rural areas or areas in the Global South with limited connectivity.

Servers

Server computers are powerful computers with large numbers of CPU cores, RAM, and storage compared to desktop computers.

• While server software like ArcGIS Enterprise can be run on desktop computers, the demands of serving multiple users usually dictate use of specialized server hardware.
• Contemporary servers are often mounted in equipment racks without keyboards or displays, and are managed entirely through network connections.
• Servers run specialized operating systems like Microsoft Server or Linux.

A virtual server is a server that shares a single physical hardware server with other virtual servers. Server virtualization uses hardware efficiently by enabling smaller virtual machines to be provisioned and removed quickly and without the difficulty and expense of having to purchase, install, and maintain separate physical hardware for each server.

The Cloud

The cloud is a collection of servers in one or more data centers that offer a variety of services to clients via the internet. Examples of cloud services include Microsoft 365, Google's suite of apps, and ArcGIS Online.

While the cloud looks to a client like a single server providing different services, in reality those services can be provided by different virtual servers in server farms located around the world to improve speed and provide backup when server hardware fails.

• The cloud permits easy sharing of data between collaborators and ready access to data on different client machines.
• With cloud infrastructure, you generally do not know (or care) exactly where your data is physically being stored or served.
• Cloud services can be moved around or scaled up and down depending on demand (and budget) in order to best serve the needs of the organization using that infrastructure.
• Use of the cloud frees businesses from the expense and difficulty of maintaining large collections of dedicated servers that are only fully used for limited portions of the day, week, or year.
• However, large organizations sometimes find that maintenance of on-premisis infrastructure is faster, more reliable, more secure, and / or more cost-effective than corresponding cloud infrastructure (Linthicum 2024).

Local vs. Server

When working with a desktop GIS program like ArcGIS Pro, you have a choice whether to keep your data locally on the desktop computer or keep the data on a server. ArcGIS Pro is tightly integrated with ArcGIS Online, making it possible to easily store your data in the cloud. But there are often times when it is more appropriate to keep your data on your desktop machine.

Uniform Resource Locators (URLs)

Services on the internet are accessed through uniform resource locators (URLs). The structure of a URL is similar to a file path, although it contains additional components.

Protocol Name

A URL starts with a protocol name or scheme.

• On the contemporary internet, this is almost always https:// (secure hypertext transfer protocol).
• You may occasionally see http:// (insecure hypertext transfer protocol), although this has largely been replaced by https://.
• ftp:// (file transfer protocol), or rtsp:// (real-time streaming protocol) are older protocols that are still occasionally used.

Fully Qualified Domain Name

A fully-qualified domain name (FQDN) identifies a server.

In URLs, FQDNs follow the protocol name and consist of three parts:

• An optional sequence of one or more sub-domains separated by periods
• A domain name
• A top-level domain name (TLD), usually .com, .org, .net, or .edu

Subdomains

Complex organizations often attach different subdomains to the front of the FQDN to represent different departments or services within an organization represented by the top-level domain.

The www subdomain has historically been used to specify a web server for internet pages within a domain, but www is commonly omitted in contemporary web server configurations.

The following are examples of subdomain names within the illinois.edu domain.

https://dres.illinois.edu/

https://housing.illinois.edu/

https://www.admissions.illinois.edu/

Subfolders and Resource Names

Optionally, the FQDN in a URL can be followed by subfolder names and/or resource names.

• Resource names usually have a suffix like .html, .jpg, or .png.
• The absence of a resource name ofen implies that the server gets the web page or service content from a database.
• Subfolders on traditional websites are often configured default resource name, such as index.html or default.html.

Query Strings

When URLs point to services or dynamic files created from database content, the URLs often contain additional request information with query strings.

A query string is an optional portion of a URL following a question mark (?) that is used to pass additional information to the server.

Parameters are separate chunks of information in a query string that are separated by ampersands (&).

In this case, the query string specifies an identification number for the desired web map.

https://univofillinois.maps.arcgis.com/apps/mapviewer/index.html?webmap=c71a6e7ac2c643d9a25d87de2f07f182

ArcGIS Servers

ArcGIS Online

ArcGIS Online is ESRI's cloud environment that provides a suite of services and web apps for processing and sharing geospatial data.

• Geospatial data storage and services to access that storage
• Creation and hosting of web maps and map-oriented web apps
• Data analysis and geoprocessing services

Organizations pay for ArcGIS Online with a negotiated license subscription charge, plus additional charges for credits that are used when services are used. Users are given credit quotas that protect organization from excessive charges if a user intentionally or unintentionally incurs credit charges for unexpectedly high use of ArcGIS Online services.

Web maps hosted in ArcGIS Online often draw on multiple services, such as a tile service for the base map and a feature service for points, lines, or polygons drawn over the base map.

ArcGIS Online is tightly integrated with ESRI's flagship desktop software, ArcGIS Pro, facilitating seamless data processing and sharing of data with collaborators and the general public.

ArcGIS Enterprise

ArcGIS Enterprise is a suite of server software from ESRI that allows organizations to provide services on their own physical or virtual servers in the same manner as ArcGIS Online.

An ArcGIS Enterprise installation has four components:

• ArcGIS Server listens for service requests and responds with formatted data from the data store or enterprise database.
• Portal for ArcGIS is a service that provides a flexible web-based user interface to access, manage, and web map data store(s) and ArcGIS Servers. Portal is the same interface used by ArcGIS Online, except it runs on your server rather than one of ESRI's cloud servers.
• Data stores are where data for feature, map, and tile services is stored. An ArcGIS Enterprise instance can have multiple data stores and enterprise relational database management systems (RDBMS) like Microsoft SQL Server or PostgreSQL can be used as data stores.
• Web adaptors route requests from the web server to ArcGIS Server or Portal for ArcGIS. This is an internal component that usually requires little attention from users or administrators after the server is set up.

ArcGIS Enterprise can be installed on Windows Server (IIS) or Linux (Apache) servers, as well a virtual cloud servers from providers like Amazon Web Services (AWS) or Microsoft Azure.

ArcGIS Online vs. ArcGIS Enterprise

The choice of whether an organization should use ArcGIS Online cloud service or ArcGIS Enterprise on their own server depends on the needs and limitations of the organization.

ArcGIS Services

A wide variety of different geospatial data services are available from ArcGIS Online and ArcGIS Enterprise servers that can be used to create map layers in ArcGIS Online interactive web maps and ArcGIS Pro static maps.

Feature Services

Feature services provide clients with access to vector features in a server geodatabase through REST endpoints.

A REST endpoint is a URL to a service on the server that includes subfolders and parameters needed to identify a specific feature service. The acronym REST comes from the type of communications protocol these endpoints use: representational state transfer.

This is an example REST endpoint URL for the US Fish and Wildlife Service's wetlands feature service:

https://www.fws.gov/wetlandsmapservice/rest/services/Wetlands/MapServer

ESRI provides a cloud environment called ArcGIS Online that enables users to publish feature classes as feature services.

• Feature services can contain multiple feature layers that can be added to maps in ArcGIS Pro, ArcGIS Online, or other geospatial apps.
• Feature services can be configured for sharing in ArcGIS Online to restrict access to just yourself, to members of a group, to everyone in your organization, or to everyone on the internet.
• Feature services can also be configured so that specific users can edit data, which permits easy collaboration when multiple users need to be able to update data.
• If you make updates to the published data, those changes will automatically be made available to everyone using your feature service.

With this flexibility comes some disadvantages:

• Feature services are dependent on the provider and can disappear at any time.
• Feature services can change at any time, which will likely be undesirable if you want specific controls over what version of the data your map users see.
• You do not have access to old versions of the data in case you need to compare the data across time.
• Feature services are often not be designed for high volumes of mission-critical activity, so they can be slow and give unreliable performance.

This is an example of a REST endpoint URL for a feature service published ArcGIS Online.

https://services.arcgis.com/GL0fWlNkwysZaKeV/arcgis/rest/services/Minn_2023_World_Bank_Indicators/FeatureServer

ArcGIS Online provides the ArcGIS Online web app that users can access in a web browser or on a mobile device to perform basic web map and feature service creation, and management of data and services in the ArcGIS Online environment.

Image Services

Image services provide clients with the ability to access raster and image data from a server geodatabase.

Image services supply data as tiles, which are map images chopped up into smaller 256 x 256 pixel images that the browser or mobile map app stitches together into a complete map image for the viewer.

Tiled web maps (also called slippy maps) load quickly and allow facile user interaction.

• The map app client makes a request for tiles that cover a specific geographic are at a given level of detail (zoom level).
• The server responds with the tiles needed to cover that area.
• The map app only needs to request new tiles as the user scrolls to new areas or zooms to new levels of detail.

This technique for quickly and efficiently delivering web maps was first developed for Google Maps in 2005, and tiles have become a ubiquitous technique for web mapping (Forrest 2021).

Tiled maps have disadvantages.

• You are dependent on the design choices of the tile provider when the tiles are published and you have no subsequent control over symbology or level of detail.
• Image tiles are designed for use on web maps and will be fuzzy if printed to higher-resolution paper or posters.
• Tiles are rendered (converted from data to imagery) when published which requires additional rendering time when tiles are published.

Different types of image services available in ArcGIS Online and ArcGIS Enterprise include:

• Dynamic imagery layers
• Tiled imagery layers
• Tile cache layers
• Map image layers

A number of different tiled imagery layers are available in ArcGIS Online through the Living Atlas. The layers are identified as Imagery Layer or Tile Layer in the search results.

Vector Tile Services

Vector tiles are tiles created as vector graphics rather than raster images, so they consume less bandwidth, can be expanded without getting fuzzy, and (sometimes) can be styled on the client side.

Vector tile services may be more appropriate than feature services or image tile services in some situations.

• Vector tiles are useful when publishing web maps with large numbers of features that would be impossibly slow to display from regular feature services and need generalization when zoomed out.
• Vector tile can effectively represent highly-detailed features that you would like to be able to print cleanly on both web maps and print maps.
• Unlike feature services, vector tile layers do not support clicking on the map to query feature attributes in pop-ups.
• ESRI's base maps in ArcGIS Online and ArcGIS Pro are often image tiles that are best suited for web maps, but some base maps are vector tiles that will print cleanly on posters and other large paper formats.

Using Services in ArcGIS Pro

Feature Service Endpoints

A portal is "a website serving as a guide or point of entry to the World Wide Web and usually including a search engine or a collection of links to other sites arranged especially by topic" (Merriam-Webster 2022).

Government agencies often make their geospatial data available through their open data portals. These portals occasionally provide links to feature services that are accessible to the general public, although they more commonly simply provide links to download the data as files.

You can find these open data portals by Googling "name_of_place open data" or "name_of_place gis data."

Although a variety of software exists to implement data portals, one package commonly used by city, county, and state governments is ESRI's ArcGIS Hub.

In this example, the Alabama GeoHub is implemented using ArcGIS Hub, which is obvious from the look of the site as well as the arcgis.com at the end of the URL.

We will use their layer of hospital locations.

1. Acquire: Copy the GeoServer link from the API dropdown. API stands for application program interface as these links are commonly used by programmers when they are creating web maps.
2. Start: Create a new map in ArcGIS Pro and give it a meaningful name.
3. Add: In ArcGIS Pro, on the Map -> Add Data dropdown, select Data From Path and paste in the endpoint URL.
4. Symbolize: Adjust the symbology appropriately.
5. Present: Create a print layout
   • Add a legend (if needed).
   • Export the map for printing or insertion as a figure.

Feature Service Download

Feature service endpoints can also be passed as the Input Features to the Export Features tool to export the contents of a feature service into a new feature class in your project database.

Downloading features in this manner can be unreliable, especially if you are downloading hundreds or thousands of features. If a link to a downloadable file is available, that will generally be a better option.

The REST endpoint URL for an feature service in an organizational portal can be found on the information page for the desired feature service layer. This example imports features from the Minn 2015-2019 ACS Counties feature service in the University of Illinois ArcGIS Online organization, with a filter limiting the counties to Illinois.

ESRI's Living Atlas

Services can be protected so that they can only be used within a specific organization, or by people pay for access to those services.

ESRI's Living Atlas of the World is a collection of geospatial data feature services that can be accessed in ArcGIS Pro or in ArcGIS Online. Some of the services are open to the general public, while other are proprietary and accessible only to licensed users.

The video below shows how to add a Living Atlas layer to a map in ArcGIS Pro:

1. Start: Create a new map in ArcGIS Pro and give it a meaningful name.
2. Add: On your map, select Map -> Add Data, and navigate to the Portal -> Living Atlas.
• Search for a variable of interest. For this example we use median household income.
• Perform a definition query if you want to limit the area displayed.
3. Symbolize: Adjust the symbology if desired.
4. Present: Create a print layout.
• Add a legend.
• Export to create a figure or printable map.
• See the properties to get the endpoint URL.
• View the endpoint to get the source info.

Image Service Download

Although Living Atlas image services are primarily intended for display, in some cases it is possible to export portions of the image data to raster files for further use in ArcGIS Pro.

For this example we demonstrate download of a portion of the USA NAIP Imagery: Natural Color Living Atlas layer covering Peoria County, Illinois to a GeoTIFF file.

1. Add the desired image service layer to your map.
2. Zoom so the extent of your map display covers the desired area for export.
3. Right click on the image layer and select Data and Export Raster tool
• Output Raster Dataset: Open the folder, navigate to the project geodatabase, and provide a meaningful name (County_NAIP)
• Coordinate System: Web Mercator (keep the default)
• Make sure the full area you want exported is visible on the map.
• Clipping Geometry: Select Current Display Extent. This will reset the Extent boxes and immediately change to As Specified Below.
• Select Maintain Clipping Extent.
• Raster Properties - Raster Size: Change the Columns and Rows to the maximum of 4000 for highest resolution.
4. Click Export.
5. Remove the ArcGIS Online tile layer and leave only the new raster layer on the map.

If you have polygon(s) to which you want to clip the raster:

1. Add the desired image service layer to your map.
2. Zoom so the extent of your map display covers the desired area for export.
3. Right click on the image layer and select Data and Export Raster tool
• Output Raster Dataset: Open the folder, navigate to the project geodatabase, and provide a meaningful name (County_NAIP)
• Coordinate System: Web Mercator (keep the default)
• Clipping Geometry: Select the polygons to which you want to clip.
• Select Use input features for clipping geometry
• Select Maintain Clipping Extent.
• Raster Properties - Raster Size: Change the Columns and Rows to the maximum of 4000 for highest resolution.
4. Click Export.
5. Remove the ArcGIS Online tile layer and leave only the new raster layer on the map.
6. You may need to Refresh your Catalog view to see the raster in your project geodatabase.

User-Created Services

Publishing service is the process of making data available as a service from software like ArcGIS Pro.

Although publishing in ArcGIS Pro is usually performed with a handful of mouse clicks and name entries, behind the scenes the publication process can involve uploading data, setting symbology, defining interactive capabilities (like pop-ups), and restricting access with permissions.

• Feature layers that you create in ArcGIS Pro can be published as "web layers," which are services that can be used in both ArcGIS Pro and on ArcGIS Online web maps.
• Published features are stored in geodatabases on ESRI's servers in the cloud and can be accessed through REST endpoints and through the ArcGIS Online web app.
• Organizations that have their own ArcGIS Enterprise servers can also host these services on those servers.

For this example, we use addresses of FedEx locations in Chambana, IL that were geocoded from a CSV file as described later in this document.

1. Add the feature class as a layer to your map and adjust the symbology to whatever you want the default to be when someone adds the new service to a map.
2. Edit the metadata for the layer to provide a meaningful title, summary, description, and credit for the original source of the data.
3. Right click the layer and click Share -> Share As Web Layer.
4. If you get a warning that, Allow assignment of unique numeric IDs for sharing web layers is disabled in Map Properties, right click on the Map in the Contents pane, and on the General tab, check Allow assignment of unique numeric IDs for sharing web layers.
5. You can adjust the sharing of your service in ArcGIS Online so that it is is visible only to you, only to people in your organization, or to everyone.
6. You can also verify that it works in a web map by opening the service in a new ArcGIS Online map viewer.

The Digital Dark Ages

The Dark Ages was a period of European history between around 500 - 1,000 AD when there was little urban life. Accordingly, there is very little written information that survives from that period of time (Encyclopedia Britannica, 2021).

In the developed world, we capture and store almost everything that can be stored: security video, electronic communications, smartphone photos of events momentous and trivial.

Almost none of that data will survive us.

Although storage becomes cheaper every year, technology changes every year. Data must be migrated to new storage media and modes of expression, or it is lost to physical degradation or technological obsolescence. It is likely that only a small fraction of the data from our time will migrated, and eventually almost all of it will be lost, leaving future generations with only a small fraction of the vast body of information we have accumulated in our time.

Data in the cloud never has a permanent physical home. The cloud is a performance and requires constant flows of capital and resources to stay in operation. Changes in the economics of the cloud will necessitate loss of some of that data.

Which data will be lost to time? Will this period be as "dark" to future historians as medieval Europe is to present day historians?

Contrast the impermanence of the digital with papyrus text from 2500 BC or clay tablets from as far back as 3300 BC.

While security camera video from an ATM where there has been no criminal activity may not be something that should outlive us, your grandchildren may want to see some of those thousands of baby pictures that you took of your son in the first year of his life. You should plan accordingly.