Introduction

Getting map data into the BlueWorx application is made easier with Service Pack 12. This article outlines the functionality in this area.

Contents

Existing functionality

Prior to SP12, BlueWorx could be configured so that, specific to a map and layer, a user on a device could download map data for the displayed location. This download would take a 'cone' of map tiles from low to high levels and any layers that were marked for offline use. In addition, on sync, the location of assets would determine what map tiles and layers, relative the Work Orders, would also be downloaded. These features are detailed elsewhere in this support site:

Download while Moving in the Map

As of SP12 where a user is viewing a Map that is configured for offline use, and the switch 'Hybrid Map Data Source' is on, then when online the map tiles they view will be downloaded to their applications map database only if not already there and thereafter will be available for subsequent online and offline use.


While this functionality introduces some great new functionality it has limitations. That because the data downloaded is limited to the location and map level that the user seeing on the device. So if a user was at a higher level and zoomed into a lower level, and then panned around extensively at that low level while all that map data would be available offline, there will be intermediate areas of the map for which there are no tiles.

Download and Upload

Users with permissions set at their BlueWorx user profile can download and upload map databases. Data in these databases includes map tiles and map layer data (where they have been configured for offline use). Data is limited to one database and on selection of a pre-populated database it replaces the existing map database.


While this does provide the ability for simple tile set databases to be generated and then shared it does not provide comprehensive map data  i.e. 100,000's of map tiles and layers in a database. Where this level of data is required then specialist ESRI practitioners should be used to manually populate the map tiles and layers - that being a far more efficient, faster and more accurate way of gathering the data. 

See section below An Approach to Creating Map Database for a summary of the approach our spatial services team took to creating a map database for one of our customers. 

Map Database Size and Performance Limit

The database used in SQLite which a very large technical size limit. That however does not suggest that the technical limit will work effectively at a technical nor performance level with BlueWorx. Customers should find the balance based on their needs and available technology.


Map data at each level proportionally increases by a power to 4. That is 1 tile at level 0 (the world) becomes 4 tiles at level 1 and then 4 x 4 at level 2, etc:

In the following summary, and purely to show how the number of tiles and size of the database is impacted by the area downloaded consider, the following:

In addition to map tile data, map layer data may be as important as the tiles. It may also be far larger than the corresponding map tile data. For example, consider a utilities company in a densely populated area and the requirement for the electricity network. With thousands on lines, junctions and connections that can be a lot of data to attempt to move into the device database.

Map Database Strategy

Each customer must decide whether the use of this functionality is of benefit to them. Things to evaluate include:

  • Do users need spatial data?
  • Does the customer have the rights to download the map and layer data to the device? This question is highlighted elsewhere in this support site, so this is just a reminder
  • Do users need spatial data offline or can they use WiFi or cellular data for online access?
  • If using cellular data, can savings be made in pre-populating data?
  • Does the customer have access to the spatial specialists to populate the database

As map tile data and layer data is very 'heavy' you only take what you need offline. Some things to consider having decided to pre-populate the data are:

  • Can I limit the tiles to areas without cell coverage? For customers in industries like Utilities, this could vastly reduce the offline data required as cell coverage generally exists in high asset density areas. Some cellular providers maintain maps showing their coverage so you could elect to download asset areas only where coverage is not provided
  • Can I used my asset base, and especially a network for service companies, to take mid to lower map data in those areas and wider 'area' data at the higher map levels. That's because with map tiles every map tile at every level becomes 4 maps tiles at the next lowest level
  • Can I pre-populate the database with tiles and layers use most often or at the higher levels and continue to use the existing BlueWorx functionality to download missing data on sync - i.e. start with a base and add to it over a period with the data I need
  • Do I have enough storage capacity on my device; how will it perform with a large database?
  • Can I segregate my operations into several areas so that users working in a specific geographical location can load a database specific to their area?
  • Do I need the same level of detail everywhere? For example in built up areas you may need detailed information to portray property boundaries, etc. where as in less intense areas you might be able to maps at a higher level
  • What's the currency of my map data - i.e. the map tiles may be generally static but the layer data may be subject to more frequent update. How, from a business process side, am I going to accomplish that? Note that the default settings in BlueWorx Spatial Admin specify the tile retention period.

Database Files Distribution

BlueWorx as a product does not provide, endorse, recommend nor offer any support for the product used to distribute the files. They should be selected by customers based on their business needs, devices, device data control and IT security policies. Solutions/ products in this category include micro SD cards, USB flash drives, Microsoft OneDrive, Dropbox, Google Files, etc.


What we have found, in our evaluation of several of the options outlined above, is that the data must be made available on the device through the file app, before selection from within BlueWorx - i.e. direct selection from the cloud, where large files are concerned, does not work consistently. 


We also found that the use of a USB flash drive was the easiest and most robust method of file transfer, albeit that it involves the use of physical hardware. It's use would also mean that, subject to storage on the device, multiple map database could be stored locally on the device and 'switched' to as required by users that frequently move large distances in their work.

Accessing the Map Database

You can see the content of the map database by exporting and then using a (customer IT approved) SQLite editor:

An Approach to Creating the Map Database

The following is a summary of the general approach taken to creating a map database for one of our customers and written by one of Zag's (Part of Accenture) spatial consultants. This is not intended to act as a 'how to' guide, but rather to provide general information on the tools used and strategy employed to generating Map Database content using ESRI FME.


BlueWorx has an option to read a pre-loaded SQLite database with an ESRI basemap and feature service data designed for users who wish to take the BlueWorx app offline and when mobile internet access is limited. 

An extract, transform and load (ETL) desktop application such as feature manipulation engine (FME) desktop, provides the ability to precisely target basemap and feature service data required to be uploaded into an SQLite database. The minimum requirement of FME desktop licensing is the FME Desktop ESRI edition. 

A targeted area would be preferred over a complete download of an ESRI basemap as this would reduce the amount of time required for FME to process both feature services and their associated tiles. Feature services can be used a reference point to determine tile definitions at various standard ESRI basemap zoom levels and extents. Note FME desktop transformers can help control the preferred areas by expanding or restricting an area of interest. 

FME workspace processing times will exponentially grow at higher zoom levels as more of the ESRI basemap would be downloaded and can be affected by the ESRI feature service availability and internet connection. Evaluate business requirements and technical/ performance limits to dictate the optimum level. And potentially look to download lower map zoom levels (i.e. from 0,1,2..) for wider geographical ‘orientation’, whilst reserving higher zoom level tile downloads  for feature layer driven locations as discussed in the previous paragraph.

For any feature services required to be uploaded into the SQLite database, the URL of the feature service and appropriate access is required by FME Desktop so the application can call the services to extract mandatory metadata and legend information. The same applies to the ESRI basemap required to download. Note that any ESRI basemap can be downloaded that’s compatible with your organisation’s BlueWorx setup (check the BlueWorx configuration files to be sure) and recommended to only use one ESRI basemap to limit the SQLite database size for mobiles. 

The ESRI basemap and feature services will have to be projected in the same coordinate system. When downloading ESRI basemaps, generally they are in the the standard EPSG:3857 / Web Mercator projection system. The FME desktop application has the necessary transformers required to reproject any recognized coordinate system to either match or to another compatible geographic coordinate system. 

Once they are compatible, FME transformers can be used to link a feature to the ESRI basemap tile using a one to one or one to many relationships. Also, the transformers will be able to extract GeoJSON data from features and convert feature service attributes to a JSON format to be uploaded into the SQLite database tables.