GIS Data Structuring and Mapping



Objective:

Develop a cartographic framework and interface that effectively emulates the functionality of Google Fusion Tables

Tasks:

  • Create structure through which coordinates and attached data are realized
  • Visualize data points using interactive map
  • Create custom URL and structure logic to allow for specific data queries
  • Develop compiler that takes company exported CSV data and converts it into usable data structure
GIS Interface showing Russian mine

Approach:

        Replicating the powerhouse that is Google Fusion Tables requires three core functions: the realization of data points on a map, the ability to manipulate data with ease, and the option to query data points based on their properties. The GUI interface and point realization is handled by Leaflet, an open-source Javascript library for interactive mapping. Since the code must run standalone, the data must reside within the framework itself. In order to feed Leaflet/Javascript friendly data into the map, I wrote a executable that takes a tab-delimited CSV file (exported by the company) and converts it into a Javascript array. Since we cannot use SQL to query from this data, I coded personalized operators and functions that emulate exactly what SQL does. For example, here is a comparison between SQL logic and my own logic:

SQL
SELECT * FROM mapData WHERE Filter02 = 282 AND Filter03 = 2 AND (Filter04 = 1 OR Filter04 = 3) AND (Filter05 >= 3 AND Filter05 <= 6)

Proprietary
F2=282+F3=2+(F4=1/F4=3)+(F5>=3+F5<=6)

        Both of these queries return 10 data points, fulfilling the following conditions: All pipes in Russia that are diamondiferous which are kimberlites or lamproites and which are at advanced exploration, future mine, mine or former stage. The filters are unnamed since company operators already know what they are, but they could be renamed to their colloquial terms (Filter02 = numeric country code, Filter03 = Diamond status, etc.) if necessary. Here is that specific query in action, implemented through a URL parameter:

The URL for this queried map would look something like:
        KaiserArts.com/map.html?lat:64.3&lng:112.4&zoom:5&filters:F2=282+F3=2+(F4=1/F4=3)+(F5>=3+F5<=6)
Along with it are other URL parameters, in this case, starting coordinates and zoom level.

Methodology:

         The method through which the logic is evaluated utilizes three main functions: the parser, the assessor, and the and/or operator. The parser takes in the raw logic text and parses each group and its following operator into an array. This means that if there is a section in parentheses, it will place the entire content into the array; since it is the sum of parenthetically distinguished logic alone that matters, this allows the parser to then later pass through the relevant content once more. After parser has broken down the string into individual indices, the assessor then reads the array and employs the necessary operator for each string. There are two operator functions, the “and” operator and the “or” operator, and these crosscheck the query with the internal data and return true if it matches. When the operator function comes across a previously parenthetic index that has more than one comparison, it simply runs the parser on said index and recursion finishes the job.
        Here is a look at the logic code found within the and operator:

Code demonstrating GIS Interface's query logic

Set is the string array that assessor has passed to the and operator, which in the case of logic F2=282+F3=2+(F4=1/F4=3) would have a size of 3 and look like: {“F2=282”, “F3=2”, “F4=1/F4=3”}. The method will check to see if any of these queries fail, in which case it will return false. When it reaches the third index, it will recognize its non-diminutive form and run parser and assessor on it to determine if the logic passes (the code for this is:
valid = (assessor(parser(set[i]).
         While the code within each sub-operator of the and method may look complicated, it is quite simple: It begins by identifying the operator and establishing its position, as it may vary depending on the Filter’s number of digits. It then compares the internal data point’s filter data with the queried data and sets a boolean to false if they do not match. The method ultimately returns the boolean, which will be true if all data points matched, and false if at least one did not. The or function works similarly, though it returns true if at least one query from the set is valid, and false if none are.

Final Product:

         The finished program along with its compiler is a customizable and sustainable replacement for the now discontinued Google Fusion Tables, and requires no subscription of any sort to maintain. The compiler produces backups of the previous data any time it replaces it. The javascript draws vectors made of simple polylines in order to maximize efficiency, making this program feasible even for thousands of data points (though loading times do increase with such vast amounts). Below is a map of about 4300 individual data points, each with its own HTML formatted data ready to pop up upon click.