Visualization in Smallworld

(the cool stuff/demo is at the end of this longer-than-normal post)

Lately I have taken an interest in Visualization techniques within the Smallworld context. I am using the term "Visualization" in the sense of using visual representations to simplify complex data relationships. The term has various other related meanings as well.

In a sense, we work with Visualization techniques in Smallworld every day... we deal with a visualization of geographic relationships between objects that are of interest to us. When you create a report using the Schematics Generator you are using visualization techniques to help you comprehend complex relationships by keeping only the salient information. Any Thematic Mapping tool that you might use also lets you visualize complex data relationships easier.

I have been intrigued by the use of heat maps to provide a way of showing intensity of some attribute spatially. I still haven't got my mind around all the applications that might be used in the utility space for heat maps. Some that come to mind are: heat map of all the transformers that are about to blow up; heat map of all the outages in a storm. While the pictures look cool, I'm trying to better understand what value a heatmap image would bring to GIS users that cannot be already provided by other kinds of thematic mapping applications.

I have a feeling that the full potential for visualizations such as heat maps and density maps has not been realized within the Smallworld realm but I sure could use some ideas from readers about what possible applications might be.

My colleague Graham Garlick has done alot of work with generating heat maps, density maps and TIN density maps within Smallworld and I want to put a plug in for him in this post. If you are interested in knowing more about how to generate these kinds of maps in Smallworld, please contact iFactor Consulting and ask for Graham's heat map demo.

So now to the final point of this post. The visualizations that I have already referred to are typically georeferenced rasters that overlay on a map and give you a sense of the spatial relationships of your data. I have been exploring another area that involves network visualizations.

My first foray into this realm was by making use of a software package called Graphviz to visually show the results of network follower trace results in PNI. This was really helpful in debugging network follower results. Typically, a Magik developer debugs network follower trace results by querying the nf_links of a network follower and trying to understand how they are linked. Using a visualization tool like Graphviz allowed me to easily create a .jpg file that I could view in any image viewer. By looking at the image output, I could easily spot places where the trace stopped unexpectedly or went down an unexpected branch. Sometimes a picture really is worth a thousand words (or a thousand minutes of Magik debugging!!!)

My second attempt at network visualization came from a slightly different tack. If visualization techniques are meant to represent complex interdependencies in an easy-to-understand manner, what is a really complex difficult-to-understand scenario in Smallworld that I could try to visualize? The answer to that question is: XML configuration file dependencies. Anyone that has ever tried to modify config.xml, gui.xml or a host of other XML configuration files in Smallworld will tell you that it can be somewhat mystifying as to which version of the file to change. Any given XML file can inherit from another XML file and its elements can supersede the same elements of a "superclass" XML file. In addition to that, XML files are considered module resource files. That means that they can be placed in customisation products which have a different pathname than the same module found in the core product. If this all sounds confusing, rest assured it is. I have seen many instances (and caused some myself) where I have changed the wrong XML file or changed too many "inheriting" XML files when it would have been more efficient to change a "superclass" XML file. But it was always difficult to get my mind wrapped around the big picture of XML file dependencies. Until now...

I have take a bit of time to write a Magik wrapper around the Graphviz engine. That in itself is not that remarkable and anyone with interest in that could do it. What is of more interest to me is the result that I was able to create with this Graphviz tool showing the interdependencies of XML files. As an example, click here to see the dependencies of the Electric Office config.xml files for applications.

• If you look at the image, you will see information about which XML files inherit from others.
• You will see the full pathname of each file. This will help you know whether to look in the actual module resource file or in a corresponding customization product file
• You will see some cells that are green. These are the cells that represent XML file elements that have a material change relative to a "superclass" file.
• You will see some cells that are red. These are cells that represent XML elements that exist in a configuration file but whose contents are no different than the same element in the inherited XML file. Many times it can be confusing to have the exact same XML element contents in two files, so looking at this visualization gives you an easy-to-understand representation of which duplicate XML elements you can remove.
• You will see some cells that are yellow. These represent XML elements that require further investigation. They indicate that the current XML element is an exact duplicate of another XML element somewhere in the graph. If you notice yellow cells in two XML files that share the same parent, you might want to consider moving those XML elements up the inheritance ladder.
• And finally, just seeing the overall flow of XML inheritance can give you a quick indication of whether you have too many dependencies or maybe where it would make the most sense to insert a suitable XML file "superclass" that would serve your own customizations most efficiently.

This was just a quick demonstration of how a network visualization tool can be used to simplify complex interdependencies in the Smallworld environment. I would love to hear from readers about other complex scenarios that might lend themselves to this sort of network visualization.