Regular Expressions in Magik

It has been bothering me for years that Magik does not seem to support Regular Expressions. I tried to make do with the ro_charindex_mixin.matches?() method. But that method only uses wildcard characters %*, %? and %\.

So when l_santi asked a question in sw-gis about Magik support for Regular Expressions, it got me thinking: WWBD (What Would Bhimesh Do)? Bhimesh is the owner of the Magik Fun blog and many of his posts explore the interaction of Magik with various OLE objects. I began to wonder if there was an OLE object that could support regular expressions. It turns out that vbscript.regexp is exactly what I was looking for.

Here is an example of testing that a telephone number string matches the correct format for North America...

MagikSF> regexp <<  ole_client.createobject("vbscript.regexp")
$
MagikSF> regexp.pattern << "^[0-9]{3}-[0-9]{3}-[0-9]{4}$"
$
MagikSF> regexp.test("303-555")
$
False
MagikSF> regexp.test("303-555-5555")
$
True

... and another example testing if a string is in a valid e-mail format.

MagikSF> regexp.pattern << "\b[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,4}\b"
$
MagikSF> regexp.test("alfred@ifactorconsulting.com")
$
True
MagikSF> regexp.test("alfred!ifactorconsulting.com")
$
False

Smallworld has found its voice

Bhimesh over at MagikFun provided an example of how to get Smallworld to talk.

Here's my attempt at getting Magik to talk in multiple languages...

_block 
  v << ole_client.create_object("sapi.spvoice")
  v.speak("ggooten abendd.  vilkommen")
  v.speak("good evening. welcome")
  v.release_object()
_endblock 
$

The German spelling is atrocious, but I found that you sometimes have to butcher the spelling to get the voice software to do a better pronunciation.

Imagine the possibilities now that Smallworld has found its voice!

Magikfun

I wanted to announce a new Smallworld-related blog at http://magikfun.blogspot.com. Welcome, Bhimesh, to the relatively new space of blogging about Magik. One of his first posts is about creating a 3D pie graph in Excel using Magik (http://magikfun.blogspot.com/2009/11/3d-pie-diagram-in-excel-from-magik.html). Looks really cool and the code is not that complex.

Smallworld Camp

As I was returning from the GE Smallworld Americas Users Conference last week, I began to reflect on the pros and cons of my experience there. A highlight of the trip was the networking I was able to do with colleagues, customers, potential customers, etc. I was able to sit in on a number of presentations, too. It turned out that the ones that I found most interesting were the ones that had some kind of live demo or at the very least some code/pseudocode that showed how a particular problem was solved.

So it seems that I have a greater attention span for technically detailed presentations than I do for high-level marketing presentations. And that got me thinking about what options were available for some kind of different way of having technical interactions in the Smallworld/Magik realm than the typical conference presentation format. My research "found" that there is exactly that kind of "different" format available in the wider coding/techie world. It goes by names such as FooCamp, BarCamp, CodeCamp, unconference.

As I read more about it, I thought "why not a Smallworld Camp?" My colleague Graham and I get together for "coffee and code" sessions where the content is completely participant-driven. And I always get alot out of those sessions. How difficult would it be to coordinate a Smallworld Camp if its participants were all motivated participants? I have not heard of any kind of Smallworld/Magik type of BarCamp. If anyone has participated in such an unconference, please let me know. I would love to hear how it went.

While waiting to hear if any other such participant-driven events have been planned, I present to you Smallworld Camp. I have borrowed heavily from the BarCamp.org wiki template. All BarCamps are about open sharing and education. A Smallworld Camp is about this open interaction within the Smallworld developer community. Having an accessible repository for presentation documents and demo code contributes to that openness.

Why participate in a Smallworld Camp?

• strengthen the Smallworld/Magik community. Having better educated Smallworld developers who also share their experiences with others can only help to make the Smallworld technology more attractive to new customers.
• get real live code examples
• meet old friends and make new connections
• give back to the community that provides us our livelihood
• improve your Smallworld/Magik skillset by receiving and providing mentoring
• more interaction than a typical conference presentation

I hope to facilitate the first Smallworld Camp in the Boulder/Denver area some time between now and GITA. The only way it works is for more than 1 person to be part of it so if you are interested, please jump in and contribute to the planning at http://smallworldcamp.pbworks.com/SmallworldCampBoulder2009

I have never participated in a BarCamp or CodeCamp, but am fascinated by the concept. If you have any comments to share about your experiences in a BarCamp, please leave a comment. I would be very interested to hear if you think such a model would work in the Smallworld community.

As a final note, Smallworld Camp is not endorsed nor supported by GE. As a matter of fact, no one organization may ever plan a BarCamp. The camps are by and for developers. Corporations are encouraged to sponsor various parts of the event but all content is always driven by the participants.

Dialog Designer 1.9.5 now available

Graham Garlick at iFactor Consulting has uploaded v1.9.5 of the open source Dialog Designer. You can get the latest version by browsing to http://www.ifactorconsulting.com and clicking on the "Download Version 1.9.5 Now..." link.

Here are the release notes for 1.9.5:

- create additions/changes sub-source directories for core code modifications

- bug fix (statusbar progress indicator element XML generation/parsing)

- exposed :allow_filtering? attribute for a tabular_list

- fixed statusbar rendering of multiple springy panes

- fixed window_pane rendering

I did not do a post for 1.9 so here are those release notes as well:

- added messages tab (define messages to support user-written code)

- fix issue where module.def 'requires' section was not updating

- various bug fixes, all minor.

- added recordset_gui_component widget

- added numeric input widget

- added 6th example dialog to the Help Document

If you have not used this tool, I would highly recommend it. You can create your GUIs without using this tool, but there are so many parameters to be aware of. Using this tool essentially allows you to incorporate known (and sometimes unknown) tips, tricks, workarounds and best practices from one of the best Magik coders around and put it into your own application.

Smallworld High Definition (part 2)

(My apologies to those who get their blog post comments on an RSS feed. This is a repeat of a comment from last night)

At the Users Conference in Glendale this week, John Eason was very excited to take me to see the "Smallworld HD" demo. All I can say is that there was some really cool stuff. With mostly only Magik changes, the developer (a Magikian in my mind) created a new interface that out-Microsofted Microsoft. The new GUI has all the standard Microsoft UI design elements but with a few iPhone-like tricks thrown in. The graphics are clear and crisp (a la the comments from "Anonymous"). The widget classes (as well as the geometry classes) now all inherit from a common set of ancestor classes to take advantage of new canvas rendering functionality and geometry caching.

As far as I am concerned, the new GUI will be the cool kid on every user's Desktop. I expect that non-GIS users will want a copy of the GUI on their desktop just to play with it and show off its cool functionality. The feature that I think is most like the iPhone is the one where you can grab the map with your pan cursor and "flick" it in a particular direction. When you are scrolling down the object list (think combination of Smallworld Explorer result list and Google Maps search result list) you can grab an object in the list and "flick" it up or down to scroll the map. While those of us who have worked with Smallworld for a long time have know of the software's power, I think these GUI changes will make the software more attractive and inviting for first-time users to interact with. There is definitely an initial "wow" factor that is combined with functionality that gives this GUI a type of "stickiness" as well. "Good job" to the team that is working this and I wish you much success in seeing it through to delivery.

If you are at conference this week, it is worth checking out this demo at the GE booth.

(According to new US Federal Trade Commission rules, bloggers are now supposed to reveal any freebies or payments they get for reviewing a product. Unfortunately for me, no freebie was provided. I can only hope that the beta version comes out soon.)

porting sw_dotnet_acp to a TICS framework

Pedro Miranda (http://magikbites.blogspot.com) has contributed the very cool sw_dotnet_acp code to the Magik Components SourceForge community (aka mclib). Recently someone asked me if I had used this library with the TICS framework instead of with the ACP framework. I had not done that but thought that would be an interesting question to ask my readers.

If anyone is interested in collaborating on a sw_dotnet_acp port to something like sw_dotnet_tics in the context of the mclib community, please post a comment to this post or send me a note to the contact information on the sidebar of this blog.

How do interact with FME in Smallworld?

I was asked a question yesterday at the conference about how I interact with FME from Smallworld and thought I would start an informal survey of readers to see what your responses are.

The question is: How do you interact with FME from Smallworld?

1) Do you see FME as an application that is launched from a Smallworld session?

2) Or do you prefer to use FME more as a standalone tool that connects to a Smallworld server process?

My own preference is to see FME as a standalone tool that makes a connection to a Smallworld server. That way any business analyst that does not have a Smallworld session running could still make a connection to a central Smallworld server port and run the workspace. Having this configuration would also facilitate use of FME Server with Smallworld. This method of interaction would be great for reading from Smallworld. But I suppose that writing to a Smallworld database might make one lean towards option#2.

With option #2, you as a Smallworld user could control which alternative the data was being written to and then you could do a final check of the data in your alternative before you committed it.

So I suppose that my preferred mode of operation includes both above-mentioned options. A "FME connects to Smallworld server" option for reading data and for selected controlled write operations. And a "Smallworld launches FME" option for the more ad hoc write operations.

I'm curious what your thoughts are.

Version Viewer

My colleague Graham Garlick has been working on a very cool Version Viewer application. It is essentially a GUI that allows you to view/compare two different versions of a dataset. It lets you compare geometries and attributes and determine what has changed between the two versions. Imagine Smallworld's conflict detector's Affected Objects Lister but the Version Viewer is easy for you to use in non-conflict situations.

What could you use the Version Viewer for?
- recovering attributes/records from data that is available in an older view
- analysing your data BEFORE a merge/post. You could build this into a pre-emptive conflict avoidance process
- selective merge/post by attribute
- data quality checks to see the who/what/when of changes in your data. Very helpful for investigating the inevitable question: "How/why did that data get changed and in what context were the changes made".

Graham also is responsible for the Dialog Designer. He used the Dialog Designer to build the complex Version Viewer GUI. As part of the Version Viewer demo, he can show you how you can dynamically change the GUI layout of that application using Dialog Designer.\

Please come by the iFactor Consulting booth at the conference and ask for a demo by Graham. If you are not able to make it to the booth, sent me a note at the contact information on the blog sidebar and I can arrange a demo.

How to make your data positionally accurate

iFactor Consulting is getting some great interest in our Web Maps Connector here at the GE Smallworld Users Conference. But one question we have been getting is "how do we line up our data with your web maps?".

Well, Graham Garlick has an answer to that. He has a presentation at the conference on Tuesday at 9:15 that presents a tool/process that he has developed to answer this question. His presentation is called "Make your GIS data positionally accurate: Why? How?".

If you can't attend his presentation, please come by the iFactor booth to get a personal demo.

See you in Arizona!

I have been looking forward to this year's GE Smallworld Americas Users Conference which will be held in Glendale, AZ starting this Sunday. I will be at the iFactor Consulting booth. Please drop by to say hello. I look forward to meeting up with as many of you as possible.

We can talk FME, Magik, Bing Maps, Google Maps, cycling or even why 102°F weather in October in the Northern Hemisphere does not seem right to me :)

Getting more from tracebacks

Now that I've spammed my readers with a useless "test post via email" note, I will try to post something of value.

I have always been fond of saying "tracebacks are our friends". In my mind, tracebacks are not to be feared... the bigger the traceback the better because that means that I have more information available to me to debug the problem.

A common debugging technique is to put a call to the !traceback!() procedure in any method from which you want to determine how the call stack arrived at that method. Placing a !traceback!() call in your code won't raise a condition or stop the code flow. It will simply give you a dump of the call stack at that point.

To demonstrate how this works, I have put a !traceback!() call in the Web Map Connector's ve_tile.draw_on() method. The traceback looks as you would expect...

---- traceback: renderer (heavy_thread 3611775) ----
time=14/10/2009 17:00:21
sw!version=4.1.0 (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=False

!traceback!()
a ve_tile(detached) .draw_on(a sw:canvas)
sw:simple_vector:[1-11].fast_keys_and_elements()
sw:keyed_set(1).valid_indices(False)
sw:keyed_set(1).fast_elements()
sw:keyed_set(1).elements()
transient_rwo_collection(road_tile).fast_elements()
sw:simple_vector:[1-11].fast_keys_and_elements()
sw:hash_table(1).valid_indices(False)
sw:hash_table(1).fast_elements()
ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
a sw:uds_geometry_set(ve_dataset,0,0).elements()
a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
sw:rope:[1-4].elements()
a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
a sw:canvas.draw(a sw:composite_geometry_set)
map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
map_view(map(Electricity),region).int!do_render(unset)
renderer(map_view(map(Electricity),region), unset)
light_thread_launcher_proc_990928()

There is a global variable !snapshot_traceback?! that you set to _true to give you a bit more information.

MagikSF> !snapshot_traceback?! << _true
$

Now the traceback looks like...

---- traceback: renderer (heavy_thread 3563153) ----
time=14/10/2009 17:01:44
sw!version=4.1.0 (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=True

[1] !traceback!()
[2] a ve_tile(detached) .draw_on(a sw:canvas)
[3] sw:simple_vector:[1-11].fast_keys_and_elements()
[4] sw:keyed_set(1).valid_indices(False)
[5] sw:keyed_set(1).fast_elements()
[6] sw:keyed_set(1).elements()
[7] transient_rwo_collection(road_tile).fast_elements()
[8] sw:simple_vector:[1-11].fast_keys_and_elements()
[9] sw:hash_table(1).valid_indices(False)
[10] sw:hash_table(1).fast_elements()
[11] ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[12] a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[13] a sw:uds_geometry_set(ve_dataset,0,0).elements()
[14] a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
[15] sw:rope:[1-4].elements()
[16] a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
[17] a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
[18] a sw:canvas.draw(a sw:composite_geometry_set)
[19] map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
[20] map_view(map(Electricity),region).int!do_render(unset)
[21] renderer(map_view(map(Electricity),region), unset)
[22] light_thread_launcher_proc_990928()

... as you can see, each traceback line is numbered. This is kind of handy to have. The real benefits of !snapshot_traceback?! are described in the public comments for this global variable.

What I wanted to show you is a modification I made to the !traceback!() procedure that gives you even more helpful information whilst !snapshot_traceback?! is _true.

Here is my hack of the !traceback!() procedure.

#% text_encoding = iso8859_1
_package sw
$
_global !traceback! <<
_proc@traceback(_optional stream, max_depth, full?)
       ##  Display traceback of calls in the current thread on
       ##  STREAM. If max_depth is given, at most that number of
       ##  stack frames will be displayed.
       ##
       ##  If !traceback_show_args?! is true, then the arguments will
       ##  be shown along with the method name.
       ##
       ##  Tracebacks normally exclude stack frames after a
       ##  condition.raise(). Setting FULL? to _true will force the
       ##  full traceback to be shown. I.e including tracebacks of any
       ##  condition handlers.

 _thisthread.traceback(stream, !traceback_show_args?!,max_depth, full?)

 _global !snapshot_traceback?!

 _if !snapshot_traceback?! _is _true
 _then
  
  _global !ts!
  _local sfile , meth

  write("These numbered source files correspond to the methods in the numbered lines in the traceback above.")

  _for i,v _over !ts!.fast_keys_and_elements()
  _loop
   meth << v.receiver.method(v.message)
   
   _if meth _is _unset
   _then
    show(write_string("[",i,"]"))
    _continue
   _endif
   
   sfile << meth.source_file
   _dynamic !show_string_length! << sfile.size
   show(write_string("[",i,"]"),sfile)
  _endloop
 _endif
_endproc
$

You can see that it calls the standard _thisthread.traceback() method, but after that it takes a list of all the methods/procedures that were reported in the traceback and shows their source files as well...

---- traceback: renderer (heavy_thread 3514507) ----
time=14/10/2009 17:05:41
sw!version=4.1.0  (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=True

[1] traceback()
[2] a ve_tile(detached)  .draw_on(a sw:canvas)
[3]               sw:simple_vector:[1-11].fast_keys_and_elements()
[4]             sw:keyed_set(1).valid_indices(False)
[5]           sw:keyed_set(1).fast_elements()
[6]         sw:keyed_set(1).elements()
[7]         transient_rwo_collection(road_tile).fast_elements()
[8]             sw:simple_vector:[1-11].fast_keys_and_elements()
[9]           sw:hash_table(1).valid_indices(False)
[10]         sw:hash_table(1).fast_elements()
[11]       ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[12]     a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[13]   a sw:uds_geometry_set(ve_dataset,0,0).elements()
[14] a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
[15]   sw:rope:[1-4].elements()
[16] a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
[17] a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
[18] a sw:canvas.draw(a sw:composite_geometry_set)
[19] map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
[20] map_view(map(Electricity),region).int!do_render(unset)
[21] renderer(map_view(map(Electricity),region), unset)
[22] light_thread_launcher_proc_990928()
These numbered source files correspond to the methods in the numbered lines in the traceback above.
"[1]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"
"[2]" "C:\DOCUME~1\Alfred\LOCALS~1\Temp\TAlfred9412q0"
"[3]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/ro_indexed_collection_mixin.magik"
"[4]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[5]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/convert2_sf/hashing/hash_values_helper.magik"
"[6]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[7]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/ds_src/dd/dd_collection_mixin.magik"
"[8]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/ro_indexed_collection_mixin.magik"
"[9]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[10]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/convert2_sf/hashing/hash_values_helper.magik"
"[11]" "C:/ifactor/products/workspace/Virtual_Earth_Connector/trunk/modules/ve_som/source\ve_dataset.magik"
"[12]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/uds_geometry_set.magik"
"[13]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/uds_geometry_set.magik"
"[14]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/geometry_set_mixin.magik"
"[15]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/rope_mixin.magik"
"[16]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/composite_geometry_set.magik"
"[17]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/mgr_src/user/drawing_surface_mixin.magik"
"[18]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/mgr_src/user/drawing_surface_mixin.magik"
"[19]" "C:\ifactor\products\workspace\Virtual_Earth_Connector\trunk\source\patches_z_proposed\p1000001_1_if_vec_map_plugin.magik"
"[20]" "C:\ifactor\products\workspace\Virtual_Earth_Connector\trunk\source\patches_z_proposed\p1000001_1_if_vec_map_plugin.magik"
"[21]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"
"[22]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"

This can be very helpful when you are trying to debug why certain code works in one of your environments but not the other. It can be very helpful to the GE Help Desk team because sending this enhanced traceback will tell them how your call stack is different than the one they are testing in. It will show which methods were changed in patches and in custom code.

Test post via email.

Preventing ACE posting

It has long been known that posting up ACE and/or CASE alternatives may have unintended consequences when the parent/child alternatives have records that are in conflict. If you ever want to prevent all users from posting an ACE alternative, you can use this code I wrote to do that.

This code snippet has been written in a patch format. You can read more about how to create your own patches here.

The define_method_synonym() approach is described here. Note that because the core method database_view.post() source code is not shipped with the product, we need to use the define_method_synonym() approach to wrap our own custom functionality around it without modifying the core restricted functionality.

#% text_encoding = iso8859_1

_package sw

$

sw!patch_software( "ds_src", 1 )

$

# use define_method_synonym() to wrap some of our behavior

# around the core method without redefining the core behavior.

_if database_view.method(:original!post|()|) _is _unset

_then

database_view.define_method_synonym(:original!post|()|,:post|()|)

_endif

$

_pragma(classify_level=restricted, usage={redefinable})

_method database_view.post(_gather args)

## Post changes in this alternative to the parent,

## performing an implicit commit() if any updates are

## outstanding.

##

## Returns _true if there were changes to post, _false

## otherwise.

##

## If the data dictionary has been modified, then the modified

## version must have been allocated a datamodel version first.

##

## External databases are checked to ensure that if they are

## updated, that they are prepared to commit (i.e. they are

## running). An error is raised if one is not prepared to

## commit.

##

## THIS REDEFINED VERSION OF THE METHOD HAS BEEN MODIFIED TO

## DISALLOW POSTING ACE ALTERNATIVES.

_if _self.name.canonical _is :ace

_then

condition.raise(:cannot_post_ace_database)

_endif

_return _self.original!post(_scatter args)

_endmethod

$

_pragma(classify_level=restricted, usage={redefinable})

## raised by database_view.post() when database_view is named

## :ace.

condition.define_condition(:cannot_post_ace_database,:user_error,{})

$

read_message_patch(:condition,_unset,:en_us)

$

:cannot_post_ace_database Posting ACE alternatives is not safe and has been disabled.

$

sw!declare_patch(1000000001,"1cst","Prevents posting ACE alternative.")

$

Smallworld High Definition

John Eason at GE was giving me grief for not posting more frequently to this blog. So here's a post for John :)

Come to the Smallworld Americas Users Conference in Phoenix in October and learn about "Smallworld High Definition". No other details were forthcoming so I had to make my own guesses.

Smallworld High Definition might mean...

- you can now see the Smallworld GUI from across the room

- even if your swmfs server goes down, Smallworld tries to guess what your data really looks like (kind of like HD TV guessing pixels when the signal is bad)

- Smallworld will be deployed in an Xbox-like console so that you can bring your data into your home theatre.

- Smallworld will allow embedding multi-media in database objects. I'm not talking about the existing related-documents functionality. I'm thinking more along the lines of Google Maps showing WikiPedia, Panoramia and YouTube.

- Smallworld data and apps go 3D.

Anyone have any thoughts on what Smallworld HD could mean?

Personally, I'm hoping that Smallworld HD might be something along the lines of multi-media integration or 3D rendering. That would be cool!

Well, enough posting... time to get back to work... and John will probably have to wait some time again before my next post.

See you all at the conference in October!

Map View vs Application vs Trail Coordinate System

In setting up the iFactor Virtual Earth Connector™ product for a few customers (both actual and evaluation), I have seen the flexible Smallworld projection functionality take more of a front-and-center role for a customer.

Typically customers have set up their SWAF application to use a particular application coordinate system and then they forget about it.

With the VE Connector, rendering can happen in any coordinate system but as you can probably appreciate, the fastest and highest quality rendering can happen if the current map view's coordinate system is the same as the coordinate system that Microsoft's Bing Maps (formerly Virtual Earth) were cached at. So the VE Connector comes with a pre-defined coordinate system (we call it "Virtual Earth") that switches the mapview coordinate system to the same one used by Bing Maps.

Changing the mapview coordinate system has typically caused our customers two "surprises".

The first surprise is that their trail measurements are now different than they used to be. The second surprise is that their application coordinate system is different that it used to be.

Both these "surprises" indicate that the trail coordinate system and the application coordinate system have been "locked" to the mapview coordinate system.

You can change these settings by choosing Tools -> Options from the menu bar.

To change the application coordinate system so that it is always what you want it to be regardless of the current mapview coordinate system, select Geometry Calculations in the Options panel.

To change the trail coordinate system so that it is always what you want it to be regardless of the current mapview coordinate system, select View Defaults in the Options panel and "unlock" the trail coordinate system from the mapivew coordinate system.

That's all there is to it. Remember that changes to the Options panel are typically saved in the ACE so for your changes to be set for all users, you will need to take the usual steps involved with propagating ACE changes down the ACE alternative structure.

Simplifying Complex Code

I saw an example in code today that cried for simplifaction. Actually, it reminded me of the first piece of code I had reviewed by someone in Cambridge and it got hacked to pieces. It was a bit hard to swallow, but that kind of feedback definitely helped me in the long run.

So here is the perfectly logical, but unnecessarily complex piece of code...

_method cit_circuit.is_bearer?

l_bearer? << _if _self.cit_circuit_type.code = "Bearer"

_then >> _true

_else >> _false

_endif

>> l_bearer?

_endmethod

$

... which can be greatly simplified to...

_method cit_circuit.is_bearer?

_return _self.cit_circuit_type.code = "Bearer"

_endmethod

$

Any time you see an if/then/else block of code that returns a boolean that should raise a red flag for you that your code structure is too complex.

View Microsoft Virtual Earth data in Smallworld

I'm very excited to share with you all that the company I work for (iFactor Consulting) earlier this week released Version 1 of the Virtual Earth Connector for Smallworld. You can read the official announcement here.

The VE Connector makes use of Smallworld Core Spatial Technology Spatial Object Manager (SOM) functionality to render landbase and aerial imagery tiles served up by Virtual Earth on your Smallworld map view.

The product page has a demonstration video that shows the functionality. If you want to try it out for yourself, you can get a full version evaluation release here.

It seems like all the big players (ESRI, MapInfo, Oracle Spatial) are going in this direction and now iFactor presents this capability for Smallworld.

I would like to acknowledge Brad Sileo (iFactor VP Business Development by day and mclib master by night) for getting the original product code going. All of the iFactor executives are also master coders!

We used alot of standards-based functionality to get this to work. Let's see how many buzzwords I can remember :) : Source Forge mclib, C#, SOAP, Web Services, SWAF, SOM, ACP.

Incidentally, from the Magik Components Source Forge library, we made good use of:

• sw_dotnet_acp (thanks to Pedro Miranda for making this code available in mclib!)
  
• mclib_package
• mclib_http_interface (thanks to Tony Sileo and Brad Sileo for contributing to this!)

So, here's a plug for Magik Components: It can greatly help you with your own projects, too!

We are operating under an agile "release early, release often" model. So, once you have the product loaded, you can check for updates regularly in the "About" dialog and download the latest revision as soon as it becomes available.

Modifying core method behavior without touching core code

The other day I was writing some patches for a custom product (see Developers: patch it yourself). One of the requirements is that the product code needs to be supported at 4, 4.1 and 4.1.1. The change is to add a new dynamic variable to the beginning of map_view.int!do_render().

Originally, I thought I would have to make three different versions of this patch file (one for each CST version that the product supports). That would seem to make sense because at each of the versions, the contents of the method could have changed slightly. As it turns out, the contents had changed between two of the versions.

Having three version-specific patches for a single method ends up being an administrative nightmare because each time a new TSB is released, I need to review all the changes in the TSB for conflicts with my patch code. In some cases this might be necessary because the patch makes significant changes in the middle of the method. But in many cases, the changes we want to put in are at the top or the bottom of the method. In those cases, I discovered that I could use method :define_method_synonym() to wrap my new code around the core code without touching any of the functionality in the core code...

# use define_method_synonym() to wrap some of our behavior
# around the core method without redefining the core behavior.
_if map_view.method(:original!int!do_render|()|) _is _unset
_then
map_view.define_method_synonym(:original!int!do_render|()|,:int!do_render|()|)
_endif
$
_pragma(classify_level=restricted)
_method map_view.int!do_render( _optional post_render_set )
##
## Draws the current visible set in the render thread.
##

_dynamic !ve_connector_current_map_view! << _self

_return _self.original!int!do_render(post_render_set)
_endmethod
$

And if you use the _gather/_scatter keywords, your wrapper does not even need to worry about whether the arguments for a method change from version to version.

This technique is also useful for putting debug code into unshipped source code. Let's say I want to get a traceback every time I get a user_error dialog with the word "coordinate" in the dialog message. The dialog is actually activated somewhere in basic_window.show_alert(). This method code, however, is unshipped so your test code might look like...

_if basic_window.method(:original!show_alert|()|) _is _unset
_then
basic_window.define_method_synonym(:original!show_alert|()|,:show_alert|()|)
_endif
$
_pragma(classify_level=restricted, usage={redefinable})
_method basic_window.show_alert(message, _optional yes_message, no_message, default, mode)

_if message.canonical.index_of_seq("coordinate") _isnt _unset
_then
 !traceback!()
_endif

_return _self.original!show_alert(message,yes_message,no_message,default,mode)
_endmethod
$

NOTE: the method comments on :define_method_synonym() say that it is "severely deprecated." But until the time that this method is removed from the core code, I find it a very useful tool.

NOTE NOTE: While this technique does minimize the touch points of your code with the core code, you still need to be aware of the classifications (basic,advanced,restricted) of a method to understand how/if you should modify the method.

Determining map_view Pixel DPI

The other day I was trying to figure out the pixel DPI on my map_view so that I could size a raster_image on the map. I had always thought that !window_system!.screen_resolution() would do that for me. It seemed to work as long as I put in some additional empirical fudge factor. I discovered later that if I am working with map_view, I can use map_view.pixel_size to calculate DPI. map_view.pixel_size returns the size of a pixel in millimetres so to convert that to DPI...

_method map_view.if!dpi
      ## if!dpi : integer
      ##
      ## returns _self's DPI

      _return (1/_self.pixel_size) * 25.4
_endmethod

Exposing hidden data from Smallworld to FME

A member of the sw-gis group recently posted a question asking how to export join attribute information from Smallworld via FME. I responded to the group but thought the answer might be worth sharing here as well.

Question:

I'm using FME with Smallworld 4.0. How can I export join attributes (e.g., export transformer bank id with the transformer)? I coudn't find it in the FME workbench.


Answer:

I think the easiest way to do this is to use the fme_pseudo_field_factory.

For example...

If you’re your record exemplar is :transformer and the transformer_bank ID attribute is called :transformer_bank_id, then you can define a new FME Pseudo Field...

_pragma(classify_level=basic, usage={redefinable})
## used by FME SWAF interface to allow a customizer to add
## pseudo fields to the schema for Smallworld tables to be
## passed to FME.
transformer.define_shared_constant(:fme_pseudo_fields,
                                 {fme_pseudo_field_factory.new_derived_field(:transformer_bank_id,:extdb_string)},
                                 _false)
$

You can actually define more than one fme_pseudo_field for a given record exemplar if you want to.

Declaring :transformer_bank_id as a new derived field will make this "field" available to the FME feature. When FME asks a "transformer" for its :transformer_bank_id, it will simply call transformer_bank_id and use that value in the workspace.

There is nothing special about the naming convention of the pseudo field. The only thing to keep in mind is that the class on which you define :fme_pseudo_fields must respond (either as a method or field call) to the name of the derived field.

So you could actually accomplish the same thing as above with...

_pragma(classify_level=basic, usage={redefinable})
## used by FME SWAF interface to allow a customizer to add
## pseudo fields to the schema for Smallworld tables to be
## passed to FME.
transformer.define_shared_constant(:fme_pseudo_fields,
                                 {fme_pseudo_field_factory.new_derived_field(:bank_id,:extdb_string)},
                                 _false)
$

As long as you also have defined a method...

_method transformer.bank_id
      ## bank_id : integer
      ##
      ## returns self’s transformer_bank_id

      _return _self.transformer_bank_id
_endmethod
$

The FME fme_pseudo_fields concept is very powerful because it allows you to send data to FME that is not stored in a single attribute in Smallworld.

I have used it in the past for exporting data from Physical Network Inventory (PNI). If you are familiar with that datamodel you will know that there are many layers of relationships. Some of the records have geometries and some do not. A customer wanted to export all underground and aerial routes (these are the features with geometry) but wanted to label the features in the exported format with data from a related non-RWO record. Using the fme_pseudo_fields approach made it very easy to create a Magik method that created a suitable labelling string based on the related records but expose the results to FME as a feature attribute.

Smallworld turns 20

Smallworld turned 20 in December and I missed it! I have had the privilege of being associated with Smallworld for 12 of those 20 years... some as a Smallworld/GE employee and the rest as a partner in the Geodata/Navigant/Smallworld/GE/RedPlanet/(and now iFactor) career track.

You can read a retrospective from Peter Batty at this blog post.