Insert coordinates from text file into Smallworld database

I recently received a query from a reader asking if I could show him how to insert coordinates from a text file into Smallworld database.  I have put together a gist that shows how to do this.  I started off by using a LWT approach but then remembered that the recommended approach is to use the record transaction APIs.  So the gist includes both styles of database writing.

Exporting Node Coordinates via FME

I received a query today from a reader that wants to export the node coordinates for each electric conductor sent to FME. The sw_gis!node table is not a user table in Smallworld and is therefore not typically exposed to the FME plugin. But if you use the GE FME plugin, you can make use of the pseudo_fields functionality to gather up the relevant node information and send it to FME as a conductor attribute.

The basic steps are:

1. define pseudo_fields on the conductor class (or whatever class you are interested in)
2. write Magik support code that populates the pseudo_fields with appropriate node-based coordinates.
3. create a FME Workspace that reads the new pseudo_fields and processes the appropriately.

This video describes these steps in more detail.

Converting Coordinate System From Smallworld to FME

When setting up an FME Workspace to interact with Smallworld, you always want to be sure that your SWORLDSWAF reader/writer coordinate system is the same as your Smallworld session's Application Coordinate System (see discussion here). The post "Smallworld/FME Configuration Tips and Tricks" has a link to a slideshow that describes how to make sure that your Smallworld Application Coordinate System and FME SWORLDSWAF reader/writer coordinate systems match . The relevant information starts on slide 31.

I recently discovered another way of figuring out the FME COORDINATE_SYSTEM_DEF that corresponds to a Smallworld coordinate_system.

The FME Workbench's CoordinateSystemDescriptioConvertor Transformer (see example here) can take a Proj.4 coordinate system definition string and convert it into a suitable FME definition.

And how do you get the Proj.4 definition string? Easy... just use the following method.

_pragma(classify_level=basic)
_method coordinate_system_mixin.proj4_string
 ## proj4_string : string
 ##
 ## returns a string that represents self in the Proj.4 format.
 ## This can be used by coordinate system converters to convert
 ## self into other representations.

 # according to the comments in
 # coordinate_system_mixin.new_proj():  "PROJ_TYPE is a symbol
 # specifying the projection type. The naming convention is
 # similar, but not identical, to that used in the USGS Proj4
 # library. See below for details."
 #
 # because the Magik names are not always identical to the Proj4
 # names, we provide a lookup table here.
 _local projection_name_lookup << hash_table.new_with(:accurate_tmerc,:tmerc,
            :accurate_utm,:utm)
 
 _local (proj,params) << _self.proj_type_and_params

 _local info << rope.new()
 info.add_last(write_string("+proj=",projection_name_lookup[proj].default(proj)))

 _for k,v _over params.fast_keys_and_elements()
 _loop
  info.add_last(write_string("+",k,"=",v))
 _endloop

 _local int_str << internal_text_output_stream.new()

 int_str.write_list_with_separator(info,space_char)

 _return int_str.string
 
_endmethod
$

The next step will be to write a Magik method that actually converts the Smallworld coordinate_system definition directly into the FME COORDINATE_SYSTEM_DEF string all within Magik. That should allow for some greater flexibility in automating various Smallworld/FME processes.

Enjoy!

When a foot is not a foot

When integrating FME input/output with Smallworld there is one unit of measure that you need to be particularly wary about... the "foot". Depending on whom you are talking to a foot can be different things.

Consider this...

	When people talk about	FME calls it	Smallworld calls it
	International Foot	IFOOT	:feet
	US Survey Foot	FOOT	:usfeet

This is important in two places when setting up your FME mapping files:

1. Finding the correct FME coordinate system for the non-Smallworld end of the mapping file
   • If a customer tells you that they are giving you data to import into Smallworld and its units is "feet" make sure that you clarify whether it is in International Feet or US Survey Feet.
   • Looking at the FME coordinate system browser, I notice that many of the State Plane Coordinate Systems use the US Survey Foot.
   • You will need to clarify the same question when a customer asks you to export data from Smallworld. Typically, though, they will ask you for the data in a particular coordinate system and FME will know what units comprise that coordinate system.
2. Finding the correct FME coordinate system to match your current Smallworld application coordinate system
• Probably the biggest confusion of feet/usfeet/internationalFeet comes when you are trying to configure an FME coordinate system to match your Smallworld application coordinate system. You need to be absolutely sure that you are using the same "feet" unit in the FME version of the Smallworld application coordinate system.
• Most application coordinate systems use "cm" or "mm" for units so you won't run into this confusion. Even in US installations where units are measured in feet and inches, the application coordinate system and database coordinate systems are often set to metric units. (Remember that what a user sees on the GIS can be set to show Imperial units even if the underlying coordinate systems use metric units).

A classic sign that you are dealing with the Foot/USFoot issue is if you try loading in data from an external source and it looks like it "almost but not quite" lines up with your existing data. In that case you should carefully review all the coordinate system projections in your data flow process to ensure that the correct "feet" units are used throughout.