Igor has a nice demo video of this feature here.
Showing posts with label debug. Show all posts
Showing posts with label debug. Show all posts
Tuesday, July 12, 2011
Emacs Magik Debugger key binding
My latest favorite tool in Magik is the Magik Debugger. The Magik Debugger has been around for quite some time, but it was always inconvenient to load it. Starting at 4.1, the Emacs that ships with SW has a key binding (F4-d) that automagikally launches the Magik Debugger for the method in which your cursor is active. It makes it very easy to launch and using the debugger can often be quicker to use than scattering debug write/show statements throughout your code.
Tuesday, March 15, 2011
A call for identifying your custom changes
Reviewing code changes is difficult at best, but when reviewing a "basic_change" in Magik, it would be very helpful if the changes were very clearly bracketed with private comments.
For example, you might have an #EMQ like this...
... which you might want to refactor to be more robust. It would be really, really helpful to future code reviewers if you very explicitly indicated which lines of code you changed. Like this...
You can see that I actually commented out the original line and then put a copy of the line back in the _try block. This makes it very easy for a reviewer (and text diff tools) to see what has changed. I would not recommend using this approach for your own product code. Source Code control systems can identify these changes for you. What I am more concerned about is cases where you have redefined a core Smallworld method to address some business requirement. Once compiled into the image, it is very difficult to find the original version of the method, so the more that you can document as a customizer, the better it will be for future code readers.
Igor over at MagikEmacs pointed out to me that Alt-c in his extended version of Emacs does exactly this "changed code block" highlighting for you. You can see an example in his Screen-cast #3 at 3:50 where he uses exactly this technique.
_method some_class.some_method()
## some_method() : some_return_value
##
## does something crazy
_global a
(a.b[c], a.b[d]) << (a.b[d], a.b[c])
_endmethod
$
... which you might want to refactor to be more robust. It would be really, really helpful to future code reviewers if you very explicitly indicated which lines of code you changed. Like this...
_method some_class.some_method()
## some_method() : some_return_value
##
## does something crazy
_global a
# BEGIN CHANGES
#(a.b[c], a.b[d]) << (a.b[d], a.b[c])
_try _with cond
(a.b[c], a.b[d]) << (a.b[d], a.b[c])
_when error
write("!!! EMQ !!!",cond.report_string)
_endtry
# END CHANGES
_endmethod
$
You can see that I actually commented out the original line and then put a copy of the line back in the _try block. This makes it very easy for a reviewer (and text diff tools) to see what has changed. I would not recommend using this approach for your own product code. Source Code control systems can identify these changes for you. What I am more concerned about is cases where you have redefined a core Smallworld method to address some business requirement. Once compiled into the image, it is very difficult to find the original version of the method, so the more that you can document as a customizer, the better it will be for future code readers.
Igor over at MagikEmacs pointed out to me that Alt-c in his extended version of Emacs does exactly this "changed code block" highlighting for you. You can see an example in his Screen-cast #3 at 3:50 where he uses exactly this technique.
Friday, March 11, 2011
How to find the Magik source file for an exemplar definition
Many of you are likely aware of the
For example, in my Web Maps Connector image, I want to figure out where map_view.int!do_render() is actually defined. I can do the following to get that information.
object.method(method_name).source_file pattern that you can use to determine the file in which the method definition is contained. But did you know that you could also use the same pattern to find the file location for the actual object exemplar?For example, in my Web Maps Connector image, I want to figure out where map_view.int!do_render() is actually defined. I can do the following to get that information.
MagikSF> map_view.method(:int!do_render|()|).source_file$"C:\ifactor\products\workspace\Virtual_Earth_Connector\trunk\source\patches_z_proposed\p1000001_1_if_vec_map_plugin.magik" But what if I want to determine how that patch of
map_view.int!do_render() has modified the original version of that method? Where can I find that original method definition? While it is not 100% guaranteed, Magik style conventions strongly suggest that object method definitions be kept in the same file as the object's exemplar definition. Great. So all we need to do is find the file in which the def_slotted_exemplar(:map_view) is called. Here is how you do that...MagikSF> map_view.method(:exemplar).source_file$"C:/Smallworld41/product/modules/sw_swaf/map_plugin/source/map_view.magik"The trick is to request the source file for the special method called
:exemplar. Happy debugging!Monday, January 17, 2011
More Emacs Videos
Igor at http://www.magikemacs.com just posted two new videos
MagikEmacs Screen-cast 5
- shows how to inspect Magik objects using Emacs' Deep Print buffer.
- shows how to inspect Magik objects using a combination of Emacs and the core Development Tools application.
- This is very cool. Provides an easy way to explore Magik objects in an easy-to-navigate Magik GUI.
- Launch the Magik Debugger tool from an Emacs Magik code buffer using F4-D. Very cool!
Thursday, May 13, 2010
FME and Smallworld Text Join fields
Smallworld ignores data from FME that is destined for text_join fields.
This becomes a problem when you want to copy data into one of these text_join fields. This is not an unusual requirement. Many Smallworld objects have at least one text_join field (e.g., Electric Office objects have a "Remarks" field; other datamodels might have a "Comments" field). Not being able to import attribute values via FME into those Smallworld objects is not an insignificant limitation.
I raised a case with GE about this recently and was told that this was a duplicate of a same case that I had raised a few years ago. I knew that there had been something familiar with this issue, but I couldn't remember that I had already encountered it a few years ago. Because there is no patch in sight any time soon for this issue, I present to you the fix I made. Like most of the FME/Magik tweaks I have had to make, the implementation is fairly easy.
(click on image to enlarge)
Debugging Smallworld-FME Error Messages
Sometimes, when I am importing data into Smallworld using FME, I get a Smallworld error message in the FME log. Unfortunately I get only the condition string and not the accompanying traceback so it is usually very difficult to debug. The traceback is also not written out to the Magik standard output.
This simple hack will cause any error conditions raised during an FME import to have a traceback dumped to the Magik standard output.
(click on image below to enlarge)
Monday, May 3, 2010
Update Triggers Fired Unnecessarily
Recently, I was working on some performance profiling on some QAQC scripts in Electric Office. It turns out that the big bottleneck was in the trigger and business rule code that was being called during the QAQC routine run. In my review of the business rules, I saw that a lot of the business rules were being called unnecessarily. That is, the update triggers were being called on objects where no material change had occurred to their attributes. If you call
Please be aware of this whenever you write code with
If you compile the following code into your image and then run any “slow” process, you will be notified of all the times a record gets updated with nothing new, causing the business rules to be fired unnecessarily…
record_transaction.new_update(proposed_values).run() and the proposed_values are all the same as the existing values for those attributes, the triggers and business rules will still be fired. Because some of the business rules called code that was expensive to run, this added unnecessary time to the whole process.Please be aware of this whenever you write code with
record_transaction.new_update() and make sure that you only update attributes where the value has actually changed. The record_transaction API will not optimize your proposed_values list for you.If you compile the following code into your image and then run any “slow” process, you will be notified of all the times a record gets updated with nothing new, causing the business rules to be fired unnecessarily…
_pragma(classify_level=basic)
_method record_transaction.new_update( record, values, _optional description )
##
## Creates an instance which specifies the update of a record.
##
## RECORD - the record to be updated
##
## VALUES - a property_list keyed on field name.
## The elements of this property_list provide proposed values
## for physical, geometry and simple valued join fields, and
## information for updating multi valued join fields.
##
## DESCRIPTION - String description on transactions to be used
## for undo messages.
##
# BEGIN CHANGE
# DO NOT PERMANENTLY PUT THIS INTO PRODUCTION CODE!!!! THIS IS
# FOR DEBUG PURPOSES ONLY.
_local anything_changed? << _false
_for k,v _over values.fast_keys_and_elements()
_loop
_if record.perform(k) <> v
_then
anything_changed? << _true
_endif
_endloop
_if _not anything_changed?
_then
write("Nothing changed for: ",record)
print(values)
# put a traceback in here to figure out which code is calling
# unnecessary new_update()
#!traceback!()
_endif
# END CHANGE
>> _clone.init( :update, _unset, record, values, description )
_endmethod
$
Wednesday, October 14, 2009
Getting more from tracebacks
Now that I've spammed my readers with a useless "test post via email" note, I will try to post something of value.
I have always been fond of saying "tracebacks are our friends". In my mind, tracebacks are not to be feared... the bigger the traceback the better because that means that I have more information available to me to debug the problem.
A common debugging technique is to put a call to the !traceback!() procedure in any method from which you want to determine how the call stack arrived at that method. Placing a !traceback!() call in your code won't raise a condition or stop the code flow. It will simply give you a dump of the call stack at that point.
To demonstrate how this works, I have put a !traceback!() call in the Web Map Connector's ve_tile.draw_on() method. The traceback looks as you would expect...
There is a global variable !snapshot_traceback?! that you set to _true to give you a bit more information.
Now the traceback looks like...
... as you can see, each traceback line is numbered. This is kind of handy to have. The real benefits of !snapshot_traceback?! are described in the public comments for this global variable.
What I wanted to show you is a modification I made to the !traceback!() procedure that gives you even more helpful information whilst !snapshot_traceback?! is _true.
Here is my hack of the !traceback!() procedure.
You can see that it calls the standard _thisthread.traceback() method, but after that it takes a list of all the methods/procedures that were reported in the traceback and shows their source files as well...
This can be very helpful when you are trying to debug why certain code works in one of your environments but not the other. It can be very helpful to the GE Help Desk team because sending this enhanced traceback will tell them how your call stack is different than the one they are testing in. It will show which methods were changed in patches and in custom code.
I have always been fond of saying "tracebacks are our friends". In my mind, tracebacks are not to be feared... the bigger the traceback the better because that means that I have more information available to me to debug the problem.
A common debugging technique is to put a call to the !traceback!() procedure in any method from which you want to determine how the call stack arrived at that method. Placing a !traceback!() call in your code won't raise a condition or stop the code flow. It will simply give you a dump of the call stack at that point.
To demonstrate how this works, I have put a !traceback!() call in the Web Map Connector's ve_tile.draw_on() method. The traceback looks as you would expect...
---- traceback: renderer (heavy_thread 3611775) ----
time=14/10/2009 17:00:21
sw!version=4.1.0 (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=False
!traceback!()
a ve_tile(detached) .draw_on(a sw:canvas)
sw:simple_vector:[1-11].fast_keys_and_elements()
sw:keyed_set(1).valid_indices(False)
sw:keyed_set(1).fast_elements()
sw:keyed_set(1).elements()
transient_rwo_collection(road_tile).fast_elements()
sw:simple_vector:[1-11].fast_keys_and_elements()
sw:hash_table(1).valid_indices(False)
sw:hash_table(1).fast_elements()
ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
a sw:uds_geometry_set(ve_dataset,0,0).elements()
a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
sw:rope:[1-4].elements()
a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
a sw:canvas.draw(a sw:composite_geometry_set)
map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
map_view(map(Electricity),region).int!do_render(unset)
renderer(map_view(map(Electricity),region), unset)
light_thread_launcher_proc_990928()
There is a global variable !snapshot_traceback?! that you set to _true to give you a bit more information.
MagikSF> !snapshot_traceback?! << _true
$
Now the traceback looks like...
---- traceback: renderer (heavy_thread 3563153) ----
time=14/10/2009 17:01:44
sw!version=4.1.0 (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=True
[1] !traceback!()
[2] a ve_tile(detached) .draw_on(a sw:canvas)
[3] sw:simple_vector:[1-11].fast_keys_and_elements()
[4] sw:keyed_set(1).valid_indices(False)
[5] sw:keyed_set(1).fast_elements()
[6] sw:keyed_set(1).elements()
[7] transient_rwo_collection(road_tile).fast_elements()
[8] sw:simple_vector:[1-11].fast_keys_and_elements()
[9] sw:hash_table(1).valid_indices(False)
[10] sw:hash_table(1).fast_elements()
[11] ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[12] a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[13] a sw:uds_geometry_set(ve_dataset,0,0).elements()
[14] a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
[15] sw:rope:[1-4].elements()
[16] a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
[17] a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
[18] a sw:canvas.draw(a sw:composite_geometry_set)
[19] map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
[20] map_view(map(Electricity),region).int!do_render(unset)
[21] renderer(map_view(map(Electricity),region), unset)
[22] light_thread_launcher_proc_990928()
... as you can see, each traceback line is numbered. This is kind of handy to have. The real benefits of !snapshot_traceback?! are described in the public comments for this global variable.
What I wanted to show you is a modification I made to the !traceback!() procedure that gives you even more helpful information whilst !snapshot_traceback?! is _true.
Here is my hack of the !traceback!() procedure.
#% text_encoding = iso8859_1
_package sw
$
_global !traceback! <<
_proc@traceback(_optional stream, max_depth, full?)
## Display traceback of calls in the current thread on
## STREAM. If max_depth is given, at most that number of
## stack frames will be displayed.
##
## If !traceback_show_args?! is true, then the arguments will
## be shown along with the method name.
##
## Tracebacks normally exclude stack frames after a
## condition.raise(). Setting FULL? to _true will force the
## full traceback to be shown. I.e including tracebacks of any
## condition handlers.
_thisthread.traceback(stream, !traceback_show_args?!,max_depth, full?)
_global !snapshot_traceback?!
_if !snapshot_traceback?! _is _true
_then
_global !ts!
_local sfile , meth
write("These numbered source files correspond to the methods in the numbered lines in the traceback above.")
_for i,v _over !ts!.fast_keys_and_elements()
_loop
meth << v.receiver.method(v.message)
_if meth _is _unset
_then
show(write_string("[",i,"]"))
_continue
_endif
sfile << meth.source_file
_dynamic !show_string_length! << sfile.size
show(write_string("[",i,"]"),sfile)
_endloop
_endif
_endproc
$
You can see that it calls the standard _thisthread.traceback() method, but after that it takes a list of all the methods/procedures that were reported in the traceback and shows their source files as well...
---- traceback: renderer (heavy_thread 3514507) ----
time=14/10/2009 17:05:41
sw!version=4.1.0 (swaf)
os_text_encoding=cp1252
!snapshot_traceback?!=True
[1] traceback()
[2] a ve_tile(detached) .draw_on(a sw:canvas)
[3] sw:simple_vector:[1-11].fast_keys_and_elements()
[4] sw:keyed_set(1).valid_indices(False)
[5] sw:keyed_set(1).fast_elements()
[6] sw:keyed_set(1).elements()
[7] transient_rwo_collection(road_tile).fast_elements()
[8] sw:simple_vector:[1-11].fast_keys_and_elements()
[9] sw:hash_table(1).valid_indices(False)
[10] sw:hash_table(1).fast_elements()
[11] ve_dataset(ve_dataset).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[12] a sw:uds_geometry_set(ve_dataset,0,0).scan_elements(a bounding_box, sw:rope:[1-1], sw:hash_table(0))
[13] a sw:uds_geometry_set(ve_dataset,0,0).elements()
[14] a sw:uds_geometry_set(ve_dataset,0,0).draw_on(a sw:canvas, unset)
[15] sw:rope:[1-4].elements()
[16] a sw:composite_geometry_set.draw_on(a sw:canvas, unset)
[17] a sw:canvas.paint(True, a sw:composite_geometry_set, unset)
[18] a sw:canvas.draw(a sw:composite_geometry_set)
[19] map_view(map(Electricity),region).super(map_view).original!int!do_render(unset)
[20] map_view(map(Electricity),region).int!do_render(unset)
[21] renderer(map_view(map(Electricity),region), unset)
[22] light_thread_launcher_proc_990928()
These numbered source files correspond to the methods in the numbered lines in the traceback above.
"[1]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"
"[2]" "C:\DOCUME~1\Alfred\LOCALS~1\Temp\TAlfred9412q0"
"[3]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/ro_indexed_collection_mixin.magik"
"[4]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[5]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/convert2_sf/hashing/hash_values_helper.magik"
"[6]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[7]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/ds_src/dd/dd_collection_mixin.magik"
"[8]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/ro_indexed_collection_mixin.magik"
"[9]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/new_sys_changes.magik"
"[10]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/convert2_sf/hashing/hash_values_helper.magik"
"[11]" "C:/ifactor/products/workspace/Virtual_Earth_Connector/trunk/modules/ve_som/source\ve_dataset.magik"
"[12]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/uds_geometry_set.magik"
"[13]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/uds_geometry_set.magik"
"[14]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/geometry_set_mixin.magik"
"[15]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/mixins/rope_mixin.magik"
"[16]" "C:/Smallworld41/product/modules/sw_core/geometry_vector/source/composite_geometry_set.magik"
"[17]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/mgr_src/user/drawing_surface_mixin.magik"
"[18]" "c:/swdev/build/410/410_PB_WINDOWS/unshipped/source/sys_core/mgr_src/user/drawing_surface_mixin.magik"
"[19]" "C:\ifactor\products\workspace\Virtual_Earth_Connector\trunk\source\patches_z_proposed\p1000001_1_if_vec_map_plugin.magik"
"[20]" "C:\ifactor\products\workspace\Virtual_Earth_Connector\trunk\source\patches_z_proposed\p1000001_1_if_vec_map_plugin.magik"
"[21]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"
"[22]" "/swdev/streams/leo/release/unshipped/source/sys_core/sys_src/general/procedure.magik"
This can be very helpful when you are trying to debug why certain code works in one of your environments but not the other. It can be very helpful to the GE Help Desk team because sending this enhanced traceback will tell them how your call stack is different than the one they are testing in. It will show which methods were changed in patches and in custom code.
Sunday, March 9, 2008
OLE_Controller Monitor
Have you ever written Magik functionality that uses OLE/COM and wondered if you are sending :release_object() to all the ole_controller objects? Wonder no longer. Have a look at the ole_controller_monitor class I created recently.
This class allows you to start/stop/reset monitoring on the ole_controller class and report to you how many un-released ole_controller objects still exist in your monitored environment.
This is a great help in cleaning up OLE-based reports (eg., Excel, Visio, Word) and preventing the eventual OLE/memory-related errors that force a user to restart their Magik session.
API and steps to use the class are included in the file header.
This class allows you to start/stop/reset monitoring on the ole_controller class and report to you how many un-released ole_controller objects still exist in your monitored environment.
This is a great help in cleaning up OLE-based reports (eg., Excel, Visio, Word) and preventing the eventual OLE/memory-related errors that force a user to restart their Magik session.
API and steps to use the class are included in the file header.
Wednesday, October 3, 2007
MUnit Testing Framework
I recently received a request from a reader wondering where the unit test application was on my blog. I don't have a post on that yet so I thought that now was as good a time as any to make one.
I'm not going to go into the details here. Suffice it to say, that the MUnit framework is based on the original JUnit framework but for Magik instead of Java.
I have found it helpful when developing code to create some kind of automated test scripts. There is much that can be debated about how much or how little test scripting to do.
Mark Field had a detailed presentation at the Smallworld 2007 Americas Users Conference this year titled "Implementing MUnit for Smallworld".I imagine you could contact him for the presentation or maybe we can get him to post more information on his blog. If you have the username/password, you can download the presentation from https://www.kinsleyassociates.com/ge/smallworld/presentations.asp
I'm not going to go into the details here. Suffice it to say, that the MUnit framework is based on the original JUnit framework but for Magik instead of Java.
I have found it helpful when developing code to create some kind of automated test scripts. There is much that can be debated about how much or how little test scripting to do.
Mark Field had a detailed presentation at the Smallworld 2007 Americas Users Conference this year titled "Implementing MUnit for Smallworld".
[NOTE: The above links are old. You can find the presentation slides and download zip file at http://sw-gis.wikidot.com/magik-munit]
GE also had/has a version of MUnit but from what I understand they are reluctant to release it to the community. I found that there was a version of MUnit created by Jan Vorel at http://www.janvorel.com. Unfortunately the current version of that site is not very user friendly like it used to be and you can no longer download Jan's code from there. But if you look at http://web.archive.org/web/20030406030601/www.janvorel.com/ you will find an archived version of the web site that still has a working link to his version of MUnit. It only works on CST 3.3 but I found that with a few modifications it works in CST 4.x as well.
The implementations of MUnit that I have seen do not allow you to test mouse/keyboard input. Mark's presentation describes how he enhanced MUnit to include those features.
One enhancement I did for the MUnit framework some time ago was to create a new test listener that exported the test results as XML and/or HTML. An example of those results is here.
You can click here for a Yahoo Groups search result on MUnit.
GE also had/has a version of MUnit but from what I understand they are reluctant to release it to the community. I found that there was a version of MUnit created by Jan Vorel at http://www.janvorel.com. Unfortunately the current version of that site is not very user friendly like it used to be and you can no longer download Jan's code from there. But if you look at http://web.archive.org/web/20030406030601/www.janvorel.com/ you will find an archived version of the web site that still has a working link to his version of MUnit. It only works on CST 3.3 but I found that with a few modifications it works in CST 4.x as well.
The implementations of MUnit that I have seen do not allow you to test mouse/keyboard input. Mark's presentation describes how he enhanced MUnit to include those features.
One enhancement I did for the MUnit framework some time ago was to create a new test listener that exported the test results as XML and/or HTML. An example of those results is here.
You can click here for a Yahoo Groups search result on MUnit.
Wednesday, September 12, 2007
Profiling a dynamic enumerator problem with XPRF
A recent post on the sw-gis Yahoo! Group made me think that this might be a good example to demonstrate how to use the XPRF Magik class to profile your Magik code and help you find potential bottle necks in it.
Because the post did not say so explicitly, for the purposes of this exercise I am assuming the "dynamic enumerator" referred to is the one that makes use of the methods :can_match?() and :match_string(). (Sometimes people use the term "dynamic enumerator" to refer to the core extensible enumerator functionality or the PNI "dynamic enumerator" functionality.)
The first thing I would do would be to do a profile analysis on the :match_string() method on that particular object. An analysis will tell you if you have an issue with database access or maybe some code inefficiencies in your match_string() method.
You could do that as follows...
temporarily rename your subclassed some_object_name.match_string() method to some_object_name.debug!match_string()
create a new method...
Then try to open the some_object_name's editor again in such a way that your match_string() code is called. Likely it will take longer now than before because all the match_string() code is being run through the XPRF profiler. When the list is populated in the editor you will be ready to investigate the performance profile.
First, make sure that the global x_profile was populated...
If it is populated, you can use various interfaces to review your profile. The least user-friendly is to call x_profile.basic_report(). This will give you potentially hundreds of lines of output at the Magik prompt... a bit overwhelming if this is the first time you have done this.
Another option is to use the :xprf_dialog class. By default it is not loaded into your image, so you need to load the :profiler_magik_gui module into your current image. Use the sw_module_dialog.open() to load the module or...
Once the module is loaded you can use the following Magik command to give you a more user-friendly GUI that will show you expandable lists of all the methods that were called within the context of your profiler.
Use that tool to see where the bulk of your time on the match_string() is being spent.
Possible problems you might have in your match_string() code...
- inefficient code in loops
- select predicates on non-indexed fields
Things that you could investigate to improve performance include:
- make your code more efficient
- index any columns that you are writing a predicate against.
- if the table contents are not changing very often you might consider caching the strings you get from the table in a reusable rope (eg., shared_variable) and only read the data from the table once per session in order to populate the cache
If all those things do not improve the performance to the degree that you would like, then you may want to consider using the CASE tool to configure that field as a "catalogue mapping". That will provide a different field editor but still give you the control over what goes into that field.
Because the post did not say so explicitly, for the purposes of this exercise I am assuming the "dynamic enumerator" referred to is the one that makes use of the methods :can_match?() and :match_string(). (Sometimes people use the term "dynamic enumerator" to refer to the core extensible enumerator functionality or the PNI "dynamic enumerator" functionality.)
The first thing I would do would be to do a profile analysis on the :match_string() method on that particular object. An analysis will tell you if you have an issue with database access or maybe some code inefficiencies in your match_string() method.
You could do that as follows...
temporarily rename your subclassed some_object_name.match_string() method to some_object_name.debug!match_string()
create a new method...
_method some_object_name.match_string(_gather args)
_global x_profile
_local ok?, result
(x_profile,ok?,result) << xprf.new_from_send(:debug!match_string|()|,_scatter args)
_return result
_endmethod
$
Then try to open the some_object_name's editor again in such a way that your match_string() code is called. Likely it will take longer now than before because all the match_string() code is being run through the XPRF profiler. When the list is populated in the editor you will be ready to investigate the performance profile.
First, make sure that the global x_profile was populated...
MagikSF> x_profile
$
a sw:xprf
If it is populated, you can use various interfaces to review your profile. The least user-friendly is to call x_profile.basic_report(). This will give you potentially hundreds of lines of output at the Magik prompt... a bit overwhelming if this is the first time you have done this.
Another option is to use the :xprf_dialog class. By default it is not loaded into your image, so you need to load the :profiler_magik_gui module into your current image. Use the sw_module_dialog.open() to load the module or...
MagikSF> sw_module_manager.load_module(:profiler_magik_gui)
$
Once the module is loaded you can use the following Magik command to give you a more user-friendly GUI that will show you expandable lists of all the methods that were called within the context of your profiler.
MagikSF> xprf_dialog.open(x_profile)
$
Use that tool to see where the bulk of your time on the match_string() is being spent.
Possible problems you might have in your match_string() code...
- inefficient code in loops
- select predicates on non-indexed fields
Things that you could investigate to improve performance include:
- make your code more efficient
- index any columns that you are writing a predicate against.
- if the table contents are not changing very often you might consider caching the strings you get from the table in a reusable rope (eg., shared_variable) and only read the data from the table once per session in order to populate the cache
If all those things do not improve the performance to the degree that you would like, then you may want to consider using the CASE tool to configure that field as a "catalogue mapping". That will provide a different field editor but still give you the control over what goes into that field.
Wednesday, February 14, 2007
controlling runaway procedures at the Magik prompt
Have you ever run a procedure at the Magik prompt and it kept running and the only way you could terminate it was to close the Magik session? There is another way!
I was working on a procedure today that parses the results of a Relational Integrity Check and summarizes the errors by rwo_type. I wasn't sure how long it would take to run, so I forked the procedure in a new thread.
For illustration here, I have created a procedure that runs an endless loop. If you ran this at the Magik prompt, you would need to terminate the session in order to end the loop. To avoid this hassle, I have forked the procedure in a new thread using fork_at(). You can review the public comments for this method in the class browser to learn about the valid arguments.
Now it is easy to query the thread to see if it is running or not...
You can pause the thread...
You can resume a paused thread...
Or you can finally terminate the thread...
I was working on a procedure today that parses the results of a Relational Integrity Check and summarizes the errors by rwo_type. I wasn't sure how long it would take to run, so I forked the procedure in a new thread.
For illustration here, I have created a procedure that runs an endless loop. If you ran this at the Magik prompt, you would need to terminate the session in order to end the loop. To avoid this hassle, I have forked the procedure in a new thread using fork_at(). You can review the public comments for this method in the class browser to learn about the valid arguments.
MagikSF> th << _proc()
_loop
_endloop
_endproc.fork_at(5)
$
Now it is easy to query the thread to see if it is running or not...
MagikSF> th.status_description
$
"runnable"
You can pause the thread...
MagikSF> th.suspend()
$
thread(unnamed suspended 5)
MagikSF> th.status_description
$
"suspended"
You can resume a paused thread...
MagikSF> th.resume()
$
thread(unnamed runnable 5)
MagikSF> th.status_description
$
"runnable"
Or you can finally terminate the thread...
MagikSF> th.kill()
$
thread(unnamed runnable 5)
MagikSF> th.status_description
$
"terminated"
Wednesday, February 7, 2007
Developers: patch it yourself
(Note: this discussion is also useful to non-developers. If you are responsible for managing developers to provide customizations for your Smallworld environment, you may want to ensure that they are providing core code changes to you in a way that simplifies future upgrades. Using the core patch mechanism is one way to manage custom changes to core code.)
In the "old days" many of us used to include basic_additions and basic_changes in our custom Magik code to indicate core classes and methods that we were enhancing or changing. Sometimes we needed to change or patch core methods because we needed to do it quickly to meet a customer business requirement. The problem with this approach is that it these basic_* folders were scattered all over the source tree and made it more difficult than necessary to upgrade the core code at a later time.
I would like to suggest a different methodology that uses the core patching mechanism. It is one that I have seen used at some customer sites and have adopted for my ongoing work for other customers.
Many methods in the Smallworld core product are classified as not redefinable or even restricted. According to protocol, customizers may not modify restricted methods or really any method that is not classified as redefinable. The reality is that often times to meet a customer business need we do need to modify such methods. This adds to future code upgrade efforts because we now have introduced code that we must manually inspect for changes at each upgrade by comparing it to the core-upgraded version of that code.
It can be argued that changing a non-redefinable method is actually fixing a defect in the code. If the code does not do what a customer expects it to do, that is a defect. And if it is a defect, then it should be reported to the GE Smallworld Helpdesk. The helpdesk may not be able to produce a patch for you in the time period your project needs it for. In that case, you need to patch the defect yourself and have some kind of mechanism in place to revisit that patch once the next TSB is released for that product.
The following steps outline a process that could be followed. We refer to an example of an actual Smallworld-FME defect.
identify the defect
e.g., FME zero-length sectors incorrectly identified as donut holes. The problem was found in fme_tics_client.geom_field_for()
report the defect to GE Smallworld Helpdesk
report at http://supportcentral.ge.com
if Helpdesk can supply patch
install it per their instructions. In this example they were unable to so we created our own patch
if Helpdesk cannot supply patch
determine what module (and module version) your magik class is part of
MagikSF> sw_module_manager.module(fme_tics_client.module_name)
$
sw_module(fme_engine,2)
determine what product your code is part of and where that product directory exists
MagikSF> sw_module_manager.module(fme_tics_client.module_name).product
$
sw_product(sw_fme_400)
MagikSF> sw_module_manager.module(fme_tics_client.module_name).product.directory
$
"E:\projects\customer\fme400"
Ensure that that product has a source/patches_z_proposed folder
NOTE: the "z" in z_proposed is not a spelling mistake. All the "patches_*" folders in a product's source folder are loaded alphabetically. We need the "z" in z_proposed in order to have our patches loaded last. Because the core SW products already have patches, it is possible that we may be "patching a patch". The only way for this to be effective is to load our patches last.
get new patch number
Unless you have been assigned a range of patch numbers by Smallworld, the patch number you use will possibly overlap one that was already used by Smallworld. You can get around this by assigning a version to it that has a string in it unique to your organization.
For example, at Red Planet Consulting (RPC), we started our patches at 1. Therefore, the patch file might have been labeled as p1_1.magik. This patch probably existed for Smallworld at one time, so to make our patch unique, we change the file to p1_1rpc.magik. A corresponding entry in the file itself declares the patch as unique, too...
sw!declare_patch(1,"1rpc","description")
$
You will want to create a patch list or log for your organization so that you do not get duplicate patch numbers with your unique organization identifier.
create patch file
using the newly-logged patch number, create a patch file in the product's source/patches_z_proposed folder.
e.g., create E:\projects\customer\fme400\source\patches_z_proposed\p1_1rpc.magik.
be sure to add a reference to this file in the patch_list.txt for that folder.
You can look at any existing patch file for a template of how to create a patch file. The important parts to include are:
The benefits of this approach include:
In the "old days" many of us used to include basic_additions and basic_changes in our custom Magik code to indicate core classes and methods that we were enhancing or changing. Sometimes we needed to change or patch core methods because we needed to do it quickly to meet a customer business requirement. The problem with this approach is that it these basic_* folders were scattered all over the source tree and made it more difficult than necessary to upgrade the core code at a later time.
I would like to suggest a different methodology that uses the core patching mechanism. It is one that I have seen used at some customer sites and have adopted for my ongoing work for other customers.
Many methods in the Smallworld core product are classified as not redefinable or even restricted. According to protocol, customizers may not modify restricted methods or really any method that is not classified as redefinable. The reality is that often times to meet a customer business need we do need to modify such methods. This adds to future code upgrade efforts because we now have introduced code that we must manually inspect for changes at each upgrade by comparing it to the core-upgraded version of that code.
It can be argued that changing a non-redefinable method is actually fixing a defect in the code. If the code does not do what a customer expects it to do, that is a defect. And if it is a defect, then it should be reported to the GE Smallworld Helpdesk. The helpdesk may not be able to produce a patch for you in the time period your project needs it for. In that case, you need to patch the defect yourself and have some kind of mechanism in place to revisit that patch once the next TSB is released for that product.
The following steps outline a process that could be followed. We refer to an example of an actual Smallworld-FME defect.
identify the defect
e.g., FME zero-length sectors incorrectly identified as donut holes. The problem was found in fme_tics_client.geom_field_for()
report the defect to GE Smallworld Helpdesk
report at http://supportcentral.ge.com
if Helpdesk can supply patch
install it per their instructions. In this example they were unable to so we created our own patch
if Helpdesk cannot supply patch
determine what module (and module version) your magik class is part of
MagikSF> sw_module_manager.module(fme_tics_client.module_name)
$
sw_module(fme_engine,2)
determine what product your code is part of and where that product directory exists
MagikSF> sw_module_manager.module(fme_tics_client.module_name).product
$
sw_product(sw_fme_400)
MagikSF> sw_module_manager.module(fme_tics_client.module_name).product.directory
$
"E:\projects\customer\fme400"
Ensure that that product has a source/patches_z_proposed folder
NOTE: the "z" in z_proposed is not a spelling mistake. All the "patches_*" folders in a product's source folder are loaded alphabetically. We need the "z" in z_proposed in order to have our patches loaded last. Because the core SW products already have patches, it is possible that we may be "patching a patch". The only way for this to be effective is to load our patches last.
get new patch number
Unless you have been assigned a range of patch numbers by Smallworld, the patch number you use will possibly overlap one that was already used by Smallworld. You can get around this by assigning a version to it that has a string in it unique to your organization.
For example, at Red Planet Consulting (RPC), we started our patches at 1. Therefore, the patch file might have been labeled as p1_1.magik. This patch probably existed for Smallworld at one time, so to make our patch unique, we change the file to p1_1rpc.magik. A corresponding entry in the file itself declares the patch as unique, too...
sw!declare_patch(1,"1rpc","description")
$
You will want to create a patch list or log for your organization so that you do not get duplicate patch numbers with your unique organization identifier.
create patch file
using the newly-logged patch number, create a patch file in the product's source/patches_z_proposed folder.
e.g., create E:\projects\customer\fme400\source\patches_z_proposed\p1_1rpc.magik.
be sure to add a reference to this file in the patch_list.txt for that folder.
You can look at any existing patch file for a template of how to create a patch file. The important parts to include are:
# at the top of the file, declare which module this patch fixes.
# This is important because the patch mechanism attempts to load ALL
# patch files regardless of the relevance of their code to the current
# image. Putting a sw!patch_software() call at the top of the file will
# prevent your patch from loading if it is not relevant to the current image.
sw!patch_software(:fme_engine,2)
$
# the very last line of the patch file should be the patch declaration.
sw!declare_patch(1,"1rpc","FME zero-length sectors incorrectly identified as donut holes")
$
The benefits of this approach include:
- Smallworld is notified of "defects" to the code. This feedback loop is important to improve the product
- Customers are given a more robust solution to introducing core fixes identified by the customizer even before the next TSB. An added benefit of a patch-based approach is that patches can be easily "backed out" of an environment if they are deemed to be causing problems in that environment.
- At TSB/upgrade time, there are fewer folders to review for changes. Each time a TSB or upgrade is released, someone will need to look at all the patches_z_proposed folders and files to either remove or modify the files depending on what core code has been update by that TSB or upgrade. It is easier to maintain core patches in one folder per product as opposed to changes in one folder (eg., basic_changes) per module.
Friday, January 26, 2007
Where is that error coming from? (an update)
Roger Neal posted a helpful comment to my original post about how to force a traceback on User Errors for debugging purposes.
If you have CST v4.1 you can use the Development Tools application option to "Show User Tracebacks". If find this Development Tools toolkit very helpful. (For day-to-day code profiling, I use the xprf_dialog class alot. This is probably another useful topic to post about later.)
If you are already at v4.0, you could upgrade to 4.1 to get these development tools or you might try loading the product\modules\sw_dev_tools product from the 4.1 CD into your 4.0 image.
If you are at version 4.0, you can do this from Magik...
The method :enable_user_error_tracebacks? was only introduced at version 4.0. If you have an earlier version you will still need to do something like the hack I described in my original post.
If you have CST v4.1 you can use the Development Tools application option to "Show User Tracebacks". If find this Development Tools toolkit very helpful. (For day-to-day code profiling, I use the xprf_dialog class alot. This is probably another useful topic to post about later.)
If you are already at v4.0, you could upgrade to 4.1 to get these development tools or you might try loading the product\modules\sw_dev_tools product from the 4.1 CD into your 4.0 image.
If you are at version 4.0, you can do this from Magik...
MagikSF> model.enable_user_error_tracebacks? << _true
$
The method :enable_user_error_tracebacks? was only introduced at version 4.0. If you have an earlier version you will still need to do something like the hack I described in my original post.
Friday, December 29, 2006
Where is that error coming from?
If a graphical user interface (GUI) has been properly coded, any user error condition will be reported to the user as a dialog similar to...
This is great for the end user because it does not look as cryptic as...
... but it is a bit of a nightmare to debug if you want to find out where in the code this condition was raised. The cryptic dialog would have been nice because then you could "Show traceback" and continue your investigation from there.
Well, you can also force a traceback with the "nicer" error message if you have access to a MagikSF prompt (typically through emacs). You need to put a !traceback!() call in basic_window.show_alert() to get a look at the callstack to find out where the problem was. Unfortunately, this method is not shipped with CST 4.0, so you will need to do some Magik to force a traceback.
(By the way, the topic of shipped/unshipped code and the Open Source "movement" in the Smallworld community will be covered soon in another article).
1. save the original basic_window.show_alert() method's procedure
2. temporarily redefine the method to show a traceback
3. Redo the GUI action that cause the user error dialog in the first place. Because of how we redefined the :show_alert() method above, we will not see the error dialog, but we will see the traceback at the command line. Use that traceback to investigate the problem further.
4. revert the basic_window.show_alert() method to its original procedure
As always, if you have any questions or comments about a post, please feel free to e-mail me or reply with a comment to this post.
Now, back to digging out from another blizzard :)
This is great for the end user because it does not look as cryptic as...
... but it is a bit of a nightmare to debug if you want to find out where in the code this condition was raised. The cryptic dialog would have been nice because then you could "Show traceback" and continue your investigation from there.
Well, you can also force a traceback with the "nicer" error message if you have access to a MagikSF prompt (typically through emacs). You need to put a !traceback!() call in basic_window.show_alert() to get a look at the callstack to find out where the problem was. Unfortunately, this method is not shipped with CST 4.0, so you will need to do some Magik to force a traceback.
(By the way, the topic of shipped/unshipped code and the Open Source "movement" in the Smallworld community will be covered soon in another article).
1. save the original basic_window.show_alert() method's procedure
MagikSF> orig_proc <<
basic_window.method(:show_alert|()|).value
$
2. temporarily redefine the method to show a traceback
MagikSF> _method basic_window.show_alert(_gather args)
!traceback!(!terminal!)
_endmethod
$
3. Redo the GUI action that cause the user error dialog in the first place. Because of how we redefined the :show_alert() method above, we will not see the error dialog, but we will see the traceback at the command line. Use that traceback to investigate the problem further.
4. revert the basic_window.show_alert() method to its original procedure
MagikSF> basic_window.method(:show_alert|()|).
value << orig_proc
$
As always, if you have any questions or comments about a post, please feel free to e-mail me or reply with a comment to this post.
Now, back to digging out from another blizzard :)
Subscribe to:
Posts (Atom)
Posts
