Archive for the 'Programming' Category

A simple Python threading example

$ python –version
Python 2.7.3


import sys
import threading
import logging
import time

def setup_logging():
  logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)

def call_me( ):
  t = threading.current_thread()
  id_str = str( t.ident )

  logging.debug( id_str )

for i in range( 10 ):
  print id_str , ' : ' , i
  time.sleep( 1 )

def main():
  for i in (1,2,3,4,5,6,7,8):
  call_me()

if __name__ == '__main__':
  setup_logging()
  sys.exit( main() )

$ time python test_thread.py

real 1m20.864s
user 0m0.080s
sys 0m0.048s

update main() and invoke call_me() in its own thread.


def main():
  for i in (1,2,3,4,5,6,7,8):
    t = threading.Thread( target=call_me )
    t.start()

$ time python test_thread.py
real 0m10.222s
user 0m0.076s
sys 0m0.044s

Notice the significant difference in the ‘real’ time ~ 70 secs.

In the first example the sequential calls to the call_me() method are blocking. Each successive call has to wait for the previous call to complete.

In the updated 2nd example which uses threading to invoke call_me() the invocations are not blocking and ready to execute without waiting for the previous calls to complete.

Also, once they are started some threads are working while the others are idle thus maximizing CPU usage.

Using multiple threads to divide and execute a large task or tasks that makes blocking calls saves time.

Python modules, packages and code organization tips

Java has namespaces and a class per file construct to organize code.

C# (CSharp) has namespaces to organize code.

When I started to code in Python I was looking for a way to organize and structure code.

Python code can be organized into modules and packages.

Packages translate to a folder in the file system.


main.py
music/
....songs.py

A class called Song in songs.py can be imported into another module using the statement

from music.songs import Song

I was getting a python module not found or class not found error even though I had the right package folder structure.

On reading the python manual closely I realized that you need to create a __init__.py file in the package folder.

The folder now looks like,


main.py
music/
....__init__.py
....songs.py

After creating an empty __init__.py file in the package folder, I was able to import the Song class from music.songs module without any errors.

Some more tips on Python code organization.

Building a JSON Axis2 REST Client for a WCF Service.

Using JSON as the message format for a WCF REST service has some advantages
if you are building a backend service for a purely browser based application.

For instance, JSON is natively supported in all of the major web browsers.

I had to build a REST client using Axis2 that uses the JSON message format.

The blog post Enabling JSON in Apache Axis2 explains the conventions that are used to map XML to JSON and how to configure axis2.xml so that your client application can use the XML/JSON mapping convention of your choice.

I chose the mapped namespace convention which can be configured by adding the two lines to your axis2.xml

<messageFormatter contentType="application/json"
class="org.apache.axis2.json.JSONMessageFormatter"/>

<messageBuilder contentType="application/json"
class="org.apache.axis2.json.JSONOMBuilder"/>

If you choose the mapped namespace convention you may get an exception that says “JSON mapped namespaces are not supported in Axis2”. If you see the above Axis2 exception then you need to patch jettison.

Instructions on the exception and the patch are in the “Building a JSON web service with Java and Axis2” post by Marcus Schiesser.

The ServiceClient (org.apache.axis2.client.ServiceClient) object can be used to make REST API calls in Axis2.

The ServiceClient can be configured using the Options (org.apache.axis2.client.Options) object.

For a REST style service invocation you need the following options,


Options options = new Options();
options.setTo(new EndpointReference("http://your.service.endpoint.url"));
options.setProperty(Configuration.MESSAGE_TYPE, "application/json");
options.setProperty(Configuration.ENABLE_REST, Constants.VALUE_TRUE);

//reuse HTTP clients
options.setProperty(HTTPConstants.REUSE_HTTP_CLIENT, Constants.VALUE_TRUE);
options.setProperty(HTTPConstants.CACHED_HTTP_CLIENT, httpClient);

The httpClient object is created via the MultiThreadedHttpConnectionManager. This is needed for managing the HTTP connections if your client is used in a multithreaded application. See the post “Improving Axis2 Client HTTP Transport Performance”

Finally, you have to engage the addressing module and set the ServiceClient options


client = new ServiceClient();
client.engageModule("addressing");
client.setOptions(options);

The ServiceClient sendReceive call takes an OMElement which I created using custom XML serialization on the message objects.

Make sure you cleanup the http connections via the httpconnectionManager used to create the httpClient. Again. see the post “Improving Axis2 Client HTTP Transport Performance” on how to do this correctly.

The sendReceive call can be made non-blocking if you are looking for further improvements in performance.

The response returned by the sendReceive call is an OMElement again and can be de-serialized using custom XML de-serialization on your message objects.

If you are looking for pure SOAP interoperability between WCF .NET SOAP client and an Axis2 service be sure to chek out my other post Axis2 and WCF .Net Interoperability (Interop)

Axis2 and WCF .Net Interoperability (Interop)

Recently I created an Axis2 SOAP/JSON service in Java that actually works with a .NET WCF SOAP client.

Axis2 has support for building JSON SOAP services.

However, I could not find an implementation that worked with WCF clients.

Here are some tweaks that I had to implement on the Axis 2 stack to get this scenario working since I had no control on the WCF client calls or the .NET WCF stack.

1. No way to specify a null target namespace for the Axis2 service.

The .NET WCF client by default assumes a service target namespace of http://tempuri.org for XML SOAP messages. .NET JSON formatters do not have the concept of the JSON namespaces and so the JSON requests generated on the .NET stack do not have any namespaces.

And to be fair, JSON is not suited for creating namespace qualified messages.

The invokeServiceClass() method in RPCUtil.java on the Axis2 source was modified to accept SOAP service requests with no target namespace.

2. Axis2 JSON message receivers expect a wrapper around the incoming request message

Strangely even setting the service type in service.xml to “doclitbare” still does not deserialize the JSON into the correct Java request object.

For the example class below,

public class GetQuoteIn
{
private String ticker;

public String getTicker()
{
return ticker;
}

public void setTicker(String _ticker)
{
ticker = _ticker;
}
}

public GetQuoteOut GetQuote(GetQuoteIn getQuoteIn)
{
...
}

This object gets serialized into { “ticker”: “msft”} by the .Net WCF JSON Serializers.

On the server side the Axis 2 JSONMessageFormatter (org.apache.axis2.json.JSONMessageFormatter.java) expects a JSON message with a wrapper

{“GetQuoteIn”: {“ticker”:”msft”}}

I found it ideal to add the wrapper in the HTTPTransportUtils.java class (org.apache.axis2.transport.http.HTTPTransportUtils.java) before the incoming Message Context was built.

The processHTTPPostRequest() method was modified to get the SOAP operation name, added the wrapper name using the convention operationName + “In” and write back the modified JSON message with the wrapper into a newly created Input Stream.

3. WCF JSON message receivers do not expect a wrapper in the response message.

The MappedXMLStreamWriter.java (org.codehaus.jettison.mapped.MappedXMLStreamWriter.java) had to be modified to strip out the “wrapper” element in the JSON output. I modified the writeEndDocument() method in MappedXMLStreamWriter.java class to remove the wrapper in the serialized JSON output.

4. Capitalize JSON property names in the response.

The JSONObject.java (org.codehaus.jettison.json.JSONObject.java) write() method as modified to capitalize property names in the JSON response sent to the WCF client.

The changes could have been made at the various layers of the Axis 2 service stack. But the Axis2 stack is built to rely on the Message Context which means that any message semantics that need to be altered has to happen before the Message Context is established. Customizing the stack using phases and handlers conflicts with this heavy reliance on the Message Context.

In my next post I will explain how to create an Axis2 REST client that communicates with a WCF service (not necessarily a SOAP service) using JSON.
Call a WCF service (non-SOAP) that consumes JSON formatted messages.

TarTool – Windows tar gzip tgz extraction tool

TarTool is a tiny windows command line tool to extract tar gzip (tar.gz or tgz extension) files.

You can download TarTool , unzip and run TarTool.exe as a command line executable.

You can also download TarTool from this alternate download location if the link above is down or does not work for you.

The source code  for TarTool is now hosted on codeplex.

I wanted to play around with the Microsoft Shared Source CLI SSCLI. The download is only available as a tgz extension. Imagine that 🙁

After searching the interwebs for tools that would extract tar gzip files , I was disappointed since there was no simple tool to extract tar gzip files on my windows machine.

There are a few tools out there like WinRAR etc., but I found them too bloated for my task.

So I wrote a little tool using SharpZipLib, the open source Zip, GZip, Tar and BZip2 library (great job guys, BTW).

SharpZipLib does most of the heavy lifting so the core of TarTool  is less than ten lines of C#. I can host the source code if there is enough interest.

The source is now hosted on codeplex.

Enjoy!

 Update (05/06/2009 17:00 CST) :

TarTool now has an addtional option to untar tar file formats.

TarTool -x sample.tar temp

will untar the sample.tar contents into the temp folder.

Update (04/02/2012 11:35 PM CDT)
bzip2 decompression is now supported in TarTool 2.0 Beta .
TarTool -xj sample.tar.bz2 temp
or
TarTool -j sample.bz2

UI Programming Models

I found this excellent post Toward a better UI programing model with pointers to various UI programming models.

Web designers are forced to think within the constraints of the web browser when it comes to user interaction design.

The post above discusses more general approaches to user interaction design and not just within the context of the web browser.

Since I am mostly involved in web UI interactions these days this post was very refreshing as it brought a new perspective to me.

While developing the UI for a web app recently I realized that the web design world is in the search-discovery cycle of  “patterns” that guide user interactions.  Patterns in user interactions has the dual benefit of helping both designers and users. Much like the “guidelines” on Windows ( Windows User Experience Interaction Guidelines ) or Mac (Apple Human Interface Guidelines ) development platforms that help Windows or Mac application developers respectively while standardizing the user interactions of the applications developed on those platforms. These guidelines ensure strict quality, improves user experience and makes the life of designers easier.

There are plenty of resources that discuss “web ui interaction patterns” .

Designing Interfaces is a good resource if you are designing web user interactions. Rob Adams one of the core developers of Adobe’s Flex  has an excellent set of introductory articles which not only apply to designing user interfaces using Adobe’s Flex but applies to user interface design in general. I especially liked the Structuring your Application part.

Everyone loves the web, even if there are a few frustrations that we still have to overcome 🙂

Happy Surfing !

Git Fast Forward

git fast forward merge is a special case of a git merge.

Check the Basic Branching and Merging section of the git community book for the definition.

Here I present a case where non-fast forward merges were rejected by git and I had to fast forward a local branch to merge remote changes.

>git branch -a

* development

mylocal-branch

origin/development

>git status

# On branch development
nothing to commit (working directory clean)
>git merge mylocal-branch (you want to merge from mylocal-branch to development branch )

> git push <repository | origin> development

Works if the remote branch ‘development’ has no simultaneous commits from a co-worker, say.

But fails with the following messages if there were simultaneous commits on the remote branch ‘development’

! [rejected]        development -> development (non-fast forward)
error: failed to push some refs to ‘<your repository name>’

Here is how to fix this scenario,

> git pull <repository | origin> +development:development

The ‘+’ option fast forwards the local ‘development’ branch to the remote ‘development’ branch

> git merge mylocal-branch

At this point you have changes from the remote and local ‘development’ branches merged

> git push <repository | origin> development

The changes were now pushed to the repository without being rejected. This is one scenario where you can use fast forward to merge changes and synchronize local and remote branches.

newline characters in PHP error log or debug output

There is a PHP Predefined constant PHP_EOL that allows you to print a newline character if you are running php CLI or if you are outputting text to an error log file.

Apparently, it is also cross platform compatible.

For a long time I was using print statements to throw debug output and had a hard time reading the blob of text output that was getting spit out.

Now my log output and debug output are much more readable 🙂