More goodies from MS

My interest in software scalability and performance often leads me to Microsoft's Parallel Framework Extensions team BLOG, which recently published this great presentation from their parallelism tour...

Is Fring a new Skype?

It will be a couple of months since I became the proud owner of an iPhone. I am really happy to be able to check my email on the move, read news and navigate around town using features of my new gadget. One of the things missing would be IM support...

JSR-305 ideas for java annotations

I just came across article on the InfoQ discussing JSR-305. The JSR proposes introduction of new annotations ,which are to benefit defect detection tools.

Is it not great to see good trends converge? As Spec# is this JSRs' equivalent in .NET

Paralell Extensions for .NET

In case if you have not noticed Parallel Extensions for .NET got released!

You can check them in action implementing e.g.

• Ray Tracing
• Mandelbrot Fractals
  

Goodies from MS Labs

There are a lot great ideas developing in Microsoft Labs. Inspired by several posts I read I went to check the projects up and found at least two of interest...

Spring Source - the viable alternative to J2EE?

J2EE projects tend to be very complex, spanning multiple layers, using various technologies from IOC, ORM to AJAX. Their clustering, performance, fail-over and deployment are extremely important aspects, which are hard to manage...

Spring Framework was very successful in helping with some of those difficult aspects. Now after getting venture capital backing, Interface21 presents us with Spring Source - an alternative to J2EE. Take a look at this article in which Rob Harrop, in usual fashion, very precisely describes what the Spring Source is all about and where it is going.

I wonder if great tooling and OSGI support will help in adoption of this project?

My gadgets use PicLens

I have added PicLens to my gadgets site 3 weeks ago. But my busy schedule did not help with putting this piece of information on the blog...

There were many publications on PicLens. What strikes me is, the neat UI implementation. Similar paradigm is used already in iPhone presenting photo content on the fly over wall of items.

NB: To make it work you need to install plugin for your browser and you are ready to go. Check it out as it looks pretty cool

Commodisation of mobile services

Now, six weeks before Xmass, we being bombarded with new exciting offers dealing with new mobile devices and development platforms...

First is Skype to announce its Skype phone. Well, I have been using Skype for some time now and indeed it is a robust free, video enabled, VOIP application. But I am not sure, what is the benefit of owning the prebuild Skype phone. As it is possible to have the Skype application installed on any phone which has fast enough internet connection and runs Windows CE OS. Anybody?

Then the Jesus phone - as this is how the iPhone has been named in blogsphere. It always makes people go WOW and ask to keep it touch it etc. I have to admit, like the user experience on offer there. The large touch screen is very well integrated with the applications, what is not the case for other platforms unfortunately. There is a problem though. The phone is expensive, locked to the provider networks and does not give much out to the developers; apparently for sake of stability and security...

And the latest is the Android, which has just published its SDKs and made it available for download. The whole idea is to open up the mobile OS platform to the developers community. There are no Android powered phones on the market at the moment, but prototype preview looks very promising. Android is being build upon Linux kernel, comes with several applications like web browser based which is a fork of Safari. Its architecture aims at componentization and mashups allowing merging 2D and 3D graphics. There is even a draw published to help with incubating new applications...

Thankfully to software developers, it seems there is more and more opportunities to build distributed software services, which can be seamlessly integrated on closer and closer to hand and more powerful devices. BTW: Should we find a new name for those new mobile devices as they are not only just phones but music and video players, cameras, navigation units etc...?

Webscape changes

There was a lot of buzz on what the Web 2.0 is or should be. The idea was to bring more semantics to support intuitive and more relevant queries when performing searches, building story lines etc. But this nirvana still seems long way off...

Never the less the landscape or web-scape is indeed changing. Service providers like Amazon keep reinventing the web by adding new offerings on top of their strong platform of web services. Now they promote HaaS (hardware as a service) with their new elastic computing cloud (EC2). EC2 plus simple storage service (S3) gives us a distributed virtualization platform, where users can define their nodes and pay for their utilization on pay as you go basis.

On software side Yahoo provides their pipes allowing us to mash feeds up and this way provide with new shapes of outputs. This leads to new ways of dealing with the web content trying to use web pages themselves as web services. There are many unanswered questions left out there, about IP, content ownership etc. But this web scratching approach seems to be the enabler technology to help with structuring the mainly unstructured web content. And in this way maybe to narrow the gap between computer and human generated content.

The end-user client tools undergo changes as well. As result of web migration of our calendars, tags, address books, maps, photo albums etc., new data access metaphors are required. Desktop applications look more web like and web apps want to share your desktop (more often now being off-line enabled). Libraries like: GWT, Sriptaculous, YUI, FLEX have plenty to offer to help matching up the desktop experience by providing gadgets and add-ons.

Recent contributors to the arena are Mozilla's Prism and Microsoft's Popfly sitting on top of Silverlight, which I have decided to test and written a Popfly components:

1. block allowing to geo-position your URL source (but this needs more work due to the URL lookup service not returning good results)
   
2. mashup to show MSN book list as a Popfly carousel
   

Do not know about you, but I tend to use more and more computers over time. This makes me migrate my data to the web, so I can easily get hold of it when needed. Thankfully the aforementioned technologies let me do it with more and more ease and less frustration. Many smart people were saying that we can see desktop applications making the full circle from being run from smart-server on dumb clients, to full clients and now coming back to the server. The thing is the server is not going to be only smart but distributed and componentized. Don't you think?

PFX, PLINQ and TPL

You must take a look at the Programming in the Age of Concurrency by Anders Hejlsberg and Joe Duffy: Concurrent Programming with PFX posted on Channel 9...

It is great to see this natural, one would think, progression of lambdas into task managing, which no doubt will be simplifying concurrent implementations. Think mutexes, semaphores, critical sections, monitors, locks, threads etc. being hidden behind API simply accepting lambdas expressing your execution requirements. The same with workload control and optimization. This looks fantastic.

Here are some more links to Joe's articles, simply copied from his blog:

1. Parallel LINQ: Running Queries on Multi-Core Processors. An overview of an implementation of LINQ-to-Objects and -XML which automagically uses data parallelism internally to execute declarative language queries. It supports the full set of LINQ operators, and several ways of consuming output in parallel.
2. Parallel Performance: Optimize Managed Code for Multi-Core Machines. Describes the Task Parallel Library (TPL), a new "thread pool on steroids" with cancellation, waiting, and pool isolation support, among many other things. Uses dynamic work stealing techniques (see here and here) for superior scalability.

By being declarative and not over specifying, isn't functional approach way for the future?

PS. Have you tried F#, which just went into production?

Testing Skype's Extensibility

It has been quite some time since I have started using Skype - the free VoIP service provider. Being away from the homeland, we have been spending hours on the phone talking to family and friends. As expected the costs of calls were huge. So as soon as Skype has become available it is pretty much all the time on the menu.

The thing with Skype is, there is no way to indicate that you are in the call (simple feature found in MSN messenger for example). This way someone dialling in, has no idea you are busy, and makes you react to the incoming call or to bear the ring tone. This is a pain especially when you are away from the computer and to help myself with this,I have committed this simple Skype plugin

This little project has helped me to relax from my J2EE studies, I have been conducting lately. It was a really nice counter-point, I have to admit. It took a bit of C++, rough COM (no ATL) and C# together to get it going. I exposed myself to dynamic class loading, created custom controls and satellite assemblies to host Polish resources. Then I have thrown some code at NUnit and Rhino Mocks for testing. And everything using just publicly available development tools including Microsoft's Visual Studio Express

I have to admit that after Eclipse or even VS professional, it was really difficult to go back to such primitive development environment. Insufficient class browsing, limited code re-factoring, no free integrated tooling, limited debugging and project options - did not encourage. It felt like going back in time. I have to admit that at one point, I thought of using Resharper and VS 2008 BETA. But the idea got dropped.

It continues to surprise me how many free and well integrated tools there are available for Java, comparing to MS .NET - Is it just sad story or opportunity, what do you think?

Anyway, now after passing the test for J2EE architect, I have my plugin working, what makes me doubly happy..

Magazine Digital Era

It has been a long time since I have started using Google Reader. It is really great and with Gears enabled, I am able to download all posts and read them off-line. Now when publishers open up, it allows not only to keep track on the news but also to gain access to my magazines...

Till now I have subscribed to hard copies of:

1. MSDN Magazine
2. Dr Dobbs
3. Architecture Journal
4. C++ Journal[only digital].

But since they all have gone digital, I have not renewed my subscription and started reading the articles over internet off my laptop's screen. I know, it is not the same as hard copy. It can be printed out, though ;) . Well, I am decided. Trying to do my bit to save the planet.

In case you find the RSS form a bit too rough - check this cool article by Digital Inspiration showing the way we are going.

I wonder if what are your favourite tech magazines, guys?

Connecting SOA with ESB

There is a lot of discussions and various blog posts on the subject of service oriented architecture - SOA. Question is: How do I employ SOA effectively, having ESB at hand?

I have found an interesting article by Bobby Woolf of IBM discussing exactly this matter. It seems Microsoft is concurring as Nick Malik is highlighting in his post.

It is great to see convergence like this. Following the thought - there are many occasions when business modellers have deep influence on the service bus. To make their life easier, would it not be great to provide with extensions to the existing BPM management tools and administer your ESB? Maybe even standardise then?

Business Intelligence Search

After, I had developed CSE Builder, now is time to demonstrate the Google Custom Search Engine which got configured using it. Since I have dedicated plenty of my professional life to dealing with Business Intelligence (BI), I though to use my expertise and develop Custom BI Search Engine.

What is the aim?
I wanted to provide with a capability to perform faster and more relevant searches within BI space. Typical to other searches, when you query the web for desired content you get plenty of irrelevant hits (imagine typing bi into search box...). This is when CSE comes handy as it provides with extension point to the Google search allowing the author (me) to influence population of the search results. And I used my tags collected on Delicious for this reason.

How does it look?
Here is the screenshot:



NB: In case you were unsure - my favourite browser of choice is FireFox ;)

How did I do it?

1. Subscribed to Google COOP services and created new search engine
2. Migrated and cleaned some of my tags to a dedicated Delicious account
3. Used CSEB to convert these tags into annotations and context (check this post for more details)
4. Uploaded the results into my custom search
5. Subscribed to Google AJAX Search API
6. Created custom page to perform restricted searches
7. Customized my delicious tag roll to allow facilitate search category application
   
8. And finally published the results on my gadgets site.

If you would like to add the Business Intelligence Search to you iGoogle page - simply click the button

- Let us know if you found it useful.

Linqs

I have been reading through many articles published by the Microsoft guys about LINQ. Since it rings the bell, as I dealt with CAL for SQL generation, I thought of aggregating some of the links to this comprehensive content. As they will quickly sink down within my shared items list as published by the Google Reader gadget.

By Scott Guthrie
...is showing what is it and how to use it...

• New "Orcas" Language Feature: Query Syntax
• Part 1: Introduction to LINQ to SQL
• Part 2: Defining our Data Model Classes
• Part 3: Querying our Database
• Part 4: Updating our Database
• Part 5 - Binding UI using the ASP:LinqDataSource Control
  

By Ric Mariani
...discusses its performance...

• DLinq (Linq to SQL) Performance (Part 1)
• DLinq (Linq to SQL) Performance (Part 2)
• DLinq (Linq to SQL) Performance (Part 3)
• DLinq (Linq to SQL) Performance (Part 4)
• DLinq (Linq to SQL) Performance (Part 5)

Custom Search Engine Builder

I have been using my Google account for quite a while and only recently came across COOP and Custom Search Engines. The whole CSE thing seems a great idea, as I can see myself wanting to be able to customize Google search engine. Funny thing is, the whole CSE definition process employs tagging techniques. This made me think, that it would be great to have a tool allowing me to use my own tagged data stored on Delicious to drive the definition of the Google Custom Search Engine. I thought of coding a prototype to do that. And here are the results of my work.

General Info
Delicious allows users accumulating bookmarks of their favourite internet sites and to categorize them by tagging with custom labels.

Google Custom Search Engine (CSE) provides utilities to allow users predefining their own search domains leading to more accurate and relevant searches. It is done by specifying list of sites, which should be included and excluded while performing the searches using standard Google search technology.

The idea
Now since CSE definition involves providing internet site details, the idea is to extract, transform and source information collected on Delicious into the CSE.
As you can see, what is being proposed here, is a simple tool allowing integrating public data between the two service providers. Let’s call it CSEB for CSE Builder.

How does the CSEB work and how do I use it?
When using the app, here are the steps to follow:

1. Provide your Delicious user id
2. Configure which tags to include or exclude from CSE
3. Provide your CSE inclusion and exclusion labels
4. Generate annotations
5. Provide your CSE details
6. Generate the context
7. Upload your annotations and context into the CSE

In case you want to see the application in action, check the Winked demo detailing the steps and outcome of taken actions.

Here is the Flash movie:

or if you reading this using RSS reader try this link.

Design aspects
APIs
Delicious API allows execution of various kinds of REST full queries. This way it is possible to retrieve information about tags and bookmarks/sites which can be fed to the client using RSS or JSON formats.
CSE provides with end-user controlled way of uploading files containing: site information (called annotations) and configuration information (called context).

Application
Since I do not host nor own any internet server, I thought of embedding all the functionality within client side Java Script. This way the prototype can be also easily distributed without need for large downloads and installations etc. The downside is that such approach requires nearly all processing to happen within the end-users’ browser.

Platform
I am not a JavaScript guru. Although after Dr.Dobbs review, I should probably choose YUI but I have decided to leverage my Java skills and employed GWT to compile my Java code into JavaScript, which could be then executed within several supported and most common browsers.


Development Environment Setup
GWT integrates really well into Eclipse I must say. This speeds up the development process tremendously as you can debug your Java code, while driving the generated web UI in the hosted browser – perfect!

So, I have downloaded Europa (the largest open source project release to date). During the setup, of the development environment, I have defined two run configurations and made them persist into files for re-usability. Check tab common\save as\shared file edit box.

First one is hosted debug. In the run configuration dialog-box I have:

• chosen java application configuration type
• chosen com.google.gwt.dev.GWTShell for main class
  
• selected arguments tab and within program arguments edit box typed

  -port 9999 -logLevel DEBUG -noserver ${project_loc}\www\org.cse.csebuilder\csebuilder.html

  where: 9999 is my port of choice as since I am running many services on my poor box so the standard ports get allocated pretty quickly
  

Second one is for unit testing. Again in the run configuration dialog-box I have:

• chosen JUnit run configuration type
• selected run all tests... option for Junit3
• selected arguments tab and within VM arguments edit box typed:

  -Dgwt.args="-out www-test"

  where: www-test is my directory of choice for storing unit test

As it is a good practice to eradicate the warnings from your project, and since GWT does support JAVA 5, I have setup the project compiler settings to use JAVA 1.4 compliance level and switched to JUNIT 3.8. But before linking the external dependencies I have defined two Java build path environment variables for GWT and JUNIT accordingly.

GWT allows generating the Eclipse project without much of a hassle. The only thing is that it generates batch files, which need to be executed for compilation, debug and testing purposes. I do not know about you, but I like to use ANT or Maven to script such tasks. So to get the project going I have setup my ANT script accordingly.

Then, I have downloaded and configured Subclipse,which is an Eclipse Subversion client. This allowed me sharing the code using Google Project Hosting Services.

Same Origin Policy – the reason why I started dealing with JSON
Since all processing is to happen within a client web browser I needed to find out the way to fetch and use the remote data. The way of doing it is to use AJAX methods and issue asynchronous requests for content. Problem is that, the data fed into the client is going to be retrieved from a different site to the CSEB. So it is subject to restrictions as defined by the Same Origin Policy (SOP). Under this policy the only thing I could do, was to use JSON format. When employing JSON one dynamically changes content of the DOM model and injects script tags. This asks the browser to download remote script contents and this way obtain new data available to your application. Additional information about JSON in GWT can be found here.

I have written custom utility class to handle this. It is made responsible for injecting the script object and managing its lifetime. The class does so, by using ID attribute within page DOM, which gets populated with the request URL and therefore allows replacing corresponding items when when multiple requests are being issued.


Data Processing delegated to Yahoo Pipes
Although Delicious provides with API, for tags and bookmarks retrieval, but the JSON APIs are subject to restrictions. You simply cannot fetch more than 100 items at a time. Such restrictions are good for the server side and scalability but are limiting the amount of things, which could be done in the CSEB application. Another thing is that, to make sure all relevant bookmarks are retrieved; the client would need to issue as many requests for the bookmarks as there are selected tags. This way one would have to handle chain of asynchronous communications on the client. What happens if one of them fails? Additionally it is possible to encounter a bookmark, which is a part of a few different categories. Therefore there is a need to handle duplicates. It tends to become resource hungry == really ugly…

To sort this out, I thought of using custom build yahoo pipes, so that the processing is delegated to the Yahoo Pipes service. Such service consists of two linked pipes.

1. First one is made responsible of fetching JSON for delicious bookmarks owned by a user and associated with category / tag provided.
   
2. Second pipe uses up to N fetching pipes. It aggregates their output, removes duplicated sites and returns JSON feed to the CSEB.

The downside of this approach is that the more services are introduced; it is more likely to suffer from connectivity and service downtime issues.

Mind you, now to achieve similar results you could employ worker pool from recently released Google Gears. This would require Gears download and refactoring, though.

Saving mashed information into a local file.
Once all the data is fetched and transformed into XML, end-user is asked to create a file to be able (in the dedicated separate step) to upload it into the CSE itself. It could be achieved in two ways: firstly by using a clipboard and file editor (which is a two-step operation) or secondly user could try persisting content into a local file from the browser (ideal). But such operation is subject to SOP again. As it would open up the way for all type of malicious Cross Site scripting attacks if local persistance was allowed.

Trying to get my local persistance working, I have coded a widget displaying generic, dialog like, page to allow end-user persisting XML content using Save Page As browser command.

The whole idea is based on the fact, that the browser will persist text exposed by the currently visible DOM objects. So providing, that your host page does not include textual fields, this widget will be able to control all the displayable elements. The widget contains several HTML widgets, styled not to be displayed, which will blend their contents to form valid XML document when the page gets saved. The whole trick is about inserting XML comments just before closing part of the root element. This way textual content of the page will end up being part of XML comment and the whole page can be persisted a valid XML (that is if you do not mind some rubbish in comments;) Here is the example of the generated XML:

<?xml version="1.0" encoding="UTF-8" ?>
<YourRootElem>
<YourElem... />

<!--
contents of the dialog goes here
-->
</YourRootElem>

GWT History
To be able to achieve local file persistence I needed to try out GWT History subsystem. It allows maintaining the state of the application when switching between populated main page and a Save as widget.
During unit testing I have found that although you can call History methods to programmatically invoke back and forward commands of the browser, but this happens without raising corresponding onHistoryChange event. I raised this as an issue on the GWT forum.

If you think you do not have to know HTML and JavaScript you might get disappointed
When coding CSEB, I have realized that it would be very difficult to develop an application without basic knowledge of HTML, CSS or JavaScript primitives. The GWT is stable but still very much evolving piece of software following the evolution of the browser technology.
This prototype required me to write a few custom widgets and implement a few lines of JSNI. This is only to get access to the required functionality making it possible to provide with desired UI metaphors.

Unfortunately due to limitations I could not test all of the JSNI code against all supported browsers but only two: FireFox and Internet Explorer.

When something worked not exaclty according to plan, I used FireFox and its great FireBug addon to debug the generated code and styles. So many thanks go to to its author Joe Hewitt.


Polishing the UI
Being of Polish origin, I wanted to have the CSEB available in my native language. To achieve this, I have localized the app, written all the necessary I18N code and developed a custom widget allowing the user to switch locale during runtime. I have written all relevant property files and things worked just fine.

GWT expects the whole locale specific data to be nicely separated in Java. Then it gets compiled into optionally obfuscated JavaScript. This works perfectly, but such approach does not allow language contributions without recompilation and IMHO presents itself as a little inconvenience. I think it would be beneficial to be able to ask for language contributions, which could be integrated with greater ease. So that locale specific data is loaded via e.g. XML from the same origin. Another reason would be that the supported languages add to the complexity of the compilation process as the GWT compiler generates files to cater for language-browser specific JavaScript combinations. So by removing the language from the combination would simplify the compilation process...


Testing, testing and once more testing
One of the things you need to love about GWT is the fact you can use JUNIT to test the functionality. It is even possible to emulate execution of the asynchronous code. I keep asking myself though, how accurate is to test the code which gets auto generated by the GWT compiler? But the similarity of the allowed subset of JAVA and JavaScript should make one quite comfortable…

The availability of unit testing most definitely contributes to the better quality of the solutions. As now there is no excuse to leave your code in un-testable and highly coupled state in JAVA. Now all the good coding practices can make their way into the JavaScript territory, providing GWT compiler will do the most out of them:

1. Introduction of proper layering and class design to facilitate
   
• Single concern principle
• Low Coupling
• High Cohesion
2. Removal of various code smells
3. Documentation
   

In case you are new to testing take a look at Google testing blog for guidance. It is really good read.


Architecture
The idea was to keep the subsystems small, low coupled and testable.
But following the rule that

A picture says a million words.

I have created the demonstration discussing the architecture on canvas of the UML diagrams. So check up:

• The use case diagram for project functional requirements
• The deployment diagram for deployment details
• Sequence diagrams describing realization of the proposed use cases
• Class diagrams for layering and internal infrastructure
• Collaboration Diagrams for the algorithm descriptions

Here is the Flash movie:

or if you reading this using RSS reader try this link.

The model has been created in MagicDraw and uploaded to here for your reference. You can easily view using free Magic Draw Reader.

Deployment
I have subscribed to Google Page Creator Services and thought of deploying the application to my custom site. The unexpected issue I have hit was related to the fact that the file upload mechanism has truncated the file names down to 41 characters max. This made code another utility to shorten the compiled names. Let me know if you would like me to publish this as well.

So finally, the CSEB application can be found here for test run.

License and Attributions.
The project including sources, documentation etc. got uploaded and is hosted by the Google Code. Check main project's page for license information.

Icons are provided by the FamFamFam.

After dealing with enterprise software it was really great fun to delve into the world of internet technologies. Let me know what you think of this all.