Welcome to Ananth's Blog!
Adobe Flex SDK Compiler - Under the hood series - Part 3 MXML Compiler Lexical Analysis and Parsing Continued
In this part, we continue to look at the details of how the MXML Compiler performs the parsing.

As mentioned in the previous article, MXML Compiler uses JavaCC to generate a recursive descent parser. The JavaCC is set to receive tokens from MxmlScanner.java, which acts as a token generator. JavaCC generates an interface TokenManager.java that can be implemented by a custom java class, which in our case is MxmlScanner. This class also handles the SAX events generated as a result of parsing the input MXML file. Hence MxmlScanner extends the DefaultHandler class of SAX parser, by defining various event handlers such as startElement( ) etc and also implementing the getNextToken( ) method as required by the TokenManager Interface (generated by JavaCC).

JavaCC also generates a Token class (Token.java). MxmlScanner, thus, needs to generate instances of Tokens as needed by JavaCC for further parsing actions. MXML compiler defines a node class for each type of tokens. These nodes are sub classed from Element class that inherit from Token.java. Thus, the nodes generated by MXML Compiler are actually instances of tokens.

Token class stores the kind of token, location of the token, image of the token, the next token seen in the input stream and a variable to store special token.

Since the basic tokens in an XML file correspond to XML elements that may be associated with their own namespaces and attribute values, the Element class is defined that encapsuates these additional information. Thus, the Element class defines the URI, attributes as a qualified name/value map, prefix mappings etc. Another important property that is a part of this class is the children property that stores the tokens (or DOM nodes) of the children of this current node. For example, a Script node may store the CDATA as its child.

For example, let us consider the statement:

<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">

In the above statement, the Uri property will be set to: library://ns.adobe.com/flex/spark, the qualified name will be: s:Application, local name will be Application

minWidth and minHeight will be stored as attributes and by themselves can be fully qualified.

For <s:Application ...> tag, the startElement handler creates a DocumentNode object, sets up the necessary information. The token information looks something like this. The kind property is set to 63 and the image becomes: <s:Application>

The Node class, Node.java is inherited from Element.java.

Any MXML DOM node, for example, DocumentNode, is sub classed from Node class and is specialized for the functionality required for that node type.

The DocumentNode instance is created when <s:Application> element is encountered.

The generated node (or token) object is stored as a list of objects (saxEvents) in MxmlScanner. This list holds the tokens and scanner errors.

The getNextToken() method returns a token to the parser (generated by JavaCC) 

In the next part I will discuss the parser generated by JavaCC 

 

MORE >>
Posted by Anantharaman Narayana Iyer at 7/11/2010 9:04 PM | View Comments (0) | Add Comment | Trackbacks (0)
Adobe Flex SDK Compiler - Under the hood series - Part 2 MXML Compiler Lexical Analysis and Parsing
In this part we will examine the way the MXML Compiler performs lexical analysis and parsing of MXML Source code.

MXML language is a tag based language built over XML and the developer can embed Actionscript code within the MXML file using <mx:Script> tags. Typically, it is easy to describe UI elements (such as Button, TextArea etc) in MXML and Actionscript provides the scripting capability so that one can build a functional Flex application that performs appropriate actions responding to user or system initiated events. The input to the MXML sub compiler is a MXML source code and the compiler generates Actionscript code as the output. The MXML sub compiler consists of 2 other sub compilers: Interface Compiler and Implementation Compiler, that we will describe later. Following are the major steps that are performed by MXML Sub compiler towards the analysis of the source code:

1. Low level XML Parsing
2. Generating tokens from the step 1 as above
3. High level language specific parsing
4. Semantic analysis

Let us consider a working example as below. This example displays a button labeled "Click Me" at the coordinate (100, 100) on the screen. On clicking the button, the message "Button Clicked" is displayed on the Alert control of Flex.

Example: Source code listing of Simple1.mxml

<?xml version="1.0" encoding="utf-8"?>

<s:Application xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600">
<fx:Script>
    
<![CDATA[
        
import mx.controls.Alert;
        
protected function button1_clickHandler(event:MouseEvent):void
        
{
            Alert.show(
"Button Clicked");
        }
    ]]>
</fx:Script>
<s:Button label="click me" x="100" y="100" click="button1_clickHandler(event);" />
</s:Application>

Since MXML is a XML based language, this MXML source code needs low level XML parsing in order to derive the MXML language tokens. As I mentioned in the part 1 of this series, the sub compiler interface has a method parse1 with the signature:

public CompilationUnit parse1(Source source, SymbolTable symbolTable)

When this method is invoked in the MXML compiler (MxmlCompiler.java), it invokes parse1 method of Interface Compiler (InterfaceCompiler.java) and this kicks off the lexical analysis and parsing actions for the MXML source. (See: parseMXML method of InterfaceCompiler.java). The parseMXML method creates an instance of MXML Scanner (MxmlScanner.java). This scanner instance performs the low level parsing of XML using the Xercius Java SAX parser library.

The following points may be noted:

1. The SAX parsing method is preferred compared to DOM parsing as it doesn't occupy memory and faster.
2. The complete MXML file is parsed using this SAX parser and a token stream that contains the nodes (sub classed from the token class) is constructed. In most compiler implementations, the parser calls the scanner to fetch a token and the lexical analyzer generates one token at a time based on the lexical rules and the input source code. However in MXML compiler, the approach is to fully parse the low level XML so that the complete source file is converted in to a token stream and kept in a buffer. The scanner simply returns the node (or token) from the buffer when the parser requires it.
3. The high level parser (Parser.java that is auto generated by JavaCC - I will explain this subsequently)  fetches the next token from the token stream already created by the scanner as in step 2 above and constructs a document object representing the syntax tree of MXML source code.

Let us look at the above steps in detail:

The parseMXML method of interface compiler pretty much performs the 3 steps listed above. Once these steps are completed where the syntax tree is built and checked for basic consistency, the interface compiler invokes semantic analysis actions - see SyntaxAnalyzer.java.

Let us look at the following code reproduced from parseMXML method:
in = new BufferedInputStream(source.getInputStream());
flex2.compiler.mxml.dom.MxmlScanner s =
new flex2.compiler.mxml.dom.MxmlScanner(in, mxmlConfiguration.enableRuntimeDesignLayers(), processComments );
Parser p =
new Parser(s);
MxmlVisitor v =
new SyntaxTreeBuilder();
p.setVisitor(v);
app = (DocumentNode) p.parseApplication();

The source code lines 1 and 2 perform the low level XML parsing, steps 3 through 6 perform the language parsing and building the syntax tree.

 

The Parser class as above is auto generated from a grammar (Grammar.jj) by JavaCC tool. JavaCC produces a top down recursive descent parser based on the grammar specified in Grammar.jj

In the recursive descent parser, there is a method for each non terminal and the parser repeatedly fetches tokens from the scanner using a function: jj_consume_token that gets the token from the scanner using the method: getNextToken( )

The starting point for the parser is the method parseApplication and this method recursively descends through other methods to generate parse actions.

 

In the next part of this series I will show how the example program (Simple1.mxml) gets parsed by the compiler.

MORE >>
Posted by Anantharaman Narayana Iyer at 6/20/2010 10:26 PM | View Comments (0) | Add Comment | Trackbacks (0)
Adobe Flex SDK Compiler - Under the hood series - Part 1 Introduction
I have been experimenting with Adobe Flex compilers of late and thought that it would be useful to share some of the insights of how the Flex compilers work. Hence I decided to publish my notes in this blog.

There are many sources of very useful and insightful information both from Adobe (see: http://opensource.adobe.com/wiki/display/flexsdk/Flex 3 Compiler Design )  as well as from other compiler engineers for your reference (For example, see Clement Wong's blog http://stopcoding.wordpress.com/ ) .

I use the term Flex compiler to refer to a several compilers. MXML and Actionscript are the languages of Flex. MXML is typically used to describe UI elements of the application and Actionscript provides the scripting capability. One can use CSS files for styling and creating visually compelling applications. Further, with Flex 4 Spark component architecture, there is a seamless workflow enabled between the designer and developer and the Flex component architecture supports Flash Catalyst (Fc). The Flash Catalyst takes in the visual design created by Creative Suite tools, which is an FXG file and the designer can add interactivity to the visual design using Fc. The Actionscript Byte Code files (ABC files) are a bytecode compiled form of MXML/Actionscript sources and the ABC compiler is used to extract type information from pre compiled bytecode. Thus, multiple compilers are invoked when an end to end Flex application is developed. These compilers are: mxml compiler, Actionscript compiler, FXG compiler, CSS Compiler, ABC compiler and i18N compiler.

These compilers are implemented in Java and hence it makes sense to define the phases of these compilers using a common interface. The SubCompiler interface in the Flex SDK compiler defines the compiler phases as an interface and each of the (sub)compilers mentioned above implement this interface. The key methods of this interface are: preprocess, parse1, parse2, analyze1, analyze2, analyze3, analyze4, generate and postprocess. These methods typically correspond to the traditional compiler phases, where the lexical analysis and parsing is implemented in parse1 and parse2, semantic analysis and optimizations are done in the analyze routines and generate is used typically for final code generation.

The Adobe web site http://opensource.adobe.com/wiki/display/flexsdk/Flex+3+Compiler+Design provides an excellent overview to get started. The architecture diagram of Flex compiler infrastructure can be found from http://opensource.adobe.com/wiki/download/attachments/12845394/compiler.png?version=1

In the next several parts we will focus on the design of MXML compiler. I will focus on the lexical analysis and parsing aspects of MXML compiler for MXML source code in the next part. In particular we will examine how the MXML Scanner works and how it integrates with the parsing mechanism.

MORE >>
Posted by Anantharaman Narayana Iyer at 6/20/2010 7:19 PM | View Comments (0) | Add Comment | Trackbacks (0)
Supporting synchronous programming model for Adobe Flex
Developer often need to synchronize different pieces of the program logic in their applications. For example, consider a problem where we search for the images using a Image Search web service API, retrieve each of these images and then displaying the images as a slide show. In this example, the image search web service returns a list of URL's that point to the image that we specified in our search string. Hence the pieces of program logic that we need to synchronize are:

1. Using a web service and retriving the URL's that correspond to the images that we are searching
2. Using the URL's that are returned in the step 1 to fetch the actual images that are pointed to by these URL's
3. Displaying each of these retrieved images as a part of the slide show

Obviously, the above 3 steps need to be executed in order as it doesn't make sense to display an image (Step 3 as above) before that image is retrieved using the URL (as in Step 2).

In an RIA programming language, such as Adobe Flex, the steps 1 and 2 are executed as asynchronous calls where the Flex client fetches the relevant data (URLs or Images) from the server using web services (or REST or AMF) that are invoked asynchronously. Here the synchronization between the 3 steps is accomplished by writing event handlers explicitly for each of the asynchronous calls that were invoked. The asynchronous execution permits greater degree of interactivity of the application as it doesn't block the UI and hence is very beneficial.

However, the process of handling synchronization through a chain of event handlers is more complex and in some situations less intuitive and error prone. Developers who are more accustomed to the traditional ways of synchronizing programs (using primitives like message passing or semaphores etc) would find doing the same using event handlers to be more complex.

Hence the question is, can we provide a mechanism by which the developer can write the program using a synchronous programming style but the underlying system executes this in an asynchronous way?

We have developed an innovative mechanism using compiler transformations that allows a programmer to write his logic as if it is executing in a synchronous environment, while the actual execution happens on an asynchronous run time, which is Flash/Flex. Here, the compiler makes it transparent the process of writing event handlers as this part is generated by the compiler automatically transparent to the programmer.

This work is published in the recent issue (May/June 2010) of IEEE Internet Computing and is titled: Compiler Transformations to Enable Synchronous Execution in an RIA Runtime by Iyer, Anantharaman P. Narayana  Chatterjee, Arijit  Kishnani, Jyoti 

For more details, please have a look at: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5481363

MORE >>
Posted by Anantharaman Narayana Iyer at 6/16/2010 11:21 PM | View Comments (0) | Add Comment | Trackbacks (0)
Developing a Rich Internet Google App Engine Application using Adobe Flex

Google App Engine (GAE) is a Cloud Computing platform where a developer can build a cloud application and host it on Google servers. GAE at this point of writing supports Python and Java programming models and offers a set of services.The Getting Started documentation of GAE provides several examples to build a GAE application. However these applications are HTML based and do not provide details of building a Adobe Flex based Rich Internet Application client. Hence my intent in this series of posts is to discuss the details of building a GAE application with Flex as the client application providing a number of examples. In this post let us examine the key concepts behind building a Flex based RIA for Google App Engine.

I will discuss the examples using Python as the backend programming language and will assume Windows XP or Vista client machine.

Let us consider a simple problem as below:

1. In the server side logic we will authenticate a user with Google Accounts
2. Depending on success or failure of authentication we will display a suitable message in the Flex application

To implement the solution to this problem, we will:

1. Develop the server code in Python that performs some function using the Services offered by GAE SDK
2. Develop the client code in MXML using Adobe Flex Builder 3.0
3. Test the application using the localhost server, bundled with GAE SDK
4. On successful local testing, deploy the application (both client side and server side) on GAE
5. Test the final deployed application

As a first step, set up the development environment as below:

1. Obtain a GAE application ID by signing up - see http://code.google.com/appengine/
2. Download the GAE SDK for Python
3. Make sure you already have Python 2.5. If not download from http://python.org/ Since GAE supports Python 2.5, it is better to download the same version though higher versions may be available from http://python.org site
4. Make sure you have the Flex Biulder 3.0 installed in your system. If you do not have one, you can download a trial version from Adobe web site. For students Flex Biulder is free.

Before we begin, it is useful to review how Flex connects to a server application.

The earlier generations of Web applications were server centric. The business logic (in this example, "Authentication") executed on server and in addition to this, the presentation content was also generated in the server. You may review the examples provided under Getting Started section of GAE where this point becomes very obvious.

We may note that the sample code in Google Getting Started documentation has a get method that emits the necessary HTTP headers and also the HTML content for the display. This is in line with typical HTML based applications that often have a page based metaphor than an application metaphor.

Since the Flex applications run on the client and are responsible for handling the presentation, they do not require HTML to be generated from the server. Instead, Flex applications just need the data from the server and they can render the data and manage them on the client. Flex applications interact with the server through well defined, standard SOA model. Broadly, three major protocols are supported:

1. SOAP/WSDL based web services
2. REST based services
3. Remote Object protocol that uses Adobe's AMF binary format

In my next post, I will show how to build the client and server applications for Flex/GAE.

MORE >>
Posted by Anantharaman Narayana Iyer at 4/26/2009 11:18 PM | View Comments (8) | Add Comment | Trackbacks (0)
Does Cloud Computing and SaaS complicate an ISV world?
I was one of the panel speakers in a Panel Discussion organized by Headstart as a part of a Cloud Computing conference at Bangalore during Jan 9th and 10th. The conference agenda is at headstart.in/Agenda/

The panel met on an interesting topic: "Software-as-a-Service delivered from Clouds". I spoke on how the cloud computing paradigm changes the ISV world. The cloud paradigm has caught the attention over the last 1 year with several major ISVs making their offering and some of the businesses that leverage this paradigm reporting major growth.

The value proposition of cloud to businesses is strong - the need for any capital investment is minimal or zero and the cloud infrastructure allows scaling and is elastic as the needs grow. However, to make a product as a cloud offering, one encounters the following challenges at a broad level:

1. Architecting the application for the Cloud: Delivery considerations
2. Developing rich internet access models that access the cloud applications: Cloud Client development considerations
3. Business Models: What challenges the traditional  ISV's need to address to make revenue out of cloud?
4. Development methodologies: How does a developer build cloud applications? What options exist? What are the considerations?

I am planning to write a series of posts to share my thoughts with the first post (this one) describing the intersection of Rich Internet Applications and Cloud computing.

Before we go deep in to RIA and cloud, let me describe the cloud computing through an architectural model. At a broad level, we may categorize the cloud application environment in to the following layers:

1. Infrastructure

This layer represents the utility computing or on-demand computing model that are typically offered by hardware system vendors, such as HP, IBM, EMC. The compute servers and storage servers are made available through data center hosting as a virtual environment. The user of this layer will be able to run his applications on the compute servers or store data in the storage servers receiving these services as a utility. Amazon is a good example of an ISV who operates in this layer with its offerings such as Amazon EZ2 elastic cloud.

2. Software

The Software layer typically hosts cloud scalable databases, application servers, frameworks and other standard software typically used in web application environment. For example, Oracle partners with Amazon to make available its database products as a hosted application.

3. Services

There are many providers who make utility services available on the cloud. These services typically web services based and are invoked through the client program. Often a client application may invoke one or more services from different providers and combine these to offer unique benefits. These are mash up applications. Examples of services are map based services, image and video services etc.

4. Application

Many hosted applications are emerging. These include office applications like word processing applications (for example, Adobe Buzzword), Customer Relationship Management applications (such as those from Salesforce.com) etc.

It is possible for someone to write an application (application layer) that mashes up services from different service providers (service layer), use the databases (from Software layer) and be hosted on the compute and storage servers (Infrastructure layer). In a generic sense, it is possible that the vendors who provide each of the services listed above can be different - that is, an application may be executed on a compute server hosted by Amazon, the data stored in a EMC server and application services may come from servers that may be owned by some other vendors.

For us to understand the role of RIA in Cloud computing environment, let us think through what happens if we are required to use our popular word processing application that we run in our desktop from the web?

As a user, we might face the following issues:

a) Since the data and application are served from the web that we typically invoke these through the browser, the user experience may be poorer. This may happen due to:
     - Poor network bandwidth
     - The choice of client technology. For example, if the presentation content  is a HTML page in the Web 1.0 style (Synchronous communication with the server with page refreshes), the user experience may not be all that great.
     - The application architecture and design

b) There may be reliability issues if the providers of the services/utility do not guarentee certain Quality Of Service. Hence, potentially one may expect situations where the server may be down or services may be unavailable online.

c) Trust and Security concerns that arise due to the fact that the hosting infrastructure is not owned by the user.

d) Typically, pay as you go mode of payment as against upfront purchase of a software product

The RIAs help to address the issue (a) as above. Since we are looking at an example where we are using our popular desktop software as a web hosted application, it is fair to have an expectation that the user experience over web remains the same, if not better, than the desktop experience. Here is where RIA's with their ability to deliver desktop experience on web play a key role. Adobe's premier RIA platforms: Flash, Flex and AIR provide a great user experience to the web applications. Adobe Flex, with its comprehensive repertoire of user interface controls, provide a web experience that comes close to the desktop experience. Since the RIA's leverage the computing power in the client, the load on the server can be reduced, thus enabling a faster response from the server.

Often it is necessary to access the cloud applications from non PC clients such as cell phones. Adobe Flash technologies support such application models.

Some cloud applications may be peer to peer based. In such situations there is a need for data synchronization between different clients that are peer devices. Often, one may also need to support different access protocols such as Publish/Subscribe besides the traditional Request/Response protocols. In such situations, the Adobe Livecycle Data Services (LCDS) or the open source BlazeDS technologies would be very handy.

Thus, the Adobe Flash/Flex/AIR technologies for the client side and the Livecycle technologies for the server side make Adobe products a very compelling product suite for the development of cloud applications.

I intend to write more on this topic and so will stop here for now





              

MORE >>
Posted by Anantharaman Narayana Iyer at 1/10/2009 9:17 PM | View Comments (1) | Add Comment | Trackbacks (0)
Synergies in SOA and Web 2.0 Trends

I delivered an invited key note speech at the Service Oriented Engineering and Optimization (SENOPT 08) workshop which was conducted as a part of High Performance Computing (HiPC) conference in Bangalore, India. This talk asserts that over the last 7 years, 2 key trends have emerged: SOA on the services front and Web 2.0 that greatly influences the client side. These two trends are not independent of each other and in fact support one another's adoption. The presentation brings out the synergies and enumerates several Adobe case studies that support this theme. I enjoyed doing this presentation to the research community and am immensely thankful to my colleague, Duane Nickull. He is the senior evangelist at Adobe and also the chair for  the OASIS Service Oriented Architecture Reference Model Technical Committee (SOA-RM TC).

Here are the abridged version of slides

MORE >>
Posted by Anantharaman Narayana Iyer at 1/4/2009 6:21 PM | View Comments (4) | Add Comment | Trackbacks (0)
Adobe Flex and AIR training material
Duane Nickull, Adobe Senior Evangelist has made available excellent material that provide insights in to Adobe RIA technologies, Flex and AIR and how to connect different backends with RIA clients. The training material has 8 hours worth of content ranging from Hello World to more advanced topics of using remoting and WSDL introspection. This is a free download, have a look at:

http://technoracle.blogspot.com/2008/12/building-service-clients-with-flex-and.html

MORE >>
Posted by Anantharaman Narayana Iyer at 12/6/2008 12:04 PM | View Comments (0) | Add Comment | Trackbacks (0)
Academic Conference - Web 2.0, AIR and Flex presentation by Ananth
Adobe Systems India conducted a workshop for academia on 12 Aug 2008 at Hyderabad. I gave an overview of Web 2.0 and the Adobe products Flex/AIR that help build Rich Internet Applications in the context of Web 2.0 -  presentation in PDF

If you want to add this as a widget in your page (iGoogle, etc) you can grab this

MORE >>
Posted by Anantharaman Narayana Iyer at 8/16/2008 8:51 PM | View Comments (10) | Add Comment | Trackbacks (0)
Flex Builder Linux Alpha2
In continuation of its commitment for Linux, Adobe released the Alpha 2 version of Flex Builder for Linux. It is available from Adobe labs site at: http://labs.adobe.com/technologies/flex/flexbuilder_linux/

This Alpha2 release incorporates bug fixes, installer enhancements to provide better messaging for 64 bit, jseclipse support, Fedora 8 support, Data visualization and automation trials, integration with Moviestar etc.

MORE >>
Posted by Anantharaman Narayana Iyer at 12/19/2007 11:50 AM | View Comments (0) | Add Comment | Trackbacks (0)