eXo Platform, reloaded...

Java Development News:

eXo Platform, reloaded...

By Benjamin Mestrallet, Tuan Nguyen, Gennady Azarenkov and Ove Ranheim

01 Aug 2004 | TheServerSide.com

Introduction

A lot of work has been done since our last article was published in December 2003. We released the eXo Platform 1.0 beta 5 in March, with many new features, including WSRP and support for JSF 1.0. Today's design is still based on the service oriented architecture that was described in our very first article. By simply adding new services it was very easy to manage a quickly growing code base and still keep it maintainable.

The objective of this paper is to describe the process that an open source software development team has to follow when the kernel design is stable and only the application built on top of it has to be developed or integrated.

Therefore, before introducing the new features of the platform and the hot stuff to come in the next months, we first go through the last phases of the platform development focusing on the main problems we encountered, and how we solved them. This first covers the refactor of the web interface and design according to the community feedback and suggestions. Then, we explain how we had to leverage the Java Server Faces framework to provide a set of reusable components to ease application integration. Finally, we introduce our new web test framework that we have used for unit tests, stress tests, and high level documentation.

Part I : Refactors and unit tests

The main remarks made after beta 5 were about the application ergonomy and web design. We had to admit that our page and layout customizers were not as good as the Microsoft Sharepoint portal's, and that our skins were not pretty enough.

We honestly admit that without the comments and help of the community we would never have been able to refactor our model. We introduced an innovative page tree mechanism, and an associated API that extends the JSR168 specifications by providing the embedded portlet information about the portal context it is deployed in. We hope this will be taken into account in the next portlet API revision.

1. Page tree

Indeed, the JSR168 specification describes the relations between the portlet container and the portlets. But, no interaction between the portlet and the portal has yet been standardized. Our model is based on a tree where each node or leaf is binded to a page that contains several portlets. Each page is an aggregation of portlets laid out in a grid containing nested rows and columns. Therefore a portal can be seen as a page tree hierarchy, allowing a finer way to create application contexts that are easier to sort and navigate through.

Take the eXo Platform web site (www.exoplatform.com) as an example. The root node is called 'Home'. It has the children Products, Community, Demo, etc. Each child in turn has several children, f.ex. Community containing the Wiki, Forum and Issue traker nodes. Hence, we build a node tree type model.

Technically we store that model as an XML structure where each node can contain children tag nodes. Each node element is an aggregation of several tags and attributes that are, for example, used to reference the XML Page fragment the node is binded with.

 <pageNodeNavigation> <uri>/</uri> <name>home</name> <label>Home</label> <description>/</description> <pageReference>exo:/home</pageReference> <node> <uri>/home/products</uri> <name>products</name> <label>Products</label> <description>Products Page</description> <pageReference>exo:/home/products</pageReference> <node> <uri>/home/products/architecture</uri> <name>architecture</name> <label>Architecture</label> <description>architecture Page</description> <pageReference>exo:/home/products/architecture</pageReference> </node> </node> </pageNodeNavigation>

To allow user navigation within this tree we provide several views of that tree hierarchy :

  • a bread-crumb which describes the current tree path
  • a horizontal menu that renders the first level page nodes, and thanks to a popup, their children
  • a vertical menu that show all the descendants of the selected first level node
  • a site map that describes the whole tree hierarchy, and allows the user to go to any page easily

It is worth noting that all these views are portlets that can read and modify the tree itself through a JSR168 extension. We call them navigation portlets. Hence, a portlet can lookup a portal context that provides several methods to read and modify the navigation tree. Thanks to this API it is quite easy to create a portlet that would, for example, render the five last visited pages. A click on any of those five links would redirect the user to the associated page.

 public Node getRootNode(); public Node getSelectedNode(); public void setSelectedNode(Node node) throws Exception; public boolean isSelectedNode(Node node); public Page getPage(Node node); public Page setPage(Node node, Page page); public Node createNodeInstance(String name); public void createNode(String parentUri, Node node, String pageName, String templateName) throws Exception; public void save() throws Exception;

The most important interface is the Node one.

 public interface Node { public String getUri(); public String getName(); public String getLabel(); public void setLabel(String name); public String getIcon(); public void setIcon(String name); public String getDescription() ; public void setDescription(String s) ; public Node getChild(int pos) ; public void addChild(Node node) ; public Node removeChild(int pos) ; public Node removeChild(String uri) ; public boolean hasChild(String uri) ; public Node findNode(String uri) ; public int getChildrenSize() ; public Node getParent() ; public void setParent(Node node) ; public String getPageReference() ; public void setPageReference(String pageRef) ; public int getLevel() ; public void setSelectedPath(boolean b) ; public boolean isSelectedPath() ; }

To illustrate the use of the API we take the example of the web site portlet that renders the whole tree hierarchy (site map).

The process is quite simple. First we lookup the root node (in other words the Home node), and recursively render all the links that references its children. When one of those links is clicked, it triggers a call to the portlet processAction() method that will use the portal context to update the selected node and render the new page. The code of the action is basically the following :

 Node rootNode_ = portalInfo_.getRootNode() ; Node node = rootNode_.findNode(uri) ; portalInfo_.setSelectedNode(node) ;

Many other actions, such as binding a page to a node, or creating a page using a template, are possible. We will extend the API to support all the features the community may require.

We believe that this API allows the creation of portlet applications that are much more powerful than currently possible :

  • summary portlets that can redirect to other nodes of the navigation tree
  • portlets that can render their content according to the current selected path. Indeed, it is possible to create a content portlet that would dynamically extract and render content stored in a Java Content Repository (JCR and JSR170) or in any other back-end content.
  • dynamically inject parts of tree to simulate mandatory pages for users or group of users.

2. Page layout

Most portals are actually based on static template mechanisms, and only allow 2 or 3 columns of portlets. Thanks to our use of the Java Server Faces framework we allow the creation of any layout that HTML code may provide. Thereby it is possible to create nested rows and columns.

A portal page is composed of two parts :

  • the portal template is a set of rows and columns that is viewable on all the pages. We call this a template portal page, which can be modified by using the 'Edit Portal Mode' link while logged in. Basically, it contains the banner and footer portlets, as well as the navigation portlets. You may also add any portlet you would like to see on all pages, such as a portlet containing ads or weather information.

    (Click here to see larger image)
  • the page content is also a set of nested rows and columns. It is binded to one or several nodes from the previously described navigation tree. The portlets located in that page content, as well as the layout, change each time the user clicks on another node of the navigation tree. The user may edit the content of the current page by using the 'Edit Page Mode' link.

Those two structures are also stored as XML files. The link between them is provided by the navigation tree model, where each node tag references a page tag located in the owner-pages.xml file. This XML file contains the set of pages that the owner has created.

The schemas of the portal template XML (owner-config.xml) and the page content XML (owner-pages.xml) are very similar, as both describe nested rows and columns. Actually, it would be more accurate to talk of a container's tag. Each container may contain several children containers as well as portlets. Each container has a type (for example row or column) that is used by the portal engine to render the content of this container.

 <page> <owner>exo</owner> <name>/home</name> <title>Home</title> <state>maximized</state> <viewPermission>any</viewPermission> <editPermission>owner</editPermission> <portlet width="*"> <portlet-style>exo-content</portlet-style> <title>Home</title> <windowId>exo:/content/DisplayContent/news</windowId> <portlet-preferences> <preference> <name>Introduction</name> <value>title=Introduction</value> <value>uri=war:/exo-site/home-intro.html</value> <value>encoding=UTF-8</value> <read-only>false</read-only> </preference> </portlet-preferences> </portlet> </page>

As page content is associated with a portal owner (which may be one user or a group), it is possible to reference one of Owner B's pages from Owner A's portal navigation tree. A security tag allows the page owner to make his page viewable, or not. Of course, when Owner A imports a page that is the property of Owner B, he cannot edit and modify it.

Each user page is saved in the PAGE_DATA table and is identified by "portalowner:/page/name" pattern.

The main configuration "secret" is in the block and floating idea. Basically You have 2 type of the blocks, one is portlet that cannot have the children block and the container that can have the portlet and containers as children. A page is a special container with some additional information tag. For the container you have 2 renderers that control the floating of its children , one is Horizontal renderer and one is vertical renderer. Consider the structure bellow

 Container ---- > horizontal renderer portlet container -----> vertical renderer portlet container ----> horizotal renderer portlet portlet portlet portlet

Will produce the layout

 inner container using vertial renderer / ------------------------------------------------------------- | | | | | | Portlet | | | | | | | |-------------------------| | | | | | | | | | | |--outer container | | | | | | Portlet | Portlet | Portlet | Portlet | using horizontal | | | | | | | | | | renderer | |-------------------------| | | | | | | | Portlet | | | | | | |-----------------------------------------------------------|

To manipulate owner-config and owner-pages.xml files, three edit modes are available : Edit Portal Mode, Edit Page Mode and Edit Navigation Mode. We already summarized the first two. The last one allows the user to view and modify the navigation tree. When creating a new node, the user can either create a new page based on templates (blank page, two columns or 3 columns templates), or he can reference an existing page. It is then possible to browse and search for all the available pages. The model is therefore modified, and the navigation portlets are automatically notified to show an updated view of the node tree.

Note that the portlet preferences, if their value has been modified from what is defined in the portlet.xml, are now stored in the owner-pages.xml which allows for great flexibility when importing and exporting the portal data - have a look at our new Backup Service if you want to know more about our import/export features. For example, the eXo Platform site uses Content portlets extensively. With this new configuration we can pre-set the portlet preference with the path to where to lookup the content to render, in the pages XML files. This way the portal administrator does not need to configure each content portlet, as was the case with the beta 5 version.

3. Web design and CSS management

The web design part, and its associated mechanism, have also dramatically changed since beta 4. The current model provides an extension of the portlet API by providing better CSS support. Of course, if you deploy a JSR168 portlet that uses the CSS classes defined in the specifications in eXo Platform, the portal will recognize those classes and your portlet will have the portal's look and feel.

Using CSS2 for most of the platform components provides a powerful way of customizing the page elements, with no modification of the generated markup, such as xHTML. Furthermore, mobile markups such as xHTML-MP support CSS2. Our portlet API extension is simply an integration of our global portal CSS management to the portlet's body content.

Remember that our page layout is a nested set of containers and portlets. Each container has a renderer type (which is indeed a JSR renderer) that generates the markup language with the correct CSS classes. In an XML file (skin-config.xml) we define the associations between portal components, renderers and CSS files that we call styles.

To manage the decorator and css file we created a skin service that read the skin-config.xml in each portlet war and merge the configuration in a central place. Each decorator is identified by the renderer type and the decorator name. To add a new skin, the admin only need to create a new skin war that contain the customized skin css files and the images. The skin service will merge the configuration base on the renderer type.

 <skin-config> <portal-decorators> <decorator> <renderer-type>PortalRenderer</renderer-type> <style name="default" url="/portal/skin/portal/default-portal.css"/> </decorator> </portal-decorators> <page-decorators> <decorator> <renderer-type>PageRowRenderer</renderer-type> <style name="default" url="/portal/skin/page/default-page.css"/> </decorator> <decorator> [...] </page-decorators> <container-decorators> <decorator> <renderer-type>ContainerColumnRenderer</renderer-type> <style name="default" url="/portal/skin/container/default-container.css"/> <style name="exo" url="/portal/skin/container/exo-container.css"/> [...] </decorator> <decorator> <renderer-type>ContainerRowRenderer</renderer-type> <style name="default" url="/portal/skin/container/default-container.css"/> <style name="exo" url="/portal/skin/container/exo-container.css"/> [...] </decorator> </container-decorators> [....] <portlet-decorators> <decorator> <renderer-type>PortletRenderer</renderer-type> <style name="default" url="/portal/skin/portlet/decorators/default-decorator.css"/> </decorator> <decorator> <renderer-type>InfoPortletRenderer</renderer-type> <style name="info-company" url="/portal/skin/portlet/decorators/info-company-decorator.css"/> </decorator> <decorator> <renderer-type>BoxPortletRenderer</renderer-type> <style name="box-2" url="/portal/skin/portlet/decorators/box-2-decorator.css"/> <style name="box-3" url="/portal/skin/portlet/decorators/box-3-decorator.css"/> <style name="box-4" url="/portal/skin/portlet/decorators/box-4-decorator.css"/> <style name="box-5" url="/portal/skin/portlet/decorators/box-5-decorator.css"/> </decorator> <decorator> <renderer-type>ElegantPortletRenderer</renderer-type> <style name="elegant-1" url="/portal/skin/portlet/decorators/elegant-1-decorator.css"/> </decorator> </portlet-decorators> <portlet-style-config> <portlet-name>default</portlet-name> <style name="default" url="/portal/skin/portlet/styles/default-portlet.css"/> <style name="exo" url="/portal/skin/portlet/styles/exo-portlet.css"/> </portlet-style-config> [..] <skin-config>

Note that we removed the UIMainColumn, UITab , UIRow and UIColumn components described in our previous articles. Indeed, the current model has only 4 main UIComponents: UIPortal, UIPage , UIContainer and UIPortlet. Each component Can have different renderers for web browsers or mobile device browsers. Each renderer can have many css decorator associate with it.

Each portlet is split into two parts :

  • the decorator is the component that surrounds the portlet body, containing the window state and the portlet mode icons. It can be associated with many decorator styles that are, as before, css files.
  • the portlet body is the component that contains your portlet. It is possible to define a css file to use with it, by adding a skin-config.xml under the WEB-INF directory of you portlet WAR. It is therefore easy to add CSS classes that are not defined by the specifications.

All those skin-config XML files are loaded into the skin service when the portal and portlets WAR are deployed. The portal engine then ties the owner-config.xml file and the owner-pages.xml file with those skin-config.xml files, as each component tag of the first two XML files may contain renderers and style tags. It is therefore easy to include the CSS files to import inside the markup language.

This model provides an easy way to customize your portal skin. Indeed, by adding a custom portlet WAR that would contain a skin-config.xml and all the referenced css files and images, it is possible to define all the skin elements to be used by your portal pages. Indeed, the renderers and styles described in your custom XML file may define new renderers, and associate new styles to renderers or portlet body components. For example, if you wish to use another CSS file for our forum portlet just reference the new style in your custom skin-config file. The portal customizer will then automatically render that style name in the style popup-list associated with the forum portlet body. Note that the customizer is nothing more than a WYSIWYG editor of the owner-config.xml and owner-pages.xml files.

Using this method, and our new Backup service that allows the import and export - as zip files - of all the data and stucture of the portal, it is now very easy to re-generate on another portal instance the site created on another computer.

Note that the JSR 168 specification defines a list of of around 50 style classes and the class has portlet name as prefix, for example :

 /* Font attributes for the <93>normal<94> fragment font. Used for the display of non-accentuated information. Normal Text */ .portlet-font { font-family : Verdana, Geneva, Arial, Helvetica, sans-serif; font-size: 10pt; font-weight : normal; color: black ; } /* Font attributes similar to the .portlet.font but the color is lighter. Dim Text */ .portlet-font-dim { } /************************************************************************************* * Message style definitions affect the rendering of a paragraph (alignment, borders,* * background color, etc) as well as text attributes. * *************************************************************************************/ /* Status of the current operation. Ex: Progress: 80% */ .portlet-msg-status { } /* Help messages, general additional information, etc. Ex: Info about */ .portlet-msg-info { text-decoration: none ; padding: 5px ; color: lightgray ; font-size: 10pt; font-family: "Arial", "Tahoma", "Helvetica", "sans-serif"; } [...]

In order to make the portlet portable, you should use only the class name that define by the sun portlet specification or use <htmltag style='....'>....</htmltag>. However we find this model is hard to manage. For example if you develop a forum portlet, you may want to use the style class name like post, thread, forum , forum-category.... But with sun spec, you can only use the name like portlet-section-header, portlet-icon-label, portlet-form-field-label, and it has no meaning to your application. The previous versions of eXo Platform stricly use style class name that define by sun spec and we end up with too many hard code <htmltag style='....'> so we extended it.

This new model has been implemented little by little by following the community feedback and user requests. It is now quite easy to customize your own site without a single line of code required. This process was also the driver for a new web design that was clearly improved, thanks to the model's ease of use.

4. JSF enhancements and UI component reuse.

Writing a portlet is not more complex than writing a simple servlet. We would even say it is easier. However, implementing a good portlet in the shortest possible time is a lot more complex. And deploying it on a mobile device can become a nightmare.

The use of a good framework clearly reduces the time to market. In our opinion, the main advantage of portlets is freeing developers from using a framework they do not master. Indeed, it is possible to deploy portlets developed with Java Server Faces, or with Struts, inside a single portal. Therefore we have implemented several bridges, to allow the use of frameworks in portlets that were developed for a servlet context.

Even though we provide Struts and Cocoon bridges, our team is convinced of the power of Java Server Faces, and JSF1.1 is now supported. Our portal is built on top of it, and we created JSF UI components and their associated renderers to be used either for the portal's internal requirements or inside portlets.

Our use of JSF is quite singular and we don't use JSP at all. (Note that our JSF bridge provides the opportunity to use JSPs to create the UI components). The idea is to define a root component that we reference in the portlet.xml file, thanks to an init parameter. We then recursively build the UI component tree starting from the root component constructor. Once the tree is built the behavior of the framework is similar to the use of JSF with JSPs. Actually, the only difference is in how we build the tree and add action listeners to UI components.

It is important to notice our extensive use of one major feature of the JSF UI component tree. A JSF UI component has a field that describes its rendering state. When that state is set to false the JSF engine does not call the render methods of that component. Therefore, the portlet developer just builds a global tree with all the UI components the portlet may require. Then, by setting the rendering flag to true or false when handling an action event the view of the portlet changes. This method is clearly inspired by models like Swing, where a tree of UI components is built and action handlers modify the next view to render.

Actually, we enhanced some parts of the framework to better suit the portlet paradigm :

  • Constructor injection of UIComponents and eXo Platform Services : after playing with the pico container innovation for about a year we decided to add this feature to the JSF framework. Hence, a portlet developer can - in a UI component constructor signature - reference its children, or the services needed to process the action events. Note that it is also possible to inject the portlet resource bundle to ease the internationalisation (i18n) of your portlet application. The portlet developer then has to add those UI components to the children list and set the rendering flag.

  •  public class UILogPortlet extends UIPortlet { public UILogPortlet(UILog uiLog, UILogError uiLogError, UIConfig uiConfig) { setId("UILogController") ; setClazz("UILogController"); setRendererType("LogControllerRenderer"); List children = getChildren(); uiLog.setRendered(true); children.add(uiLog); uiLogError.setRendered(false); children.add(uiLogError); uiConfig.setRendered(false); children.add(uiConfig); }
    
  • Generic decode method : portlet renderers generate markup language that contains portlet action URL links. When one of these links is clicked, the JSF engine calls all the decode methods of the tree's UI components. These methods have to analyse the incoming input and generate events to be processed by action handlers' classes. To ease this work we have provided a generic method that automatcially triggers the action event for you when it receives a well defined parameter. The portlet developer just has to define this action parameter name and bind it to the action handler class that has to process the dynamically generated event.

  •  public UIGroupExplorer(OrganizationService service, ResourceBundle res) throws Exception { setId("UIGroupExplorer") ; setRendererType("GroupExplorerRenderer") ; service_ = service ; childrenGroup_ = service_.findGroups(null) ; addActionListener(ChangeGroupActionListener.class, "changeGroup") ; addActionListener(AddGroupActionListener.class, "addGroup") ; adminRole_ = hasRole("admin") ; }
    
  • Action handlers interceptors : due to our experience with AOP and interception, we wanted to provide such convenient features to the JSF framework. We therefore enhanced the Action Listener mechanism by providing hooks to allow registration of interceptors that may be called before and after the execute method of the action handler. One basic use of this is for security checks before calling an action - an interceptor will check if the current user that calls the action has the rights to do so. Note that in this case the way to register the action listener is different from the previously shown code.

  •  public UIExportData(BackupService service, ResourceBundle res) { setId("UIExportData") ; setRendererType("ExportDataRenderer") ; setName("Export Data"); CheckRoleInterceptor checkAdminRole = new CheckRoleInterceptor("admin", res.getString("msg.require-admin-role")) ; checkAdminRole.setHasRole(true) ; ExceptionHandler unknownException = new ExceptionHandler(res.getString("msg.error")); addFacesListener(new ExportAllActionListener(). setActionToListen("exportAll"). addInterceptor(checkAdminRole). setExceptionHandler(unknownException)) ; addFacesListener(new ExportDataActionListener(). setActionToListen("exportData"). addInterceptor(checkAdminRole). setExceptionHandler(unknownException)) ; adminRole_ = checkAdminRole.hasRole() ; }
    
  • Exception handlers : when dealing with component trees it is interesting to be able to propagate the exception throw in one action handler associated with a dedicated UI component to its ancestors. Therefore, we allow to bind exception handlers with UI components. If a child excute method of an action handler throws an exception and the child does not define any exception handler, the parent is used. This provides a powerfull inheritance model that speeds up development time. The code above also describes the registration of an exception handler into the action listener class.

  • Static inner action handlers class : most of the time, action handlers' objects may be shared by all the instances of the UI components they are binded to. To better manage memory we have introduce the use of static inner action handler classes. When defining the action handler class to call when a dedicated action event is called, a portlet developer can directly reference the static inner class that should process the action event.

  •  static public class ChangeGroupActionListener extends ExoActionListener { public void execute(ExoActionEvent event) throws Exception { UIGroupExplorer uiExplorer = (UIGroupExplorer) event.getComponent() ; String groupName = event.getParameter("groupName") ; uiExplorer.changeGroup(groupName) ; } }
    
  • Message mechanism : each eXo UI component can implement the InformationProvider interface. The methods it contains allows the portlet developer to manipulate messages and define the view type that will be used to render those messages.

     public interface InformationProvider { final static public int FOOTER_MESSAGE_TYPE = 0 ; final static public int POPUP_MESSAGE_TYPE = 1 ; final static public int BODY_MESSAGE_TYPE = 2 ; public int getDisplayMessageType() ; public void setDisplayMessageType(int type) ; public void addMessage(Message message) ; public void clearMessages() ; public List getMessages() ; public boolean hasMessage() ; }
    

    In an action listener class, it is then possible to abtain that provider and register the messages. If the component itself is not implementing the interface then we will look throught all the ancestors to get the first provider to handle messages.

     InformationProvider iprovider = findInformationProvider(comp) ; iprovider.addMessage(new Message(res_.getString("msg.empty-login"))) ;
    

When we moved from our own MVC portlet framework à la Struts (described in our September 2003 article) to this enhanced JSF framework, we decreased the overall code by 60%, we reduced the portlet development time, we increased the maintainability and promoted JSF components reuse.

An example of the great benefit of such a framework is when the portlet has to be rendered in several markup languages, for example to support mobile devices. We are currently working with a French telecom company team to port the entire portal to mobile devices. The work is simplified by the use of Java Server Faces for the portal and most portlets. First we have to extract the markup and client type information from the incoming request. Then we have to select (and write) the correct xHTML-MP renderer for all the portal components and portlets. The UI component tree is the same, even if some sub nodes may not be rendered because of the size of the mobile device screen. During the renderering JSF phase the encodeXX() methods of the xHTML-MP renderers are called to generate a correct markup to send to the client mobile or PDA.

5. Web unit test

When a Java developer gets used to unit tests it wants to test everything. But testing user web interfaces is not that easy. It certainly has nothing in common with basic services unit tests. One would like to create test suites that could simulate user behavior and assert the returned content with the HTTP response. Actually, we have envisioned such a tool since we used the Sun Microsystems TCK test suite, when we confirmed our JSR168 compliance.

To develop such an in container web unit test framework, we used the HTTPUnit library and built a programmatic API on top of it. Hence, it is quite easy to test portlets' web interfaces, and creating a test suite that can simulate a user session can be done very quickly. For example, it is possible to assert that once the link A in portlet block B has been clicked, the returned markup language contains the text T. Many possibilities are available and we are steadily creating more methods to test ever finer conditions.

 goToPage(path) ; addWebUnit(new ClickLink("ClickEditorLink", "Try to open the editor"). setTextLink("Editor"). setBlockId("file-explorer"). addValidator(new ExpectFormNameValidator("contentForm"))); addWebUnit(new SubmitFormUnit("PreviewContent", "Preview the content"). setFormName("contentForm"). setField("op","preview"). setField("content","Test create new file"). setBlockId("file-explorer"). addValidator(new ExpectLinkWithTextValidator("Back"))) ; addWebUnit(new ClickLink("GoBackToEditor", "Go back to the editor"). setTextLink("Back"). setBlockId("file-explorer"). addValidator(new ExpectFormNameValidator("contentForm"))); addWebUnit(new SubmitFormUnit("SaveContent", "Save Content with file name empty"). setFormName("contentForm"). setField("op","saveContent"). setField("content","Test create new file"). setBlockId("file-explorer"). addValidator(new ExpectTextValidator("You need to enter a name for the file"))) ;

We then add these test suites in one class that thereby wraps all the actions a specific user would do. Actually, we provide two classes like that : NormalUser and AdminUser. Obviously, the AdminUser gets more privileges as we simulate a login with the correct login and password. It is thereby possible to test the correct use of rights within the portal and portlets, as well as using the admin user to first configure the states that a normal user would need - for example the creation of portlet categories and portlets in the portlet registry to allow a normal user to add those configured portlets in his pages.

During the last phase of the platform development we decided to create a test suite for all the portal features (mainly the WYSIWYG customizer) and for all portlets. The main advantage is to allow easy refactors of portlets and portlet frameworks. Indeed, a regressive change in the Java Server Faces bridge, or in one of the default UI components, would be directly discovered during the execution of the web test unit suite.

Furthermore, we can validate if the returned markup is xHTML valid, and a CSS parser to test for CSS2 compliance is also being integrated.

We use this web test framework as a stress test tool as it is possible to configure the execution of several threads. This way we simulate various client types' simultaneous interactions with the portal. The only difference with the previous use (ie : web unit test) is that this time the number of threads defined in the test framework's properties file is higher than one. The following test simulates 50 users requesting the portal at the same time.

 [java] task left: 0 [java] Suite Name: NewAccountSuite [java] Description: Create a new user account [java] Unit Name Counter Error Malformed Avg(ms) Avg(kb) Sum(kb) [java] NewSession 250 0 0 1178 21.66 5415.059 [java] GoToMyPortalPage 250 0 0 476 18.277 4569.404 [java] CreateNewAccount 250 0 0 1351 18.391 4597.864 [java] [java] Suite Name: LoginSuite [java] Description: Go to the home page and login, using web client name for user name and passw ord [java] Unit Name Counter Error Malformed Avg(ms) Avg(kb) Sum(kb) [java] NewSession 250 0 0 970 18.391 4597.864 [java] Login 250 0 0 1140 25.218 6304.556 [java] [java] Suite Name: RandomPageBrowse [java] Description: Go to the various pages in exo portal [java] Unit Name Counter Error Malformed Avg(ms) Avg(kb) Sum(kb) [java] GoToProductPage 250 0 0 645 23.101 5775.434 [java] GoToServicesPage 250 0 0 893 22.994 5748.718 [java] GoToCompanyPage 250 0 0 798 21.285 5321.4 [java] GoToPressPage 250 0 0 670 21.945 5486.292 [java] GoToCommunityPage 250 0 0 511 25.897 6474.292 [java] GoToMyPortalPage 250 0 0 376 21.71 5427.696

These stress tests highlight the portlets that take too much time to render their markup, and when run for several hours, the existence of memory leaks.

This double purpose is quite powerful as it bypasses the need to configure several tools like JMeter. Consequently, the portlet developer has more time to focus on the creation of a well detailed test suite that is highly reusable.

As a side effect, we have also decided to use this web test framework for producing up-to-date high level documentation for expert users. Indeed, when a portlet developer creates a test suite he basically has to simulate all the use cases. As for any type of unit tests, they always have to be in synchronisation with the portlet features. Therefore, if each use case phase is well described with comments, it is possible to produce HTML or PDF reports after the test has been executed. By producing XML reports we allow XSLT/XSL manipulation of test data to produce documentation reports. As stated, this documentation is not intended for end users, but for expert users and platform admins who may get up-to-date reports on the portlets they use.

The five points above were clearly the most critical issues we encountered during our last development phase. The experience clearly proves that an open source development process driven by the community's requirements and feedback tends to create a much more stable, tested and reusable code.

Part 2 : What's new

The features summarized in Part 1 of this article are refactors of previously existing code. This second part focuses on new user features that have been, or are currently being developed. Among all the new stuff, the most important is probably the Java Content Repository early access level 1 implementation, the wiki integration, and the SMS gateway service and portlet integration.

1. Java Content Repository level 1 (JSR 170)

From the start we clearly defined our objective as providing an integrated portal and content management platform. Now the Java Content Repository (JCR) specifications (first draft), defined by the JSR170, has finally been released. It defines two levels of compliance :

  • level 1 : read, write and search in a hierarchical content repository, as well as node type definitions to provide application specific content nodes with well defined meta-data
  • level 2 : enhanced features such as locking, versioning and security checks.

Based on the experience designing our own repository (described in our September 2003 article), we have been implementing the JSR170's level 1. It is important to understand that the JCR is an abstraction API to a high level back end content. As an analogy we could compare the JCR API to the JDBC, that is also an abstraction layer on top of several database backends. As for all services, our implementation is completely based on our service stack. It leverages our IoC architecture and eases service lookup and unit tests.

The specifications mainly describe 4 back end types :

  • a file system implementation : we provide such a back end
  • a WebDAV server access : we are currently implementing that abstraction layer by providing adapters that allow transformation of JCR request to WebDAV compatible request calls.
  • a database backend : a deeper integration with the xWiki Hibernate store backend (see the Wiki integration paragraph) is currently being discussed with the xWiki team.

As can be imagined, it is quite easy to plug a new back end type with most classes being reused. As soon as the level 1 implementation for those 4 back ends is completely implemented and stringently tested, we will start to implement level 2 of the specifications. Note that according to the specifications, a level 1 compliant repository would be necessary in 80% of the cases. Finally, as with the portlet API specifications, note that until we sign the TCK Sun Microsystem license and pass the TCK test suite, we cannot claim compliance.

At the same time as we develop the JCR service, we provide new portlets or enhance existing ones, to take the content repository into account.

The JCR explorer portlet has the same functionnalities as the file exporer portlet. It aims to provide an easy navigation through the JCR tree. It also allows read and write access on the content. The mime-types supported by the file explorer portlet are also supported. The features of the explorer will grow as we add JCR level 2 support.

The content display portlet looks up HTML texts from several locations, such as file system, resources located in a WAR, or in the JCR. With the common use of a WYSIWYG HTML editor we provide an online editing system. The eXo Platform site provides a nice example of this content portlet use. The portlet first loads the HTML document from the content WAR file, as described in its portlet preference default value. If a portal user modifies the content of that file by editing it online, the new text will be stored in the JCR and the associated JCR URI will be stored in the portlet preference. The next time the text will be required it will be loaded from the JCR and not from the WAR. As soon as the specifications are finalised and our implementation of level 2 is stable, we will integrate these publishing functionalities, with locking and versionning features of the JCR, as well as with our workflow engine.

2. Wiki portlet and services

Wiki applications are a new way to share knowledge between members of a project team, or within a community. By creating a graph of documents that are easy to edit and modify, the use of a wiki reduces documentation costs.

We soon saw the value that wikis would add as a portlet, and we were pushed by a strong demand from our community. So we started a test phase of most of the java wikis, both open and closed source ones. The 3 most popular wikis were JSPWiki, SnipSnap and Confluence. The last two probably provide the most complete features, and Confluence's skin was quite close to JIRA that we use as our issue tracker tool.

We then discovered xWiki, and realized that their approach is much more innovative than the others, and that the strategy of the xWiki team really was quite similar to our own. Moreover, xWiki is Open Source.

First of all, xWiki was implementing most of SnipSnap or Confluence's features, such as :

  • a powerful wiki rendering module : as Snip-Snaps and Confluence, xWiki is based on the wiki rendering module radeox.
  • versioning : old document versions are stored, and it is possible to visualize the differences between versions
  • exports : documents can be exported in several formats, such as RSS or HTML.
  • XML-RPC API : a remote access to the wiki stored is possible
  • SOAP API : a web service access to the stored wiki is currently being developed.
  • blog support : xWiki is what we call a bliki, in other words a wiki that allows users to create blogs.

All these features are basics that have to be implemented by any wiki intended for use in an enterprise context.

But, xWiki's true innovation is something else : the documents may contain scripts to be interpreted before the final rendering. In other words, not only is the wiki syntax interpreted by the radeox engine, but there also exists a programmatic layer that will interpret the scripting language that is currently Velocity based. Note that a Groovy layer is also in the pipeline.

Integration with the eXo Platform is of great interest, and the synergies are obvious. On the one hand it is possible to embed the wiki as a portlet that uses the eXo Platform Organization and Security services, and on the other it is posssible to bundle the eXo Platform SOA stack in the wiki WAR. Both ways the scripted documents may access the services. It is, for example, possible to write a small script inside a text document that would access the chart service and fill it with data extracted from a database. The result is a document that is updated in real time.

Our team's analogy is the use of Microsoft Visual Basic Applications (VBA) programming language inside office applications such as Excel and Word. The xWiki team prefers the analogy of Access applications, as it is also possible to define simple databases in the scripted documents. You choose as you prefer.

The eXo and xWiki teams met several times in Paris, and we must admit that our visions of the future of web applications is quite similar. On one hand we have portlets that look up a service stack, and on the other you have wiki documents that manipulate those services. With the wiki integration the eXo Platform service stack can now be reached with three clients : portal, web services (WSRP), and wiki (stand-alone or portal-embedded).

Note that the xWiki team is currently refactoring its design to adopt the service oriented architecture used by eXo Platform. This move will allow the reuse of xWiki services in portlets or applications other than the wiki itself.

Finally, as quoted in the JCR section, xWiki's Hibernate content store has a design that is quite close to the Java Content Repository. Our teams are currently studying the possibility of using that content store as the database backend for our JCR implementation.

3. New communication services : SMS gateway

Communications services like mail, forum, wiki, and chat are all the more important in a portal context. Actually they are the basics for great groupware functionalities. And, as mobility makes businesses efficient, increasingly powerful mobiles and PDAs are the name of the game.

As already stated, we have started to create a mobile version of the portal. And now we present support for a new communication service : the SMS Service. The uses are limited by the imagination. One could, for example, message coworkers from our SMS portlet, and e-commerce system may alert all registered customers of a new available product, etc.

These use cases may be implemented easily thanks to our service that provides a nice API with a set of methods to send the SMS by abstracting the lower layers. It is, for example, possible to send as well as to acknowledge that the message was received, and to monitor the status of the service.

Technically, this service is an abstraction on top of our partner's - PSWinCom - low level API, that provides a gateway to reach telecom networks all over the world.

The API features:

  • Bulk messaging
  • Multiple recipients per message
  • Gateway and recipient error management
  • Extensibility to support other gateways
  • Either Plain Socket or HTTP based gateway communication
  • See list of Supported operators

To start messaging you need a gateway account. Simply enter the help mode of the SMS Portlet, and you'll manage your subscription from there. The subscription is free, but it requires you to transfer an amount of money to your account.

Here are the prices :

 

Purchase price Message quantity Price pr SMS
EUR Number of SMS EUR
25 and up 362 and up 0,069
120 and up 2.033 and up 0,059
600 and up 12.000 and up 0,050
2.500 and up 54.347 and up 0,046
10.000 and up 232.558 and up 0,043

Once registered you'll receive an e-mail with username and password, hence enter edit mode of the sms portlet, or configure the Provider.getOperator().setUsername(..) and .setPassword(..) and you're ready to go.

Currently the API is limited to PLAIN_TEXT and UNICODE_TEXT support, but in later version we will support the core formats:

  • MessageFormat.PLAIN_TEXT
  • MessageFormat.RINGTONE
  • MessageFormat.OPERATOR_LOGO
  • MessageFormat.CALLER_GROUP_GRAPHIC
  • MessageFormat.PICTURE
  • MessageFormat.VCARD
  • MessageFormat.VCALENDAR
  • MessageFormat.RAW_BINARY_UDH
  • MessageFormat.UNICODE_TEXT

SMS Message example:

 public void sendMessage() throws Exception { ServicesManager manager = ServicesManager.getInstance(); service_ = (SmsService) manager.getComponentInstanceOfType(SmsService.class); Message message = messages_.addMessage(); message.setFrom("4744113344"); message.addRecipient(service_.createRecipient("33655667788")); message.addRecipient("+33 6 55 66 77 88"); message.setFormat(MessageFormat.PLAIN_TEXT); message.setContent("Plain text message"); Provider provider = service_.createProdatProvider("USERNAME", "PASSWORD"); Sender sender = service_.createSender(provider); sender.prepare(messages_); sender.send(); if (messages_.getLogonStatus().equals(LogonStatus.FAILED)) { // Handle gateway authorization error } if (messages_.hasErrorOccured()) { // Handle some message error Recipient r1 = (Recipient) message.getRecipients().get(0); System.out.println("Error for Recipient 1 was: " + r1.getStatus()); } }

Technically it should be possible to adapt any SMS gateway provider - it's supposed to be gateway transaparent. What's required to enhance the service are:

  • Implement the Adapter to prospect custom provider communication (provider, request, response)
  • The Provider to make delegated calls to the operator and messenger
  • The Resolver to implement custom gateway formatting and convertion with respect to plain text, unicode text, ringtone, vCards etc.
  • Integrate with the SMS monitor service

Conclusion

eXo Platform 1.0 RC1 furthers integration of many aspects of enterprise intranets, such as portal and content management systems. This article summarizes the processes we have followed in refactoring and testing the platform to suit our community's requirements. We hope to convince you that an open source software team that uses normative development techniques tends to produce a much more stable and extensible code base.

We would like to thanks :

  • Fahrid Djebarri from RS2i for all the feedbacks he gave us while creating the new navigation model.
  • Ludovic Dubost from xWiki for making the integration possible
  • Knut Rodum from Music Interactive International for the management advises.
  • Roald Brekkhus from PSWin.com for great development support

We will not forget...

Related Content

Related Resources