Real Time Analytics Engine

My current project at work is to architect a solution to capturing network packet data sent over UDP, aggregate it, analyze it and report it back up to our end users. At least, that’s the vision. It’s a big undertaking and there are several avenues of approach that we can take. To that end, I’ve started prototyping to create a test harness to see the write performance/read performance of various software at different levels in our software stack.

Our criteria are as follows (taken from a slide I created):

Our Criteria
Our Criteria

Based on these four requirements pillars, I’ve narrowed down the platform choices to the following:

SERVER

nodejsStrengths

      • Single-threaded, event loop model
      • Callbacks executed when request made (async), can handle 1000s more req than tomcat
      • No locks, no function in node directly performs I/O
      • Great at handling lots of small dynamic requests
      • Logo is cool

Weaknesses

      • Taking advantage of multi-core CPUs
      • Starting processes via child_process.fork()
      • APIs still in development
      • Not so good at handling large buffers

logo-white-big

Strengths

        • Allows writing code as single-threaded (Uses Distributed Event Bus, distributed peer-to-peer messaging system)
        • Can write “verticles” in any language
        • Built on JVM and scales over multiple cores w/o needing to fork multiple servers
        • Can be embedded in existing Java apps (more easily)

Weaknesses

        • Independent project (very new), maybe not well supported?
        • Verticles allow blocking of the event loop, which will cause blocking in other verticles. Distribution of work product makes it more likely to happen.
        • Multiple languages = debugging hell?
        • Is there overhead in scaling over multiple nodes?

PERSISTENCE

For our data, we are considering several NoSQL databases. Our data integrity is not that important, because our data is not make-or-break for a company. However, it is essential to be highly performant, with upwards of 10-100k, fixed-format data writes per second but much fewer reads.

mongo

Architecture

        • Maps in memory space directly to disk
        • B-tree indexing guarantee logarithmic performance

Data Storage

        • “Documents” akin to objects
        • Uses BSON (binary JSON)
        • Fields are key-value pairs
        • “Collections” akin to tables
        • No conversion necessary from object in data to object in programming language

Data-Replication

        • Mongo handles sharding for you
        • Uses a primary and secondary hierarchy (primary receives all writes)
        • Automatic failover

Strengths

        • Document BSON structure is very flexible, intuitive and appropriate for certain types of data
        • Easily query-able using mongo’s query language
        • Built-in MapReduce and aggregation
        • BSON maps easily onto JSON, makes it easy to consume in front end for charting/analytics/etc
        • Seems hip

Weaknesses

        • Questionable scalability and write performance for high volume bursts
        • Balanced B-Tree indices maybe not best choice for our type of data (more columnar/row based on timestamp)
        • NoSQL but our devs are familiar with SQL
        • High disk space/memory usage

 

cassandra
Architecture

        • Peer to peer distributed system, Cassandra partitions for you
        • No master node, so you can read-write anywhere

Data Storage

        • Row-oriented
        • Keyspace akin to database
        • Column family akin to table
        • Rows make up column families

Data Replication

        • Uses a “gossip” protocol
        • Data written to a commit log, then to an in-memory database (memtable) then to disk using a sorted-strings table
        • Customizable by user (allows tunable data redundancy)

Strengths

        • Highly scalable, adding new nodes give linear performance increases
        • No single point of failure
        • Read-write from anywhere
        • No need for caching software (handled by the database cluster)
        • Tunable data consistency (depends on use case, but can enforce transaction)
        • Flexible schema
        • Data compression built in (no perf penalty)

Weaknesses

        • Data modeling is difficult
        • Another query language to learn
        • Best stuff is used by Facebook, but perhaps not released to the public?

 

467px-Redis_Logo.svg

            TBD

 

        FRONT-END/REST LAYER

For our REST Layer/Front-end, I’ve built apps in JQuery, Angular, PHP, JSP and hands-down my favorite is Angular. So that seems like the obvious choice here.

AngularJS-large

Strengths

      • No DOM Manipulation! Can’t say how valuable this is …
      • Built in testing framework
      • Intuitive organization of MVC architecture (directives, services, controllers, html bindings)
      • Built by Google, so trustworthy software

Weaknesses

      • Higher level than JQuery, so DOM manipulation is unadvised and more difficult to do
      • Fewer 3rd party packages released for angular
      • Who knows if it will be the winning javascript framework out there
      • Learning curve

Finally, for the REST API, I’ve also pretty much decided on express (if we go with node.js):

express

Strengths

      • Lightweight
      • Easy integration into node.js
      • Flexible routing and callback structure, authorization & middleware

Weaknesses

      • Yet another javascript package
      • Can’t think of any, really (compared to what we are using in our other app – java spring

These are my thoughts so far. In the following posts, I’ll begin describing how I ended up prototyping our real time analytics engine, creating a test harness, and providing modularity so that we can make design decisions without too much downtime.

Mapping Google Datatable JSON format to Sencha GXT ListStores for charting

By now, if you’ve been keeping up with my blog, you’ll notice that we make extensive use of charting in our application. We decided to go with Sencha GXT charting because we were under time pressure to finish our product for a release. However, we weren’t so crunched for time that I decided to model the data in an unintelligent way. Since charts essentially graphically display tabular data, I wanted to ensure that if we were to move to a different graphing package (which we are, btw) in the future, our data was set up for easy consumption.

After tooling around a bit, I decided on google datatable JSON formatting for our response data from queries made to our server. Google datatable formatting essentially just mimics tabular data in JSON formatting. It’s also easily consumable by google or angular charts. Having used angular charts in the past, I knew that the data format would map nicely to charts. However, the immediate problem still loomed, which is how to convert from google datatable JSON format to consumable ListStores for consumption by GXT charts?

Solution: In the end, I ended up writing an adapter with specific implementations for consuming our backend charting data. I needed to write a Sencha/GXT specific one. Here’s an example of the data we receive from our REST query.

//Note: some data omitted for readability
{
    "offset": 0,
    "totalCount": 1,
    "items": [
        {
            "title": "Registration Summary",
            "data": {
                "cols": [
                    {
                        "id": "NotRegistered",
                        "label": "Not Registered",
                        "type": "number"
                    },
                    {
                        "id": "Registered",
                        "label": "Registered",
                        "type": "number"
                    }
                ],
                "rows": [
                    {
                        "l": "9608",
                        "c": [
                            {
                                "v": "5",
                                "f": null
                            },
                            {
                                "v": "4",
                                "f": null
                            }
                        ]
                    },
                    {
                        "l": "9641",
                        "c": [
                            {
                                "v": "2",
                                "f": null
                            },
                            {
                                "v": "5",
                                "f": null
                            }
                        ]
                    },
                    {
                        "l": "9650SIP",
                        "c": [
                            {
                                "v": "3",
                                "f": null
                            },
                            {
                                "v": "0",
                                "f": null
                            }
                        ]
                    }
                ]
            }
        }
    ]
}

And here’s an example of how it’s converted into a Sencha ListStore:

[
    {
        v1=4,
        label0=NotRegistered,
        v0=5,
        label1=Registered,
        id1=Registered,
        type1=number,
        id0=NotRegistered,
        type0=number,
        f1=null,
        l=9608,
        f0=null
    },
    {
        v1=5,
        label0=NotRegistered,
        v0=2,
        label1=Registered,
        id1=Registered,
        type1=number,
        id0=NotRegistered,
        type0=number,
        f1=null,
        l=9641,
        f0=null
    },
    {
        v1=0,
        label0=NotRegistered,
        v0=3,
        label1=Registered,
        id1=Registered,
        type1=number,
        id0=NotRegistered,
        type0=number,
        f1=null,
        l=9650SIP,
        f0=null
    }
...
]

Step 1: Define the ChartModel I/F to map to the JSON datatable format.

Here’s an example of a ChartModel java object that maps to JSON object.

public abstract class ChartModel {
	
	private static ChartFactory factory = GWT.create(ChartFactory.class);
	private static JsonListReader<ChartObjects, ChartObject> reader = new JsonListReader<ChartObjects, ChartObject>(factory, ChartObjects.class );
	
	/**
	 * These constants are fixed per the value retrieved from JSON
	 */
	public static final String POINT = "point";
	public static final String COL_LABEL = "label";
	public static final String COL_TYPE = "type";
	public static final String COL_ID = "id";
	public static final String CELL_VALUE = "v";
	public static final String CELL_FORMATTED_VALUE = "f";
	public static final String ROW_LABEL = "l";
	
	/*
	 * Define the data model
	 */
	
	public interface ChartObjects extends ModelList<ChartObject>{}
	
	public interface ChartObject {
		String getTitle();
		ChartData getData();
	}
	
	public interface ChartData {
		List<ChartCol> getCols();	//represents # series
		List<ChartRow> getRows();	//an array of array of cells, i.e. all the data
	}
	
	public interface ChartCol {
		String getId();
		String getLabel();
		String getType();
	}
	
	public interface ChartRow {
		List<ChartCell> getC();	//an array of chart cells, one per column
		String getL();	//label for this group of chart cells (e.g. a timestamp)
	}
	
	public interface ChartCell {
		String getV();	//get value
		String getF();	//get formatted value
	}

	public abstract ListStore<DataPoint> getListStore(ChartObject chartObject) throws Exception;
...
}

You can see that I also added an abstract getListStore method that requires any consumer of the ChartObject to define how the ChartObject gets modeled to the ListStore.

Step 2. Create an Adapter that extends the ChartModel and implements the function getListStore.

Here’s an example of a StackedBarChart model (the models turned out to be different for multiple series in a stacked bar chart and for example, a pie chart. Line charts and stacked bar charts turned out to be the same.


public class StackedBarChartModel extends ChartModel {
	@Override
	public ListStore<DataPoint> getListStore(ChartObject chartObject) throws Exception {
		return new GXTChartModelAdapterImpl(chartObject).getSeries(ChartType.STACKED_BAR);
	}
}

You can see that this method instantiates a new adapter instance for GXT charts, passes in the chart object (modeled to the datatable JSON format) and returns the appropriate chart series (in this case a BarSeries GXT object).

Step 3. Map the chart object to a ListStore.

public class GXTChartModelAdapterImpl implements GXTChartModelAdapter {


	@Override
	public ListStore<DataPoint> getSeries(ChartType type) throws Exception {
		switch(type) {
			case LINE:
				return getLineSeriesListStore();
			case STACKED_BAR:
				return getStackedBarSeriesListStore();
			case BAR:
				return getLineSeriesListStore();
			case PIE:
				return getPieSeriesListStore();
			default:
				throw new Exception("Unknown data series type.");
		}
	}

/**
	 * Returns a ListStore for a BarSeries GXT object to add to a chart
	 * @return
	 */
	private ListStore<DataPoint> getStackedBarSeriesListStore() {
		ListStore<DataPoint> _listStore = new ListStore<DataPoint>(new ModelKeyProvider<DataPoint>() {
			@Override
			public String getKey(DataPoint item) {
				return String.valueOf(item.getKey());
			}
		});
		
		int numPoints = this.getNumPoints();  //The number of rows in the rows obj in ChartObject
		int numSeries = this.getNumSeries();  //The number of rows in the col obj in ChartObject
		
		for (int i = 0; i < numPoints; i++) {	
			//This key must be unique per DataPoint in the store
			DataPoint d = new DataPoint(indexStr(ChartModel.POINT,Random.nextInt()));
			
			for (int index = 0; index < numSeries; index++) {
				d.put(indexStr(ChartModel.COL_LABEL, index), chartObject.getData().getCols().get(index).getLabel());
				d.put(indexStr(ChartModel.COL_TYPE, index), chartObject.getData().getCols().get(index).getType());
				d.put(indexStr(ChartModel.COL_ID, index), chartObject.getData().getCols().get(index).getId());
			}
			
			ChartRow row = chartObject.getData().getRows().get(i);	//get the i-th point
			d.put(ChartModel.ROW_LABEL, row.getL());
			for (int j = 0; j < numSeries; j++) {
				if (row.getC().get(j) != null) {	//if ith-point is not blank
					d.put(indexStr(ChartModel.CELL_VALUE, j), row.getC().get(j).getV());
					d.put(indexStr(ChartModel.CELL_FORMATTED_VALUE, j), row.getC().get(j).getF());
				} else {	//otherwise, assume 0 value
					d.put(indexStr(ChartModel.CELL_VALUE, j), "0");
					d.put(indexStr(ChartModel.CELL_FORMATTED_VALUE, j), "");
				}
			}
			_listStore.add(d);
		}
		
		return _listStore;
	}
...
}

A few things to note here.

  • A DataPoint is basically just a map of keys and value pairs. Since the DataPoint object is specific to the ListStore, the implementation is in the interface GXTChartModelAdapter. To do this, I used the instructions here: https://www.sencha.com/blog/building-gxt-charts/
  • Next, you can see that for each bar I have (not each stack), I create a new DataPoint object that contains a the corresponding value of the specific row in the ChartObject.rows object. I know that each column corresponds to a stacked bar, so I increment the “v” value by 1 and use this as the key / value pair to the DataPoint.
  • Finally, because the number of columns defines the number of stacks I expect in each bar, if there isn’t a value (because our backend didn’t provide one), I assume the value is 0. This enforces that the v(n) in the first bar will correspond to v(n) in any subsequent bars.
  • I added an “l” value to each row in the ChartObject.rows obj which corresponds to a label for the entire stacked bar. I know that in the datatables JSON format you can specify the first row in each row obj to be a String and correspond to the label for each bar. However, it’s harder to enforce this type of thing in java and we expected numeric data back each time.


Step 4. Pass in the data into a Sencha GXT Chart!

Now the easy part. Essentially the data is transformed from the datatables format to a ListStore with each DataPoint key: v0 – vN corresponding to a stack in a bar.

Thus when instantiating the chart, this is all I had to do:

private BarSeries<DataPoint> createStackedBar() {

		BarSeries<DataPoint> series = new BarSeries<DataPoint>();
		series.setYAxisPosition(Position.LEFT);
		series.setColumn(true);
		series.setStacked(true);
		
		//numSeries is the number of bars you want stacked, it conforms to the chartObject.cols().length
		for (int i = 0; i < numSeries; i++) {
			MapValueProvider valueProvider = new MapValueProvider(ChartModel.CELL_VALUE + i);
			series.addYField(valueProvider);
		}
		
		return series;
	}

When I instantiate the BarSeries, I just pass in a key for each the number of stacked bars in my response data.

Finally, the finished product.

Screen Shot 2014-04-09 at 5.03.44 PM

Custom Resized Portlets using Sencha GXT 3.1 Beta

Sencha makes a container called the PortalLayoutContainer available through its API which allows the developer to add portlets to the container. The container extends the VerticalLayoutContainer and works by the user adding portlet widgets to the container in predefined column widths. The PLC doesn’t allow users to specify a grid with a predefined portlet height so that portlets dropped in will fit that size. Hence, we had some portlets that didn’t render nicely because they were too big for screens with low vertical resolutions. Also, because we don’t know what kind of screen resolution our users are going to use, we couldn’t just set a fixed height and forget about it.

In other words, sometimes one just wants to add a portlet that say, takes up 50% of the vertical screen real-estate rather than a fixed 450px. How did I solve this problem? Disclaimer: I admit that this solution is a bit of a hack, but without any other exposed API I did what I had to do.

Where's the rest of my portlet?
Where’s the rest of my portlet?

Solution: I had to resort to event handlers that listened for a resize of the PortalLayoutContainer. Essentially what this method does is traverse up the widget hierarchy until it finds a PortalLayoutContainer, then binds a resize handler for the specific portlet to the PLC. When the resize event is triggered on the PLC, each individual portlet is resized based on the initial parameter of how much screen real estate the portlet should take up. I also set a minimum size so that the portlets would not continue to resize if the overall PLC got too small (where the data started looking bad). You could easily set a maximum size as well so the portlets would not continue resizing after a certain point.

/**
	 * Attaches a resizeHandler to the parent (which is assumed to be a PortalLayoutContainer
	 * You need to call this method after the _portlet is added to the container
	 * @param width - the percentage of width of the window to resize to
	 * @param height - the percentage of height of the window to resize to
	 */
	public void bindResizeHandler(final double width, final double height) {
		if (_portlet != null) {
			_config.setResizable(Boolean.TRUE);
			
			//traverse up widget hierarchy until we reach PortalLayoutContainer
			Widget w = _portlet.getParent();
			while (w != null && !(w instanceof PortalLayoutContainer)) {
				w = w.getParent();
			}
			
			//Resize
			PortalLayoutContainer plc = ((PortalLayoutContainer) w);
			if (plc != null) {
				plc.addResizeHandler(new ResizeHandler () {

					@Override
					public void onResize(ResizeEvent event) {
						int newWidth = Math.max((int) (event.getWidth() * width) - 10, _config.getMinWidth());
						int newHeight = Math.max((int) (event.getHeight() * height) - 10, _config.getMinHeight());
						
						//Only resize if width or height > minWidth or minHeight - note min width doesn't quite work right
						//because GXT widgets still try to autoresize
						if (_config.getWidth() != newWidth || _config.getHeight() != newHeight) {
							_config.setWidth(newWidth);
							_config.setHeight(newHeight);
							resize(_config.getWidth(), _config.getHeight());
						}
					}
					
				});
			}
		}
	}
	
	@Override
	public void resize(int width, int height) {
		if (_config.getResizable()) {
			_portlet.setWidth(width);
			_portlet.setHeight(height);
			_portlet.forceLayout();
		}
	}

I know that this turns the portal container into more of a “dashboard,” but that’s what our requirements called for. If you want a portlet to have a fixed size, simply don’t bind the handler to that specific portlet and the height will remain fixed.

Here’s how I bound each portlet to the PLC during instantiation.

//a PLC is made with two columns, each 50% and one column that's 100% (a hack to get a 50/50/100 split
			_portalContainer = new PortalLayoutContainer(3);
			_portalContainer.setColumnWidth(0, .5);
			_portalContainer.setColumnWidth(1, .5);
			_portalContainer.setColumnWidth(2, 1);
			_updateList = new ArrayList<UpdateablePortlet>();

//A portlet config is my own object that has parameters like whether the portlet is resizable and options specific to the data used to populate the portlet
			PortletConfig _tjpConfig = new PortletConfig();
			_tjp = new TestJobsPortlet(_tjpConfig);
			_portalContainer.add(_tjp.asWidget(), 1);

//This will add the portlet to the top right corner of the PLC and bind a resize handler to take up 50% of the width and 50% of the height (i.e. 1/4 of the screen) whenever the PLC is resized
			_tjp.bindResizeHandler(.5, .5);

And here’s the result:

much better!
much better!

Custom axes with Sencha 3.1 GXT Charts

Here’s a short idiom for today. If you have a Sencha chart with multiple BarSeries, you can click on the legend by default, which will “exclude” certain values from the recalculation of the y-axis minimum and maximum. We found that there are certain times when Sencha will compute the max range of a chart as a value less than 10, while the axis has 10 tick marks. This leaves some ugly decimal points in the y-axis labels, which just doesn’t make sense in some instances (e.g. you can’t have a fraction of a whole object).

Bad! You can't have a fraction of a device!
Bad! You can’t have a fraction of a device!

In order to get whole numbers here, I added a legend handler which recomputes the axis and sets a minimum value if Sencha would compute a < 10 max for the y-axis. Here's the code snippet:

			legend.addLegendSelectionHandler(new LegendSelectionHandler() {

				@Override
				public void onLegendSelection(LegendSelectionEvent event) {
					if (axis.getTo() < 10) {
						axis.setMaximum(10.0);
					} else {
						axis.setMaximum(Double.NaN);
						axis.setInterval(Double.NaN);
					}
					axis.calcEnds();
					axis.drawAxis(false);
					_chart.redrawSurface();
				}
				
			});

And here's the result:

Screen Shot 2014-04-01 at 2.04.52 PM
Ahh, much better.

Mapping tooltip of different key onto a GXT BarSeries

Here’s a little idiom for when you’re using one key for the value of a stacked bar chart but want to map a different ValueProvider onto your tooltip (or label).

		//Tooltip Config - gets (f) value but if null, just return the int
	    SeriesToolTipConfig<DataPoint> config = new SeriesToolTipConfig<DataPoint>();
	    config.setLabelProvider(new SeriesLabelProvider<DataPoint>() {
	        @Override
	        public String getLabel(DataPoint item, 
	        		ValueProvider<? super DataPoint, ? extends Number> valueProvider) {
	        	String vindex = valueProvider.getPath();
	        	String findex = vindex.replace(ChartModel.CELL_VALUE, ChartModel.CELL_FORMATTED_VALUE);
	        	MapLabelProvider labelProvider = new MapLabelProvider(findex);
	        	String tooltip = labelProvider.getValue(item);
	        	return tooltip == null || tooltip.equals(_chartConstants.emptyLabel()) ? 
	        		  String.valueOf(valueProvider.getValue(item).intValue()) : tooltip;
	        }
	    });
series.setToolTipConfig(config);

By default, Sencha GXT charts only passes in the valueProvider and the data item used to render the stacked bar chart. So what if you want to provider a different value for the tooltip or label? How do you add text? Since be follow the google datatables JSON format, we use a key of “v” in a cell array to represent the numeric value used to generate the chart. We use a key “f” in the same cell array to represent the tooltip.

Here’s what our JSON looks like and here’s the resultant store (After the adapter transforms the data).

{
    "offset": 0,
    "totalCount": 1,
    "items": [
        {
            "title": "Registration Summary",
            "data": {
                "cols": [
                    {
                        "id": "Stacked bar one",
                        "label": "Hamburgers",
                        "type": "number"
                    },
                    {
                        "id": "Stacked bar two",
                        "label": "Fries",
                        "type": "number"
                    }
                ],
                "rows": [
                    {
                        "l": "McDonalds",
                        "c": [
                            {
                                "v": "5",
                                "f": "5 hamburgers sold!"
                            },
                            {
                                "v": "2",
                                "f": "2 fries sold"
                            }
                        ]
                    },
                    {
                        "l": "Burger King",
                        "c": [
                            {
                                "v": "0",
                                "f": "0 hamburgers sold :("
                            },
                            {
                                "v": "3",
                                "f": "3 fries sold"
                            }
                        ]
                    }
... /* More omitted */
                ]
            }
        }
    ]
}

And here’s the resultant store:

[
    {
        v1=2,
        label0=Hamburgers,
        v0=5,
        label1=Fries,
        id1=Stacked bar two,
        type1=number,
        id0=Stacked bar one,
        type0=number,
        f1=2 fries sold,
        l=McDonalds,
        f0=5 hamburgers sold!
    },
    {
        v1=3,
        label0=Hamburgers,
        v0=0,
        label1=Fries,
        id1=Stacked bar two,
        type1=number,
        id0=Stacked bar one,
        type0=number,
        f1=3 fries sold,
        l=9650SIP,
        f0=0 hamburgers sold 😦
    }
... /* more omitted */
]

As you can see, we use v0 – vN to as values to render per bar in our chart, where each of v0 – vN corresponds to 1 stack within the bar. Here’s the resulting tooltip:

Custom tooltip
Custom tooltip

Drill-down charting with Sencha GXT 3.1

My latest assignment at work was to use charting in GXT and build a widget that has the ability to render one chart and drill down into charts of greater detail when the user clicks on an object in the parent chart. To complicate matters, the data used to render the drill-down chart is variable depending on the parameters passed in through our REST API. Finally, I wanted to encapsulate all this behavior within a portlet that could be added to a portal container by the user. How did I solve this?

When I started out, I knew that there were already handlers that could be attached to a series within a chart, but as with all things Sencha, nothing is as easy as it seems at first glance. Furthermore, I was battling with Sencha’s currently buggy beta release of GXT 3.1, which led me more than once to waste several hours thinking that I was doing something wrong when in fact, there was a problem on Sencha’s end. At any rate, this is how I solved the problem:

      Step 1: Define a chart object whose store can be manipulated through a configuration

For the drill-down chart, I decided to implement my own object which takes an object called ChartConfig. My ChartConfig class is just an object containing various options on how to render the chart, including what optional parameters to pass into my REST call.

public class ChartConfig {

	public int height;
	public int width;
	public Boolean initiallyInvisible;
	public List<Color> seriesColors; //Color of series in order
	public HashMap<String, String> options;	//REST options

}

I then defined my custom chart object to take this configuration into account when rendering itself.

public class MyBarChart extends UpdateableChart implements IsWidget {	
	
	public RegistrationBarChart() {
		super();
		_model = new StackedBarChartModel();
	}
	
	public RegistrationBarChart(ChartConfig _config) {
		super(_config);
		_model = new StackedBarChartModel();
	}
	
    @Override
	public Widget asWidget() 
	{
		if (_chart == null) 
		{	
			try {
				buildChart();
			} catch (Exception e) {
				e.printStackTrace();
			}
		}

		return _chart;
	}
}

buildChart() is simply a method that uses the configuration, instantiates a new Chart and sets the ListStore that's used to render the chart. Don't worry about the StackedBarChartModel(). This is a custom data model and adapter that I had to implement to convert from JSON (following google's datatable format here: https://developers.google.com/chart/interactive/docs/reference#dataparam) to a ListStore consumable by Sencha (an entirely different battle. To be described in a follow-up post).

In any case, UpdateableChart is an abstract class that executes the REST API call based on the ChartConfig and regenerates the ListStore. Since I figured that all of our charts would follow the same logic here, I made it an abstract class with the subclasses needing to implement only one method: buildSeries();

public abstract class UpdateableChart {
	/**
	 * Abstract Methods
	 */
	public abstract void buildSeries();

	/**
	 * Refreshes the current chart with parameters passed in through the ChartConfig
	 * These parameters, in the HashMap<String,String> options map, get serialized
	 * and appended to the base REST URL.  
	 * 
	 * If options == NULL, the default url is used.
	 * 
	 * Upon success, the function will get a newstore and will call
	 * 	updateStore(newStore)
	 * @param c
	 */
	public void refresh(ChartConfig c) {
		String url = (c == null || c.getOptions().isEmpty()) ? _model.getListUrl() : serializeOptions(_model.getListUrl(), c.getOptions());
		
		_config = c;	//update config upon refresh
		
		RequestBuilder restReq = new RequestBuilder(RequestBuilder.GET,
				URL.encode(url)); //Use your flavor of request builder
		
		restReq.sendRequest(null, new () {

			@Override
			public void onSuccess(Response response) {
				JsonListReader<ChartObjects, ChartObject> reader = _model.getListReader();
				ListLoadResult<ChartObject> charts = reader.read(null, response.getText());
				//ListLoadResult<ChartObject> charts = reader.read(null, _resources.jsonPieData().getText());
				List<ChartObject> chartList = charts.getData();
				
				try {
					ChartObject chartObject = chartList.get(0);	//right now, only render first chart
					ListStore<DataPoint> newStore = _model.getListStore(chartObject);
					updateStore(newStore);
				} catch (Exception e) {
					e.printStackTrace();
				}
			}
			
			@Override
			public void onFailure(RestError restError) {
				//logger.error("Rest error" + restError.getErrorMessage());
			}
		});
	}
}

In the method above, you can see that after the REST Call is made and the JSON response is parsed into a ChartObject (basically a schema defined to work with Sencha's JSONReader), I use the adapter to convert the ChartObject into a ListStore which is then used to update the store which renders the chart. Here's an example of an implementation for my bar chart of buildSeries().

	@Override
	public void buildSeries() {
		for (int x = 0; x < _chart.getSeries().size(); x++) {
			_chart.removeSeries(_chart.getSeries(x));
		}
		
		BarSeries<DataPoint> series = createStackedBar();
		_chart.addSeries(series);
		
		refreshChart();  //regenerates chart axes
		setVisible(true);  //turn visible - after REST call returns
	}

In a nutshell, the work flow is like this:

  • Create a model to represent a ListStore which is used in a chart.
  • Instantiate a chart with a dummy store, which models this ListStore
  • Implement buildSeries() which will take the new ListStore object (in the class) and use it to regenerate the series
  • Re-render the chart
  • This framework makes it so that any charts that we add to our portal are able to be updated simply by calling the “refresh(ChartConfig config)” method.

        Step Two: From your entry point, add a method that will rerender a chart based on some variable parameter

    In our portlet, we have a pie chart that contains drill-downs to bar charts depending on what slice you pick. Before binding the drill-down event to the actual pie chart, I just wanted to make sure that charts could rerender properly.

    I did this by creating an entry point to the refresh(ChartConfig config) method on my bar chart object. I added a selector to our portlet widget and bound it to a “toggleView()” method which basically just generates the ChartConfig based on the selected value and calls refresh on the correct chart. Notice that I set both charts to visible(false) so that they don’t appear to the user. When implementing the buildSeries() method for both my pie and bar charts, I actually make the chart visible after the rest call returns.

    
    private void toggleViews(SeriesSelectionEvent<DataPoint> event) {
    		displayParent = !displayParent;
    		
    		//Show Pie Chart
    		if (displayParent) {	
    			_pieChart.setVisible(false);
    			_pieChart.refresh(_pieChartConfig);
    			_barChart.setVisible(false);
    			_backButton.disable();
    		}
    		
    		//Show Bar Chart (Drill Down)
    		else {
    			_pieChart.setVisible(false);
    			_barChart.setVisible(false);  //set invisible until rest call loads, buildSeries() will make the chart visible again
    			
    			//refresh registration bar chart depending on what slice is selected
    			_barChartConfig = _barChart.getChartConfig();
    			_barChartConfig.setOption(ChartConfig.TYPE, event.getItem().get(ChartModel.COL_LABEL));
    			_barChartConfig.setOption(ChartConfig.FILTER, "registrationState(EQ)"+event.getItem().get(ChartModel.COL_LABEL));
    			_barChartConfig.setOption(ChartConfig.LIMIT, "10");
    			_barChartConfig.setOption(ChartConfig.ORDER_BY, "count");
    			_barChartConfig.setOption(ChartConfig.ORDER_DIR, "DESC");
    			
    			if (_barChartConfig.getOption(ChartConfig.TYPE).equalsIgnoreCase(pc.registered())) {
    			   List<Color> c = new ArrayList<Color> ();
    			   c.add(new RGB(148, 174, 10));
    			   _barChartConfig.setSeriesColors(c);
    			} else if (_barChartConfig.getOption(ChartConfig.TYPE).equalsIgnoreCase(pc.unregistered())) {
    			   List<Color> c = new ArrayList<Color> ();
    			   c.add(new RGB(17, 95, 166));
    			   _barChartConfig.setSeriesColors(c);
    			} else if (_barChartConfig.getOption(ChartConfig.TYPE).equalsIgnoreCase(pc.partially_registered())) {
    			   List<Color> c = new ArrayList<Color> ();
    			   c.add(new RGB(166, 17, 32));
    			   _barChartConfig.setSeriesColors(c);
    			}
    			_barChart.refresh(_barChartConfig);
    			_backButton.enable();
    		}
    	}
    

    Finally, the last step was to bind the handler to the actual slices of my pie chart. One thing to keep in mind here. Because the handler is bound to the series of the pie chart, I couldn’t rebuild the pie series otherwise the event handler would actually fall off. Thus, I knew that after the first time the series was built, I needed to bind the handler, then leave it up to the ListStore to regenerate the pie chart. To do this, when I instantiate the pie chart, I had to set an empty ListStore to the store to render the pie chart. I’ve also included my buildSeries implementation for the pie chart below so you can see that I don’t rebuild the series at all with the pie chart.

    public class MyPieChart extends UpdateableChart implements IsWidget {
    	
    	private PieSeries<DataPoint> _series;
    	
        @Override
    	public Widget asWidget() 
    	{
    		if (_chart == null) 
    		{	
    			try {
    				buildChart();
    			} catch (Exception e) {
    				e.printStackTrace();
    			}
    			
    	    	//refresh(_config);	//refresh the chart store	
    		}
    
    		return _chart;
    	}
    	
    	/**
    	 * 	Iterates over each series and add a line series for each
    	 *  This is done only once.
    	 * @throws Exception 
    	 */
        @Override
    	public void buildChart() throws Exception {
    		_chart = new Chart<DataPoint>();
    		_chart.setDefaultInsets(10);
    		_store = _model.getBlankStore(numSeries, 1);   //This is where I bind the blank store to the chart
    		_chart.bindStore(_store);
    		_chart.setShadowChart(false);
    		_chart.setAnimated(true);		
    		_chart.setVisible(!_config.getInitiallyInvisible());
    		
    	    _series = createPieSeries();   //Note that this is only created once.
    	    _series.addColor(slice1);
    	    _series.addColor(slice2);
    	    _series.addColor(slice3);
    		_chart.addSeries(_series);
    		
    		final Legend<DataPoint> legend = new Legend<DataPoint>();
    		legend.setPosition(Position.RIGHT);
    		legend.setItemHighlighting(true);
    		legend.setItemHiding(true);
    		legend.getBorderConfig().setStrokeWidth(0);
    		_chart.setLegend(legend);
    		
    
    	}
     
            @Override
    	public void buildSeries() {
    		//Only build the Series the first time so we don't lose the selection handler binding
    		if (_store.size() > 0)
    			setVisible(true);
    	}
    }
    
        Step 3: Bind the drill down event handler to the series of the parent chart

    In the portlet widget, which as described above has both a pieChart and a barChart, after instantiating the pieChart, I bound the SeriesSelectionHandler to the pieChart in order to enable the drilldown. I didn’t have to wait for the REST call return to have data either, because I created a PieSeries with my dummy ListStore. This part I’m not that proud of. Because I bind the SeriesSelectionHandler outside the actual MyPieChart object, the implementation violates encapsulation. I could have implemented the SeriesSelectionHandler directly in MyPieChart, but I didn’t want to do this because I felt it would confuse developers later on why a PieChart had a BarChart object in it. I thought it would also set a bad precedent for development because it would seem like we needed to nest objects within objects whenever we wanted to create a drilldown. My implementation, though perhaps not ideal, increased our flexibility because it allows me to create PieCharts that don’t necessarily have to have barChart objects in them and yet still maintain the drilldown capability. In the future, we could create a datamodel that contains a tree-like hierarchy for parent/child drilldowns, where the back button always rerendered the parent, and the drilldown always rendered the child clicked. Something to think about. In any case, here is the binding of the toggleView() method to the SeriesSelectionEvent.

    	@Override
    	public Widget asWidget() {
    _pieChart.addSeriesHandlers(new SeriesSelectionHandler<DataPoint> () {
    
    				@Override
    				public void onSeriesSelection(
    						SeriesSelectionEvent<DataPoint> event) {
    					toggleViews(event);
    				}
    				
    			});
    }
    

    That’s it for now! Let me know if you have any questions or comments. In a post to come, I’ll describe how I modeled the google datatables JSON format into a ChartObject and how it’s consumed by GXT charts.

    The end result:

    The top level pie chart
    The top level pie chart

    Drilling down by clicking on a pie slice.
    Drilling down by clicking on a pie slice.

    Node.js vs Vert.x (Part 1)

    Now for a change of pace. Recently at work, we’ve been trying to figure out what platform to build an application that will handle serving realtime data to customers. We want the system to be fast, scalable and able to handle hundreds, maybe thousands or even tens of thousands of requests per second.

    I did a bit of prototyping in both node and vert.x to see how they performed under pressure. To do this, I built a cute webapp that makes a bunch of requests on basic web servers written in both node.js and vert.x to see how fast they could respond and how they would handle under a heavy load of requests. Here’s a picture of the ui made for the webapp (build in angular.js).

    Node webapp
    Node webapp

    I created a form that allows for various inputs. In it, one can specify several variables including the following:

    The variables:

    • Iterations – number of http requests to make
    • Block Size – How often a result is computed (reports start time for request, end time, average time per call (ms) and total time (ms) )
    • Range – How many results to display on screen – the graph tracks this
    • Polling – Toggle On/Off (this will start polling requests as fast as node.js can handle them. These are serial in nature).
    • Show Graph – toggle graphing on/off (off will provide better javascript performance)

    Thanks to angular-seed for the fast prototyping and angular-google-chart for charting.

    Benchmarking Parameters: Each request is a simple get request made to the respective webserver, which then writes a header and a “Hello” response. The requests are made through angular’s $http method. When a successful response is received, the callback function calls another $http request, until the number of successful responses received equals the number of iterations specified. I measure the time it takes from the time the request is made until the number of requests received per block is complete.

    Time Keeping: I try to avoid all delays that could be attributable to javascript rendering (e.g. the timestamp is taken when the first request in the block is made. Then the timestamp is recorded when the # of responses in a block is received. I send both these parameters to a javascript function, which is responsible for rendering the results to the display. I also added a function to enable polling requests to be made, which makes $http requests as fast as responses can be received in order to add stress to the server’s load. This is enabled with the “polling” button.

    Here’s a snippet of the webserver source code.

    In Node.js:

    StaticServlet.prototype.handleRequest = function(req, res) {
      var self = this;
    
    
    
      var path = ('./' + req.url.pathname).replace('//','/').replace(/%(..)/g, function(match, hex){
        return String.fromCharCode(parseInt(hex, 16));
      });
    
      console.log(path);
      if (path == './show') {
    
          res.writeHead(200, {'content-type': 'text/html'});
          res.write("Hello: ");
          res.end();
    
      } else if (path == './version') {
        res.writeHead(200, {'content-type': 'text/html'});
          res.write("Node.js version: " + 'v0.10.26');
          res.end();
        
      } else {
        var parts = path.split('/');
        if (parts[parts.length-1].charAt(0) === '.')
          return self.sendForbidden_(req, res, path);
        else {
          fs.stat(path, function(err, stat) {
            if (err)
              return self.sendMissing_(req, res, path);
            if (stat.isDirectory())
              return self.sendDirectory_(req, res, path);
            return self.sendFile_(req, res, path);
          });
        }
      }
    }
    

    In Vert.x:

    
    server.requestHandler(function(req) {
    var file = '';
    
      if (req.path() == '/show') {
      	  req.response.chunked(true);
      	  req.response.putHeader('content-type','text/html');
          req.response.write("Hello: ");
          req.response.end();
      } else if (req.path() == '/version') {
      	  req.response.chunked(true);
      	  req.response.putHeader('content-type','text/html');
          req.response.write('Vertx version: ' + '2.0.2-final (built 2013-10-08 10:55:59');
          req.response.end();
      } else if (req.path() == '/') {
        file = 'index.html';
        req.response.sendFile('app/' + file);   
      } else if (req.path().indexOf('..') == -1) {
        file = req.path();
        req.response.sendFile('app/' + file);   
      }
      
    

    See? Dead simple. Of course there are lots of flaws with this methodology (e.g. the webservers are only serving static data, are writing short responses, are not optimized, etc.). It wasn’t my intention to come to a hard conclusion with this. I just wanted a data point and to experiment with both platforms. It turns out they (at least in this test) came very close to one another in terms of performance. Both servers were running on my machine, which specs are listed below.

    System Specs: Macbook Pro, mid-2012, 2.3ghz with 16gb ram and a 512gb ssd. Both webservers are running locally on my machine with a bunch of other apps open.

    And here are some preliminary results:

    Here’s the node.js webserver, with polling turned on:

    Node while polling, 1000 requests, 50 request block size
    Node while polling, 1000 requests, 50 request block size

    Here’s the vert.x webserver, with polling turned on:

    Vert.x while polling, 1000 requests, 50 request block size
    Vert.x while polling, 1000 requests, 50 request block size

    You can see that they’re very close. Next I tried stressing both servers a bit through running several concurrent queries and several “instances” of the web app. In a later post, I’ll put up some more detailed results with trying to stress both webservers out. The response time definitely slows down as more are being made.

    Conclusions: Both webservers are surprisingly close in terms of response/processing/overhead time. My CPU usage goes a bit higher on the vert.x server, but I do have several other applications running. I also haven’t tested running multiple instances of the same verticle yet in vert.x, or trying to fork processes in node.js. Both webservers are as barebones as they get. So in other words, I can’t make any hard conclusions yet, except to say that both servers are

    • Able to handle high loads of requests per second (probably on the order of 500 – 1000
    • Out of the box, both servers run roughly equivalently

    These results seemed a little bit surprising to me, given that on the web vert.x seems to have faster results. One factor that may contribute to this is the lack of realism in server response. It’s probably not the case that so many requests would be coming in simultaneously (or there would be multiple server instances to handle such requests), and the response size is very small. Since both servers are basically just writing to a stream, as long as the I/O is written with performance in mind, this may be roughly as fast as a my CPU can handle writing responses. Another hypothesis is perhaps that vert.x really shines in handling multiple instances. I’ll have to experiment and report my results.

    Postscript: In case you want to try it out for yourself, I’ve made the source code available on my github @ https://github.com/rocketegg/Code-Projects/tree/master/ServerPerformance I know this test has been done with much more sophistication and by a lot of people out there, but hopefully you can play around with this webapp, have fun and learn something.

    Custom Theming with Sencha themebuilder: Sencha GXT Theme Builder HTML

    In Sencha’s GXT 3.1 beta, it includes an html file that runs a script that can parse a theme file and generate the approximate look and appearance of several sencha widgets. This allows for fast iteration of .theme development (and enables you to not have to rebuild the theme.jar file with every change). It definitely is extremely valuable in terms of saving time, however, the tool is far from perfect. It’s located in ../gxt-3.1.0-beta-20140225/themebuilder/bin/widgetdatasample/sliceme/index.html. You’ll have to extract the files from the .jar first.

    Screen Shot 2014-02-28 at 10.56.52 AM

    An example rendering using the html form included with the themebuilder.
    An example rendering using the html form included with the themebuilder.

    A few of the deficiencies:

    • It didn’t work in chrome
    • Not every widget gets rendered
    • You can’t see widgets within widgets
    • You’ll need to modify the reset.css if you want to see custom styling
    • A lot of the widgets don’t have text, so you can’t see what text would look like in them
    • It’s difficult to determine what the widgets map to since they are prefaced with the name of the job that the themebuilder uses to “slice” the look into images for cross-browser support for older browsers.

    Here’s a few tips and tricks I’ve learned:

    • It didn’t work in chrome
      • Solution: It did work in firefox for me.
    • Not every widget gets rendered
      • Solution: Not much you can do here. Hopefully you can get to a point where your .theme is close enough so you can narrow down the widgets that you need to see to fix.
    • You can’t see widgets within widgets
      • Solution: Same as the above.
    • You can’t add custom styling because the .theme file doesn’t support it. If your app uses custom styles on top of the theme.jar, you won’t be able to see them.
      • Solution: There is a reset.css file in the same directory as the HTML file that is used to help style the rendered page. You can add .css classes here and using debug tools or firebug, insert your css class into the widget to see how it would look. You can do the same thing with external fonts.
      • See an example here:

        An example of modifying inline styles to get the look and feel right with compiled widgets.
        An example of modifying inline styles to get the look and feel right with compiled widgets.
    • A lot of the widgets don’t have text, so you can’t see what text would look like in theme
      • Solution: Same as the above. Except modify the inline HTML and add some text inside the innermost div.
      • An example of modifying the inline text of the widget using firebug.
        An example of modifying the inline text of the widget using firebug.
    • It’s difficult to determine what the widgets map to since they are prefaced with the name of the job that the themebuilder uses to “slice” the look into images for cross-browser support for older browsers.
      • Solution: This is kind of guess and check, but here’s a list of some of the less obvious mappings from the .theme file to what they correspond to in terms of gwt widgets:
      • In your .theme file,
      • 
            /* Dialog boxes - background
             */
            window {
              backgroundColor = theme.bgColor
              borderColor = util.darkenColor(backgroundColor, 0.2)
              headerGradient = util.gradientString(theme.headerGradientDetails)
            }
        
           /*
            * These are the overlays that disable fields.  Since they're grey, change the font to white.
            */
            mask {
              backgroundColor = "#000000"
              opacity = 0.5
              box {
                backgroundColor = "#303030"
                borderColor = "#555555"
                borderRadius = 0
                borderStyle = "solid"
                borderWidth = 2
                loadingImagePosition = "0 center"
                padding = util.padding(5)
                radiusMinusBorderWidth = util.max(0, borderRadius - borderWidth)
                text = theme.riverbedBodyWhite
                textPadding = util.padding(0, 0, 0, 21)
              }
            }
        
            //Dropdowns menus 
            fieldset {
              ...
              border=util.border('solid', '#bbbbbb', 1)
              text=theme.text  //Color of the unselected values in a dropdown menu
            }
        
            field {
                text=theme.text  //Color of the selected value in a dropdown in a toolbar
            }
        
        tabs {
              ...
              lastStopColor="#000000"  //color of the bottom of the tab
              tabHeight=20 //height of a tab
              tabSpacing=1 //horizontal spacing between the tabs
        
            }
        

    GWT/GXT Theming Part 3 – Adding Custom Fonts

    Okay, a softball for today. In order to add custom fonts to your application on top of Sencha’s themebuilder, you can take a few different approaches.

    Yay different fonts!
    Yay different fonts!
    1. You can add the font type to an external css file and pass in the font-family into your .theme file
    2. You can add custom appearance classes for each widget to get a particular font. See here.

    I ultimately settled on the first approach because it saved me the time of having to create a separate appearance class for each widget I wanted to style. That and Sencha’s .theme file actually has a “text” value for most of its widgets. It’s probably one of the most customizable aspects of the themebuilder.

    Here’s how I did it:

    theme {
    
      name = "customTheme"
      basePackage = "com.example"
    
      ...
    
      // CUSTOM FONTS
      /* TRADE GOTHIC/ARIAL */
        customSmallFont = util.fontStyle("\'TradeGothicW01-BoldCn20 675334\', Arial, Helvetica, sans-serif", "13px", "#000000", "normal")
    }
    

    You can see here that I escaped the single quotes for a font-family that contained spaces. This font string gets compiled through and actually makes it out to the ultimate .css that styles various widgets. Prior to using single escaped quotes, I just entered in the entire string in double quotes, which compiled and built a theme, but did not make it through to the .css in the end.

    If your font doesn’t have spaces, you can simply use the entire string, but escaping with single quotes seems to work well. Here’s what my external .css file looks like at the end:

    /* my external css file */
    @font-face {
        font-family: 'TradeGothicW01-BoldCn20 675334';
        src: url('fonts/tradegothicltpro-bdcn20-webfont.eot');
        src: url('fonts/tradegothicltpro-bdcn20-webfont.eot?#iefix') format('embedded-opentype'),
             url('fonts/tradegothicltpro-bdcn20-webfont.woff') format('woff'),
             url('fonts/tradegothicltpro-bdcn20-webfont.ttf') format('truetype'),
             url('fonts/tradegothicltpro-bdcn20-webfont.svg#TradeGothicW01-BoldCn20 675334') format('svg');
        font-weight: normal;
        font-style: normal;
    
    }
    
    

    That’s all there is to it!

    GWT/GXT Theming Part 2 – Adding Custom Appearance Classes on top of Sencha Themebuilder

    Likely one of the first things you’ll encounter if using the themebuilder is how to create a custom style for something the themebuilder doesn’t support. In my adventures trying to build on top of Sencha’s themebuilder, it became apparent to me that they intended the tool to be used in a standalone fashion. That is, one runs the themer.sh or themer.bat file to generate the theme.jar file and you plug and play it in conjunction with GWT.

    However, because the theme file is applied uniformly across your application, what happens if you don’t want buttons in one place to look like buttons in another? This came up when building our webapp, where we wanted our top-level nav to have one color and appearance on hover and other buttons within the application to have another color. As it turns out, there’s no way to specify this in the .theme file.

  • One set of buttons (rendered by the themebuilder)
  • Screen Shot 2014-02-27 at 2.25.46 PM

  • Another set of buttons (custom appearance)
  • Screen Shot 2014-02-27 at 2.29.29 PM

  • Yet another set of buttons (customer appearance)
  • Screen Shot 2014-02-27 at 2.30.20 PM

    So how did I do this? In fact, there isn’t an easy way currently. I had to go digging through the compiled appearance classes made by the themebuilder.jar file in search of an answer. Here’s an example of what the themebuilder spits out for a button made with your custom .theme file.

    Step 1: Create an custom appearance class for your button/widget that extends Sencha’s appearance class

    So the first step I found to creating custom styling on top of the themebuilder was to create my own appearance class. However, I didn’t want to start from scratch because Sencha already provides a good basis (e.g. by applying the fonts, default colors, creating sliced image files, etc). It’s true that a lot of these things go out of the window when creating your own appearance class and you can lose multi-browser support, but in our case, it was better than keeping all the buttons looking exactly the same.

    Here’s example of a custom appearance class for the dark grey buttons you see above.

    /**
     * HeaderButtonAppearance extends a theme appearance created by Sencha's
     * themebuilder.jar file.  It uses the gifs and base css created by Sencha's
     * themebuilder then adds additional css styling and modifies the rendering
     * logic to display header buttons.  
     */
    public class HeaderButtonAppearance<M> extends ButtonCellDefaultAppearance<M> {
    
      	final HeaderButtonStyle customStyle;
    	
    	/**
    	 * These are "resources" that comprise a ButtonCellAppearance
    	 * This interface describes the resources used by this button appearance.  This is not 
    	 * Sencha-specific.  I had originally tried extending the Css3ButtonResources class that 
             * the themebuilder spits out, but it required keeping the entire .css class in my package as well.
    	 */
      	public interface HeaderButtonResources extends ClientBundle {
      		@Source("HeaderButtonCell.css")
      		HeaderButtonStyle style();
      	}
      	
            //These methods are my new styles.
      	public interface HeaderButtonStyle extends CssResource {
      		String headerButton();
      		
      		String outerButton();
      	}
      	
      	public HeaderButtonAppearance() {
      		HeaderButtonResources headerResources = GWT.create(HeaderButtonResources.class);
      		customStyle = headerResources.style();
      		StyleInjectorHelper.ensureInjected(headerResources.style(), true);
      	}
      	
      	@Override
        public void render(ButtonCell<M> cell, Context context, M value, SafeHtmlBuilder sb) {
      	    Css3ButtonResources resources = GWT.create(Css3ButtonResources.class);
      	    Css3ButtonStyle style = resources.style();
                //Additional rendering stuff here, use style.____() to access the custom themebuilder styles
                //Sencha by default uses a two nested divs to render a button.  You may have to extract the precise
                //.class file from the theme.jar file to see how the html is constructed.
    
       }
    }
    
      There are a few quirks regarding this appearance class to be aware of:

    • In the *ButtonStyle interface, the string method maps to a css class name in the *ButtonResources interface css file.
    • The .css file that is used for the mapping must be in the same package as the appearance class
    • In order to use the styles made available through the compiled css and appearance classes built by the themebuilder, you have to instantiate a ClientBundle and access the css classes through the Css3ButtonStyle interface.
    • Because the default rendering of the button renders the themebuilder’s classes (e.g. for buttons, there are two nested divs, where the outer has .button {} applied and the inner has .buttonInner {} applied, I had to rewrite the render method myself. In my replacement, I swapped out the .button and .buttonInner classes for my own css classes. I maintain everything else.
    • I think there has to be a better way to do this. For example, I experimented with injecting html (an innermost div) into the cell which accepts only my css. That way, I could avoid having to @Override the render class and modify the html on the backend. However, it didn’t end up working very well because even with injecting my own css, I could not overwrite the styling applied to the parent divs. Ultimately, I think one could write a utility to go in and scrape out the other classes applied by the themebuilder, but this seemed like it would be a real hassle.

    Step 2: Define the .css classes in a file in the same package for your custom appearance

    Here’s an example of what my .css class looked like.

    .outerButton {
      border: none;
      background-color: #3892D3;
      background: transparent;
      padding: 4px;
      cursor: pointer;
      outline: none;
      padding-right:30px;
    }
    
    .headerButton {
      	padding: 0px;
      	text-align: left;
    	font-family: Arial, Helvetica, sans-serif;  /* actual font is not arial */
    	font-size: 17px;
    }
    
    .headerButton:hover {
    	color: white;
    }
    

    If you look at my HeaderButtonStyle class, you’ll notice that the css class names map directly to the functions in the interface. GWT takes care of the actual mapping of values. These are the classes that I can use to style my widget and are accessible via my instance of the HeaderButtonStyle. Here’s an example of how I render the HeaderStyleButton:

    
    @Override
        public void render(ButtonCell<M> cell, Context context, M value, SafeHtmlBuilder sb) {
                //The default style CSS Resource in case I need to access styles built by the themebuilder
    
      	    Css3ButtonResources resources = GWT.create(Css3ButtonResources.class);
      	    Css3ButtonStyle style = resources.style();
    
                //Begin actual construction of the button widget
      		
      	    String constantHtml = cell.getHTML();
      	    boolean hasConstantHtml = constantHtml != null && constantHtml.length() != 0;
    
      	    boolean isBoolean = value != null && value instanceof Boolean;
    
      	    String text = hasConstantHtml ? cell.getText() : (value != null && !isBoolean)
      	        ? SafeHtmlUtils.htmlEscape(value.toString()) : "";
    
      	    String scaleClass = "";
      	    String iconClass = "";
      	    String additionalCssClasses = customStyle.headerButton();
    
                //Size-dependent styling
      	    ButtonScale scale = cell.getScale();
    
      	    switch (scale) {
      	      case SMALL:
      	        scaleClass = style.small();
      	        break;
    
      	      case MEDIUM:
      	        scaleClass = style.medium();
      	        break;
    
      	      case LARGE:
      	        scaleClass = style.large();
      	        break;
      	    }
    
                //Outer button class, you could add anything else you want here
      	    String buttonClass = customStyle.outerButton();
    
      	    boolean hasText = text != null && !text.equals("");
      	    if (!hasText) {
      	      buttonClass += " " + style.noText();
      	    }
     
                //The css classes applied to the inner div
                String innerClass = "";
      	    innerClass += " " + scaleClass;
      	    innerClass += " " + additionalCssClasses;
    
      	    SafeHtmlBuilder builder = new SafeHtmlBuilder();
    
      	    builder.appendHtmlConstant("<div class='" + buttonClass + "'>");
      	    builder.appendHtmlConstant("<div class='" + innerClass + "'>");
    
      	    builder.appendHtmlConstant(text);
      	    builder.appendHtmlConstant("</div></div>");
      	    sb.append(builder.toSafeHtml());
      	  }
    
    

    Step 3: Instantiate your widget using your custom appearance.

    TextButton button = new TextButton(new TextButtonCell(
            			new HeaderButtonAppearance<String>()), "Text for Button");
    

    You can see that when I instantiate a button, I pass in an appearance class instantiated with the type of value that the button receives (a string in this case). If you look at the prebuilt java classes (e.g. Css3ButtonCellAppearance) from the themebuilder, you’ll notice that the default constructor with no parameters will instantiate a cell with the default appearance. That’s why it’s necessary to use the constructor where you pass in the actual custom appearance class.