rdfQuery 1.0 released

Nadeem | | Sunday, July 19th, 2009

I travelled down to Oxford last weekend to attend the rdfQuery Dazzle event. Fairly early on we decided that one of the things we wanted to achieve was to get v1.0 of rdfQuery released. This involved a fair bit work, not only did we need to get unit tests working across all the major browsers, but we also wanted to fix some of the existing bugs and get documentation written. I spent the bulk of the weekend restructuring the unit test suite so the 1100 unit tests we have could be run together. I also introduced an ant script to make it easy to generate documentation, as well as create distributions. Jeni and Kal managed to get all the api documentation written, and got rdfQuery working in IE7, IE8, Firefox and Safari. It was a great effort by everyone involved.

We also spent time talking about some of the Ontology support that Jeni is thinking of adding to rdfQuery, which could be very useful. We also wanted to get some closer integration with the Talis Platform, so I’ve also been working on adding Changeset support. All in all it was a great weekend.

You can download and learn more about rdfQuery here.

We’re hiring …

Nadeem | | Sunday, July 19th, 2009

We are currently recruiting for a number of different positions at Talis, amongst these are several openings to join our development teams. Rob has already discussed the Web Application Technical Lead role, and I’ll like to mention that we are also looking for Senior Developers to join both our Platform and Education divisions.

Senior Software Developer Education

It is the vision of our Education Division to connect faculties, students and educators together using technology, with the aim of creating joined up learning environments and providing seamless access to education resources and pedagogical expertise. We are currently working on delivering Talis Aspire into a number of Higher Education institutions here in the UK and abroad. Aspire is built entirely upon our semantic web platform, and is one of the few truly native linked data applications. Whilst the underlying technology is important, we have to balance this with an excellent user experience. If you are interested in being an early part of publishing large amounts of data on the semantic web? And building truly compelling software that provides an excellent user experience for millions of users, then you might be interested in applying. The job spec provides more details, we only ask that when applying you try to answer two of three questions below:

  1. Discuss the different types of automated testing that are needed to maintain high quality
    software. What kinds of programming language are best suited to each type of testing? What
    automated techniques could be used to test web based applications and user interfaces? And
    how can code design and refactoring be affected by choice of language?
  2. When working with data that you do not own, there are no guarantees about the cardinality of
    fields or the presence of data you might want to consider mandatory. Traditional approaches to
    working with data from elsewhere have relied on cleaning, validating and then importing data
    into an application’s own database. The Semantic Web allows data to be shared at runtime.
    What techniques or strategies could be employed within an application to handle unreliable or
    unexpected data when sharing databases with other applications at runtime?
  3. Web applications are often composed of multiple interoperating systems, connected by APIs or
    other endpoints, and deployed across multiple environments. As usage patterns change, the
    application may need to scale rapidly, whilst maintaining performance and reliability levels.
    How can applications be designed to allow for such scaling?

Senior Platform Developer

Our Platform division is always looking for experienced developers to join the team. The job spec for this role provides far more details but when applying we ask that you try and answer at least two of the following three questions:

  1. The Web can be modelled as a network of nodes labelled with URLs and connected by directed arcs. Suppose we want to find all the URLs linked to and from any given URL, and all the URLs that are linked from any two given URLs. What kind of data structures might be suitable for representing and querying a network with 10^8 nodes each having between 10 and 50 arcs?
  2. Discuss the different types of automated testing that are needed to maintain high quality software. What kinds of programming language are best suited to each type of testing? What techniques could be used for testing asynchronous processes and for processes that operate over large volumes of data? Are there any situations that you wouldn’t test?
  3. Large-scale systems composed of many cooperating application servers often need to share and cache configuration. Suppose any server can initiate changes that need to be reflected in real time to the other application servers in the cluster. What strategies could you use for coordinating this kind of behaviour and how are they tolerant to various failure conditions?

Finally I think Rob summed it up best when he said: All in all though, we’re looking for great people to come and help us do great stuff. Get in touch!

Seth Godin on the tribes we lead

Nadeem | | Tuesday, May 19th, 2009

Seth Godin argues the Internet has ended mass marketing and revived a human social unit from the distant past: tribes. Founded on shared ideas and values, tribes give ordinary people the power to lead and make big change. He urges us to do so.

This is a great talk by Seth Godin, here’s how he describes Tribes …

What tribes are, is a very simple concept that goes back 50 thousand years. It’s about leading and connecting people and ideas. And it’s something that people have wanted forever. Lots of people are used to having a spiritual tribe, or a church tribe, having a work tribe, having a community tribe. But now, thanks to the internet, thanks to the explosion of mass media, thanks to a lot of other things that are bubbling through our society around the world, tribes are everywhere.

The internet was supposed to homogenize everyone by connecting us all. Instead what it’s allowed is silos of interest. So you’ve got the red-hat ladies over here. You’ve got the red-hat triathletes over there. You’ve got the organized armies over here. You’ve got the disorganized rebels over here. You’ve got people in white hats making food. And people in white hats sailing boats. The point is that you can find Ukrainian folk dancers. And connect with them … You can tell when you’re running into someone in a tribe. And it turns out that it’s tribes, not money, not factories, that can change our world, that can change politics, that can align large numbers of people. Not because you force them to do something against their will. But because they wanted to connect … That what we do for a living now, all of us, I think, is find something worth changing, and then assemble tribes that assemble tribes that spread the idea and spread the idea. And it becomes something far bigger than ourselves.

The talk resonates deeply with me at the moment particularly given that I’m currently thinking long and hard about what leadership actually means, how to build teams, what to look for when recruiting people, how to ensure everyone feels that they are empowered and that their contributions are valued. However the more I think about it the more I’m beginning to believe that whilst its possible to create an environment in which people can have the freedom to affect change it ultimately requires the individual to first believe in what they are doing, to have committed to it and to the people around them. They will then want to move things forward, improve things, affect real change and want to break the status quo. Seth crystallizes this sentiment beautifully in this talk when he says …

What all these people have in common is that they are heretics. That heretics look at the status quo and say, This will not stand. I can’t abide this status quo. I am willing to stand up and be counted and move things forward. I see what the status quo is. I don’t like it. That instead of looking at all the little rules and following each one of them, that instead of being what I call a sheepwalker, somebody who’s half asleep, following instructions, keeping their head down, fitting in, every once in a while someone stands up and says, “Not me.” Someone stands up and says, “This one is important. We need to organize around it.” And not everyone will. But you don’t need everyone. You just need a few people (Laughter) who will look at the rules, realize they make no sense, and realize how much they want to be connected.



You don’t need permission from people to lead them. But in case you do, here it is. They’re waiting, we’re waiting for you to show us where to go next. So here is what leaders have in common. The first thing is, they challenge the status quo. They challenge what’s currently there. The second thing is, they build a culture. A secret language, a seven second handshake. A way of knowing that you’re in or out. They have curiosity. Curiosity about people in the tribe. Curiosity about outsiders. They’re asking questions. They connect people to one another. Do you know what people want more than anything? They want to be missed. They want to be missed the day they don’t show up. They want to be missed when they’re gone. And tribe leaders can do that. It’s fascinating because all tribe leaders have charisma. But you don’t need charisma to become a leader. Being a leader gives you charisma. If you look and study the leaders who have succeeded, that’s where charisma comes from, from the leading. Finally, they commit. They commit to the cause. They commit to the tribe. They commit to the people who are there.

This is a great, passionate and deeply profound talk and I recommend everyone take the time to watch it.

2009 an update …

Nadeem | | Sunday, February 22nd, 2009

During a conversation with a certain cat loving friend of mine earlier on in the week it was suggested to me that its been a while since I’ve blogged anything other than short pieces highlighting bits of news or content out there on the web. She’s absolutely right. So here’s an update on 2009 so far, and what the next few months hold …

The last three or four months have been particularly busy for me. I took a long holiday before Christmas in order to give myself time to reflect on everything that happened in 2008. As far as year’s go it pretty much sucked! I had a lot of personal stuff to deal with most of it around coping, or failing to cope, with the deaths of a number of people who were close to me – including my father. I’m pretty good at burying myself in work as a way of not having to deal with other things unfortunately that only works for so long, in fact I’m surprised I actually got to the end of the year before finally accepting that things were broken inside. I’m lucky though, I have a lot people around me who keep an eye on me, and care enough to give me a kick when I need it – and they did. I was convinced to switch off from work and everything related to work and focus on dealing with the things that I knew I needed to.My hiatus over christmas was spent with my family trying to understand everything that had happened last year which inevitably meant finally accepting that I needed to grieve.

I sometimes hold it half a sin
To put in words the grief I feel;
For words, like Nature, half reveal
And half conceal the Soul within.

     from In Memorium, Alfred Lord Tennyson

So I wanted to firmly place the events of 2008 in the past and move forwards again. 2008 was painful and difficult yet I also enjoyed a number of personal and professional successes. In 2009 I want build on those successes, and leave the past firmly where it is.

In terms of my personal life there are already some big changes I’m in making but I’ll leave discussing that for another day, suffice to say that I think I’m happier now than I have been in years :).

Professionally I was appointed Head of Development for our Xiphos Division at Talis. I’m still trying to settle into the role which brings its own challenges :) . However leading up to christmas our division had successfully entered into piloting a new product called Aspire at Plymouth University and since then it’s also been deployed as part of a wider pilot at Sussex University. Functionally Aspire is a resource lists product that helps lecturers and students make best use of the educational material for their courses. Technically Aspire is a Linked Data application built directly on top of our Talis Platform, a platform that provides the infrastructure for building Semantic Web applications. I’m loving the work, its technically very challenging there’s so many different things that need to be considered. Lots of people and organisations are talking about the semantic web but there’s only a relatively small number of organisations that are actually building real world products and solutions using these technologies – products and solutions that are actually targeted at end users – for me this is primarily why the work is both exciting and hugely rewarding.

Building Aspire is forcing us to innovate and explore ideas and possibilities that we might not have otherwise considered. A case in point is the way in which we have embedded RDFa into the our list page and our editing tool manipulates this model directly within the HTML DOM simplifying the process. This is discussed in a W3C Case Study, and was commented upon by Ivan Hermann last month. Much of the work we are doing at the moment is around adding more features to Aspire during our beta phase. Whilst part of this will be around specific features aimed at users, we are also looking at linking to other data sets and exploring what we else can do within this ecosystem of rich semantic data.

Finally, I mentioned on several occasions last year about work I was doing in my own time around building a tool that aided in visualising and exploring the socio-semantic web. That work got shelved towards the end of 2008 largely because I couldn’t focus on it with everything else that was going on. However a new year, a new beginning means that project now has a new lease of life … and it finally has a name: Omnius. It’s named after a thinking machine from the Legends of Dune series which I was re-reading around the time I was thinking about a name for this project. I had actually wanted to call it Erasmus but that name had already been taken on google code ( and erasmus_browser sounded sucky! ). I’ve created home for this project on google code, I’ll be adding more information very soon. However please remember I am only working on this during my spare time, for me its both a hobby and an interesting technical (and UX) diversion … it hasn’t yet turned into an obsession so the rate at which I’ll be adding to it is limited in terms of the time I’m able to devote to it :)

… so watch this space.

Do what you love …

Nadeem | | Saturday, October 4th, 2008

Rob wrote a wonderful piece earlier this week on Passion, personal brand and doing what you love. We’ve both spent time reflecting, individually, on how our paths led us to Talis and the work we are now doing. We’ve also discussed this subject on a number of occasions and I think we’ve both reached the same conclusion – life is too short to waste it away doing something you dont love. It takes time to come to a realisation like this, and sadly, it often require some external event to force you to stop and reflect on your life, how you got to where you are and most importantly where is it your heading. I know some people describe this as a kind of awakening, and I guess if you honestly believe that life is a journey then it’s moments such as these that can feel like a fork in the road…

One path leads back to the world that you know, its the world that you’ve become used to, it’s the world in which you don’t have to love what you do to get by, it feels comfortable because there is no risk, you don’t have to deal with the unknown too often, you don’t have to rock the boat. In many ways you’ve already reached a destination … or is it more true to say that you’ve reached an empasse?

The other path leads to somewhere else, unfortunately you don’t know what’s down there. You don’t necessarily even know where you’re going. Here’s the thing though … maybe you don’t have to care about the destination, maybe it’s less about the where, and much more about the how?

I guess that’s how I see things these days. There’s a part of me that believes that if do what I love, then it won’t feel like my life is just slipping away, one monotonous day to the next. I lived my life like that once, it’s so easy to do, you become so used to it that you don’t even realise that something is wrong … I actually had to come to within a heartbeat of losing my life before I realised that life is far to precious a gift to waste like that. To spend so much of it doing something that I felt completely indifferent towards, even hated at times … Isn’t it fascinating how the the worst prisons are the ones we create for ourselves?

There’s two quotes I want to end with, the first is from Paul Graham’s essay entitled How to do what you love, I remember reading it a couple of years ago, and although I don’t agree with some of it, it’s still a wonderful piece. His essay ends with this obeservation:

Finding work you love is very difficult. Most people fail. Even if you succeed, it’s rare to be free to work on what you want till your thirties or forties. But if you have the destination in sight you’ll be more likely to arrive at it. If you know you can love work, you’re in the home stretch, and if you know what work you love, you’re practically there.

The second quote is much shorter, but far more profound, you see for me loving what I do is a part of a much greater truth, one that underpins everything I’ve said, and I think almost everything I now hold dear:

There is life in every breath

Test-Driven JavaScript Development with JsUnit

Nadeem | | Saturday, September 20th, 2008

The last time I used JsUnit was when I first joined Talis. At the time my colleague Ian Davis asked me to write a JavaScript client library for one of our platform API’s to make it easy for developers to perform bibliographic searches. It wasn’t a particularly difficult task and I did it relatively easily. It was around the same time that Rob was extolling the virtues of Test Driven Development to me, and to try to prove his point we agreed to do an experiment: he asked me to set aside the library I had written and to see if I could develop the library again using test driven development. It meant I had to figure out how to unit test JavaScript, and thats when I found JsUnit. I did the exercise again and even I was impressed with the results. By having to think about the tests first, and design the interface to the library as I wrote each test it evolved very differently to my original solution. Consequently it was also far superior.

Anyway fast forward two and half years and I find myself in a similar situation. We have only just begun to start writing bits of JavaScript code based around prototype.js to help us create richer user experiences in our products if we detect that JavaScript is enabled in the browser. This now means I want to ensure that we are using the same rigour when writing these bits of code as we do in all other parts of the application – just because its JavaScript and executed inside the browser this doesn’t mean it shouldn’t be tested.

I’ve just spent the morning getting JsUnit installed and figuring out how to get it to run as part of a continuous integration process, as well as thinking about how to write tests for some slightly different scenarios. Here’s what I’ve discovered today:

Installing JsUnit

Couldn’t be easier … go to www.jsunit.net and download the latest distribution, and extract into a folder on your system somewhere, lets say
/jsunit for now. The distribution contains both the standard test runner as well as jsunit server which you will need it if you want to hook it into an ant build.

Writing Tests

In JsUnit we place our tests in a HTML Test Page which is the equivalent of a Test Class, this test page must have a script reference to the jsUnitCore.js so the test runner knows its a test. So lets work through a simple example. Let’s say we want to write a function that returns the result of adding two parameters together. The Test Page for this might look like this:

  1.  
  2. <html>
  3.  <head>
  4.   <title>Test Page for add(value1, value2)</title>
  5.   <script language="javascript" src="/jsunit/app/jsUnitCore.js"></script>
  6.   <script language="javascript" src="scripts/addValues.js"></script>
  7.  </head>
  8.  <body>
  9.     <script language="javascript">
  10.     function testAddWithTwoValidArguments() {
  11.         assertEquals("2 add 3 is 5", 5, add(2,3) );
  12.     }
  13.   </script>
  14.  </body>
  15. </html>
  16.  

For now lets save this file to /my-jsunit-tests/addTest.html

To run the test you need to point your browser at the following local url:

file:///jsunit/testRunner.html?testPage=/my-jsunit-tests/addTest.html

The test will not run since we haven’t defined the add function. Let’s do that (very crudely):

  1.  
  2. function add(param1, param2)
  3. {
  4.     return param1 + param2;
  5. }
  6.  

Now if you go to that URL it will run the test and report that it passed. Excellent, we’ve written a simple test in JavaScript. Now lets extend this a little, lets say I want to write something more complicated like a piece of JavaScript that uses Prototype.js to update the DOM of a page. Is this possible? Can I do that test first? It turns out that you can …

Lets say we have a div on the page called ‘tableOfContents’ and we want to use Prototype.js to dynamically inject a link onto the page that says [show] and lets say we want to write a function that will toggle this link to say [hide] when the user clicks on it, this link will also set the visible state of the table of contents itself which for now we’ll say is just an ordered list (OL). Our test page is going to be slightly more complex …

  1.  
  2. <html>
  3.  <head>
  4.   <title>Test Page for multiplyAndAddFive(value1, value2)</title>
  5.   <script language="javascript" src="/jsunit/app/jsUnitCore.js"></script>
  6.   <script language="javascript" src="scripts/prototype/prototype-1.6.0.2.js"></script>
  7.   <script language="javascript" src="scripts/tableOfContents.js"></script>
  8.  </head>
  9.  <body>
  10.     <div id="tableOfContents">
  11.     <h2 id="tableOfContentsHeader">Table of contents</h2>
  12.     <ol id="list-toc">
  13.     </ol>
  14.     </div>    
  15.     <script language="javascript">
  16.     function testTOC()
  17.     {
  18.         var title = $(‘lnkToggleTOC’).title;
  19.         assertEquals("should be Show the table of contents", "Show the table of contents", title);
  20.        
  21.         toggleTableOfContents();
  22.        
  23.         var title = $(‘lnkToggleTOC’).title;
  24.         assertEquals("should be Hide the table of contents", "Hide the table of contents", title);
  25.                    
  26.     }
  27.   </script>
  28.  </body>
  29. </html>
  30.  

There are some differences in this test. Firstly the html contains some markup, that I’m using as the containers for my table of contents. The table of contents has a header and the contents in the form of an empty ordered list. Now I know that I want the javascript to execute when the page is loaded, so I’ve written this test to assume that the script will run and will inject and element called ‘linkToggleToc’ which is the show/hide link next to the heading. Therefore the first line of the test uses prototype.js element selector notation to set a local variable called title to the value of the title of the element that has the id ‘linkToggleToc’. If the script failes to execute then this element will not be present and the subsequent assert will fail. If the assert succeeds, then we call the toggleTableOfContents function and then repeat the same evaluation only now we are checking to see if the link has been changed.

The code for tableOfContents.js is as follows:

  1. Event.observe(window, ‘load’, injectTOCLinks);
  2.  
  3. var titleShowTOC = ‘Show the table of contents’;
  4. var titleHideTOC = ‘Hide the table of contents’;
  5.  
  6. function injectTOCLinks()
  7. {
  8.     $(‘list-toc’).hide();
  9.     $(‘tableOfContentsHeader’).style.display = ‘inline’;
  10.    
  11.        
  12.     var elShowHideLink = new Element(‘a’, { ‘id’: ‘lnkToggleTOC’, ‘href’: ‘javascript:toggleTableOfContents()’, ‘title’: titleShowTOC, ‘class’: ‘ctr’ }).update("[show]");
  13.    
  14.     $(‘tableOfContentsHeader’).insert({‘after’:elShowHideLink});
  15. }
  16.  
  17. function toggleTableOfContents()
  18. {
  19.     var toc = $(‘list-toc’);
  20.    
  21.     if (toc.visible())
  22.     {
  23.         toc.hide();
  24.         $(‘lnkToggleTOC’).update(‘[show]‘);
  25.         $(‘lnkToggleTOC’).title = titleShowTOC;
  26.     }
  27.     else
  28.     {
  29.         toc.show();
  30.         $(‘lnkToggleTOC’).update(‘[hide]‘);
  31.         $(‘lnkToggleTOC’).title = titleHideTOC;
  32.     }
  33. }

Now if we run this test in the same way we executed the previous test it will pass. I accept that this example is a bit contrived since I know it already works and I’ve skimmed over some of the details around it. The point I’m trying to make though is that you can write unit tests for pretty much any kind of JavaScript you need to write, even tests for scripts that do dom manipulation, or make AjaxRequests etc.

Setting up the JsUnit server so you can run it in a build

JsUnit ships with its own ant build file that requires some additional configuration before you can run the server. The top of the build file contains a number of properties that need to be set, here’s what you set them to ( using the paths that I’ve been using in the above example)

  1.  
  2. <project name="JsUnit" default="create_distribution" basedir=".">
  3.  
  4.   <property
  5.     name="browserFileNames"
  6.     value="/usr/bin/firefox-2" />
  7.  
  8.   <property
  9.     id="closeBrowsersAfterTestRuns"
  10.     name="closeBrowsersAfterTestRuns"
  11.     value="false" />
  12.  
  13.   <property
  14.     id="ignoreUnresponsiveRemoteMachines"
  15.     name="ignoreUnresponsiveRemoteMachines"
  16.     value="true" />
  17.  
  18.   <property
  19.     id="logsDirectory"
  20.     name="logsDirectory"
  21.     value="/my-jsunit-tests/results/" />
  22.  
  23.   <property
  24.     id="port"
  25.     name="port"
  26.     value="9001"  />
  27.  
  28.   <property
  29.     id="remoteMachineURLs"
  30.     name="remoteMachineURLs"
  31.     value="" />
  32.  
  33.   <property
  34.     id="timeoutSeconds"
  35.     name="timeoutSeconds"
  36.     value="60" />
  37.  
  38.   <property
  39.     id="url"
  40.     name="url"
  41.     value="file:///jsunit/testRunner.html?testPage=/my-jsunit-tests/tocTest.html" />
  42. </project>
  43.  

You can then type the following command in the root of the jsunit distribution to launch the jsunit server, executes the test, and outputs a test results log file, formatted just like JUnit, and reports that the build was either successful or not if the test fails.

  ant standalone_test

Remember that in this example I’ve used a simple Test Page, however JsUnit, like any XUnit framework allows you to specify Test Suites, which is how you would run multiple Test Pages. Also the parameters in the build file woudn’t be hardcoded in you continuous integration process but would rather be passed in, and you would want to call it from your projects main ant build file … all of which is pretty simple to configure, once you know what is you want to do and what’s possible.

Visual Interfaces to the Social and the Semantic Web (VISSW2009)

Nadeem | | Saturday, September 20th, 2008

I’ve recently been invited (and accepted) to join the Program Committee for the Visual Interfaces to the Social and the Semantic Web workshop which will be held as a part of the 2009 International Conference on Intelligent User Interfaces which takes place in February in Florida. Here’s a brief description of the workshop:

This workshop aims to bring together researchers and practitioners from different fields, such as Human-Computer Interaction, Information Visualization, Semantic Web, and Personal Information Management, to discuss latest research results and challenges in designing, implementing, and evaluating intelligent interfaces supporting access, navigation and publishing of different types of contents on the Social and Semantic Web. In addition, the workshop also serves as an opportunity for researchers to gain feedback on their work as well as to identify potential collaborations with their peers

I’m quite excited about this workshop, I’ve been spending some of 10% time at work on playing with ideas around how to visualise data on the Semantic Web just over eighteen months ago I was already using and extending Moritz Stefaner‘s excellent Relation Browser to visualise semantically structured data, and was effectively walking around entire graphs of data:

However there were problems with this approach, performance over large sets of data was one, and I spent a considerable amount of time extending the tool. However it occurred to me that with Relation Browser and other similar tools you are forced to take a very node centric view, when sometimes whats more useful is to be able to take a graph or named graph centric view – its not only important to be able to center on one node and see it’s direct relationships it’s often just as important to understand where that node sits with a much wider context. To that end I’ve been experimenting with visualisations that allow you to not only center on a particular node and ‘follow your nose’ but to also pan out take a more hollistic view of that graph of data – I’ve also been thinking about ways to visualise the provenance of data.

I wish I had known about this workshop sooner, it might have been the kick I needed to actually finish demonstrators of these ideas, I think I might still have time though …

With reference to the workshop itself though here’s some important dates and other bits of information if you’re interested in submitting a paper:

  • Paper submission deadline: 14th November, 2008
  • Notication of acceptance: 7th December, 2008
  • Camera-ready paper submission deadline: 14th December, 2008
  • Full papers which should be between 6 and 10 pages.
  • Short papers and position papers which should be up to 5 pages.
  • Demo papers – 2 page description with a screenshot of the working prototype or preferrably a link to an online demo.
  • Submissions must be in PDF format and prepared according to the IUI format.

It promises to be an excellent conference, and one that I will definitely be attending.

CK-12 Next Generation Textbooks

Nadeem | | Sunday, September 14th, 2008

Came across CK-12 whilst looking for information on tools and approaches used to help learners achieve their goals. It also appears as though their textbooks are available under a Creative Commons Attribution Sharealike license, but I need to confirm this. It’s a really interesting project and one I’ll be watching closely:

Our mission is to reduce the cost of textbook materials for the K-12 market both in the US and worldwide, but also to empower teacher practitioners by generating or adapting content relevant to their local context. Using a collaborative and web-based compilation model that can manifest open resource content as an adaptive textbook, termed the “FlexBook”, CK-12 intends to pioneer the generation and distribution of high quality, locally and temporally relevant, educational web texts. The content generated by CK-12 and the CK-12 community will serve both as source material for a student’s learning and provide an adaptive environment that scaffolds the learner’s journey as he or she masters a standards-based body of knowledge, while allowing for passion-based learning

Also found the following Google talk that provides some useful background information from the creators of the project

Is LinkedData really more important than the Large Hadron Collider?

Nadeem | | Sunday, September 14th, 2008

I’ve just read Daniel‘s recent post entitled Linked Data is more important than the Large Hadron Collider. Like Daniel I am also a passionate advocate of Linked Data and am currently working on deploying number of real world Linked Data applications along with my colleagues at Talis. Sadly though I have to confess that I found myself cringing whilst reading his piece.

Like many other scientific endeavors the Large Hadron Collider project attempts to provide scientists with huge quantities of data that might help them answer questions about the origin of the universe. As a project in it’s own right it is massive, combining the efforts of thousands of scientists from around the world.

To dismiss it, as Daniel has done, because it’s “too expensive”, or because “it wont find the cure to cancer, or HIV”, or question its relevance because “we’re still going to be here whether or not the Large Hadron Collider was successful”, is bad enough but to then use those rather specious arguments as a prop to advocate Linked Data is absolutely ridiculous.

Worse is that it overlooks the rather obvious rebuttal which is that Linked Data wont cure cancer, it wont cure HIV, and we’ll all still be here whether we have Linked Data or not :-). Even more importantly though … should anyone in our Community and by that I mean the Linked Data community really be questioning the value of any project that’s sole purpose it generate data? To then say this …

Just imagine a world where you can easily browse through the history of the atom, and then delve into the science found on the atom, and then go deeper into the subatomic level, and then browse back out into the historic realm, finding out about experiments that happened and whether it had any impact on society.

… completely misses the following point: the data to do this exists, not because of you and I Daniel, but because of the fact that since man appeared on this planet his thirst for knowledge is what has driven him forward to the point where people like you and I can sit here and say … “if you format your data like this, and give everything a dereferncible uri – that’ll be really useful!”. I’m serious … Linked Data is not a radical technology change, nor is the Semantic Web, both represent a paradigm shift, a new understanding, a new way of doing things but the fact is that the technology has been around for ages, we are only now understanding the importance of being more open, of having common vocabularies to describe things, and the importance of linking concepts together in this web of data.

The absolute last thing we want to do is to start saying to scientists, not matter how obscure ther field of research is, or how relevant we consider that research to be (personally), that it’s somehow less important than what we, as a community, are doing … because it absolutely isn’t. Are you really sure you want to be asking people to believe that answers about the origin of the universe and our existence in it are less important than an “interesting browsing experience?”

To Smarty, or not to Smarty?

Nadeem | | Sunday, September 14th, 2008

At Talis we have been doing more than just our fair share of PHP development, in fact we are using the language to implement a number of new prototypes, products and services. One of the questions that I’ve been struggling with recently is whether or not we should be using the Smarty template engine. Whilst we have used it with a measure of success on several projects, I’ve been struggling with what Smarty represents. Smarty is written in PHP and allows you to create templates that are marked up using Smarty’s own language syntax however this means that generated pages are precompiled at runtime down to pure PHP before being served. What strikes me is that Smarty is a Template Engine written in a language that was designed primarily for templating. So the question I’m left struggling with is: Why do you want to add the overhead of a secondary language to do what your first language already does and is fully capable of doing?

The more I think about it, the more I’m convinced that the answer is that you dont! I’m going to try to explain why by examining some of the reasons that are often cited for using Smarty.

Separating PHP from HTML

I’ve often heard his mantra, but I would describe it in a slightly different way and that’s by saying that what’s actually desired is to keep keep your “business logic” separate from your “presentation logic”, which is subtly different. If you accept that the two are indeed different then you should realise that all that’s actually required is a way to keep your Views and Controllers separated which can be achieved using plain PHP in your view scripts (templates), without the overhead of having to add a new language into the mix.

Easier to read

I’ve always struggled with this and here’s why … does anyone really think that this

  1. {section name=aValue loop=$list}
  2. Value is: {$aValue}
  3. {/section}

…is really easier to read than its pure PHP equivelant? here …

  1. <? foreach ( $list as $aValue ) { ?>
  2. Value is: <?=$aValue?>
  3. <? } ?>

… I honestly don’t think that it is. I know that there’s a view in the community that there are front-end only Developers out there or Web Designers that shouldn’t have to learn PHP, but are they really better off being taught a different language? Firstly I’d question if there are actually Developers out there who know Smarty and don’t have an understanding of PHP, but for the purposes of this exercise lets assume that these mythical smarty only developers do exist then I have to confess I agree entirely with Harry Feuks[1] when he says

Walk them through a few sections of the PHP Language Reference
and show them how PHP‘s control structures and simple functions like
echo() works and that’s all they need.

and also with Brian Lozier[2] who describes it much more convincingly when he says:

Basically, it just provides an interface to PHP with new syntax. When 
stated like that, it seems sort of silly. Is it actually more simple to 
write {foreach --args} than ? If you do think it's
simpler, consider this. Is it so much simpler that there is value in 
including a huge template library to get that separation? Granted, 
Smarty offers many other great features (caching, for instance), but it 
seems like the same benefits could be gained without the huge 
overhead of including the Smarty class libraries.

Speed

I’ve read several articles online that try to explain that there is a performance overhead when including Smarty. Yes I know you can cache your Smarty templates to improve performance in fact it’s not a question of ‘can’ it’s a question of ‘you really have to’ as Harry points out in the same article

if you hadn’t chosen to write your own programming language
and built an interpreter with PHP to parse it, you wouldn’t have slowed
down your application in the first place!

I performed a little test to try and illustrate what happens when you simple include Smarty, let alone actually use it. First off create yourselves a file called test.php and add the following line of code to it:

  1. <?php echo="Hello world" ?>

Now let’s benchmark this using Apache Benchmark, thanks to Vivek for providing some useful documentation on how to do this[3]. Open up a terminal window and issue the following command:

  1. ab -c 10 -n 100 http://localhost/test.php

Which on my system ( mac osx 10.5.2 4gb ram php5+apache2 ) resulted in the following:

nadeemshabir$ ab -c 10 -n 100 http://localhost/test.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /test.php
Document Length:        11 bytes

Concurrency Level:      10
Time taken for tests:   0.58753 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      19695 bytes
HTML transferred:       1111 bytes
Requests per second:    1702.04 [#/sec] (mean)
Time per request:       5.875 [ms] (mean)
Time per request:       0.588 [ms] (mean, across all concurrent requests)
Transfer rate:          323.39 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.5      0       2
Processing:     2    4   5.4      4      34
Waiting:        1    4   5.5      3      33
Total:          2    5   5.3      4      34

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      5
  80%      6
  90%      8
  95%     14
  98%     27
  99%     34
 100%     34 (longest request)
Nadeems-Computer:smarty-test nadeemshabir$

So the important figures are Request Per Second: 1702.04, Time per request: 5.875 [ms] (mean), Time per request: 0.588 [ms] (mean, across all concurrent requests) and Transfer rate: 323.39 [Kbytes/sec] received. Now let’s create a new test file called test-smarty.php, and add to it the following:

  1. <? include_once( ‘smarty/libs/Smarty.class.php’ ); ?>
  2. <?php echo "Hello world" ?>

and run the same test again …

  1. ab -c 10 -n 100 http://localhost/test-smarty.php

which results in the following:

nadeemshabir$ ab -c 10 -n 100 http://localhost/test-smarty.php
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient).....done


Server Software:        Apache/2.2.9
Server Hostname:        localhost
Server Port:            80

Document Path:          /test-smarty.php
Document Length:        11 bytes

Concurrency Level:      10
Time taken for tests:   0.310262 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      19500 bytes
HTML transferred:       1100 bytes
Requests per second:    322.31 [#/sec] (mean)
Time per request:       31.026 [ms] (mean)
Time per request:       3.103 [ms] (mean, across all concurrent requests)
Transfer rate:          61.24 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2   4.7      0      17
Processing:     9   26  14.3     22      84
Waiting:        8   25  14.1     22      82
Total:         13   28  14.7     24      87

Percentage of the requests served within a certain time (ms)
  50%     24
  66%     28
  75%     35
  80%     37
  90%     46
  95%     70
  98%     84
  99%     87
 100%     87 (longest request)
Nadeems-Computer:smarty-test nadeemshabir$

Now we can see a dramatic difference those important figures are now – Request Per Second: 322.31, Time per request: 31.026 [ms] (mean), Time per request: 3.103 [ms] (mean, across all concurrent requests) and Transfer rate: 61.24[Kbytes/sec] received. If you take simply the requests per second we are now only serving almost a sixth of the requests we originally served, and that’s without even using Smarty, that’s from just simply including the codeline!

Conclusions

To my mind those results and the arguments I’ve cited are pretty emphatic, I can’t justify using a template engine that adds so much overhead when you can achieve the same results using a pure PHP implementation. This view seems to be widely held and accepted, it’s part of the reason why many other Template Engines like Savant, Zend_View and Solar_View all embrace a different ethos to Smarty i.e they don’t compile your templates into PHP because they use PHP as the template language.

If you are at all unconvinced by the arguments I’ve presented then like me you might want to consider the words of Hasin Hayder who is the author of the popular book Smarty PHP Template Programming and Applications which I own a copy of :). He also created this Smarty Cheat Sheet which I along with several of my colleagues have copies of on our desks. Earlier this year Hasin wrote an article entitled Once upon a time there was Smarty, in this article Hasin touches on many of the points that I and others have made, and whilst the depth of my knowledge of smarty could easily be challenged Hasin is without question an expert, so he when he said …

I seriously don’t think there is need of Smarty anymore. Its dead! If 
you guys don’t agree with me, you can spend hell lot of time learning 
that {$name} actually does what you could do with “echo $name”. If 
you write a dispatcher which is smart enough to bring the variables 
from a controller scope to the scope of a view, why do u need to learn 
a separate parser like smarty? ... Learning all those functions, loops, 
logics, caching and others in smarty takes so much time that I would 
love to suggest you to learn writing efficient PHP code in template layer 
rather than learning something like smarty ...

… I stopped and listened … and then I found myself agreeing with his closing statement …

Sometime it’s better to realize thats its time to unlearn something.

  1. phpPatterns, http://www.phppatterns.com/docs/design/templates_and_template_engines [back]
  2. Template Engines, http://www.massassi.com/php/articles/template_engines/ [back]
  3. How to performance benchmark a web server, http://www.cyberciti.biz/tips/howto-performance-benchmarks-a-web-server.html [back]
Next Page »

Powered by WordPress | Theme by Roy Tanck