there's a better formatted version of this (with links that work) on my blog :Seki Web Application Framework : Summer Update
tl;dr for a first concrete application, I've been building Seki up as a CMS. For basic functionality it's now getting close to being usable.
Call for Funding
So I've had a little burst of activity on Seki in the past few weeks. A few blind alleys, but significant forward progress. Overall it's evolving, and the ideas have been crystalizing a little, so I'll lay them out a bit here. I only intended writing a brief update, but then... At some later date I'll probably pull out the bits that aren't Seki-specific, there's quite a bit that I suppose would come under the umbrella of Web app best practices.
I'm now considering Seki a Web Application Framework with a long list of potential applications. They range from fairly generic things like e.g. workflow/issue management or info aggregator to more vertical things like a creative writing assistant or recording studio equipment inventory system. As well as scratching a few of my own itches, I'd like to use Seki as an experimental testbed, kicking the tyres of the relevant technologies. There are plenty of WAFs around, several (like Seki) built on node.js. Seki's Unique Selling Proposition is essentially it's Semantic Web foundation, though as far as I'm aware the full decoupling from the datastore is also unique.
I need to break down the potential applications list pretty soon to help with deciding priorities, but some components are obviously common to many apps, providing top priorities. A minimum viable product (in the sense of something I can usefully make live) drops out of this - a simple CMS.
The components are:
At a system level, there's also the overarching need to support configuration and extensibility/plugability. I'll bundle these under System Admin.
I'll start with this because it's a well-known area.
A CMS requirements checklist can get very long. But the core functionality is about creating, organizing and publishing documents. So the system needs front-end tools for editing content, storage that allows structuring and indexing (with associated user tools) and different ways of presenting the documents. A fairly minimal system would support blogging: create/edit posts (and comments), post discovery (categories, search, related resources etc.) and at least views for single posts and a front page for recent posts. A blogging engine suits me as a relatively short-term goal, I'm a big believer in eating your own dogfood, and as soon as viable I'll start using Seki for my own blog.
Most CMSs are (unsurprisingly) highly focussed on HTML. It is after all the primary format on the Web. However, both HTTP and RDF (Seki's data model) are format-agnostic, and the conceptual model of RDF covers any kind of data, not just documents.
Not unrelated, HTML isn't the only fruit. There are an awful lot more media types out there. Any reasonable CMS will be able to deal appropriately with things like images, audio and video files, but when we're talking about more generalized hypermedia (with an emphasis on hyperdata, Seki's core domain) it's necessary to keep an eye on HATEOS. (though in a broader sense than Roy frames it, with the H standing for Hypermedia rather than just Hypertext). As Mike Amundsen has put it: "Hypermedia Types are MIME media types that contain native hyper-linking semantics that induce application flow. For example, HTML is a hypermedia type; XML is not." (See also : Exploring Hypermedia with Mike Amundsen). In June 2013, after the Web's been around for more than 20 years, it's remarkable that you can pretty much count the media types that support links on one hand (including knuckles). The most widespread ones are probably HTML, Atom, SVG, RDF (all formats) and one or two proprietary systems like Flash and PDF. Even then, to qualify for Roy's HATEOS the type also needs human/agent interface(s) that support the hyperlink affordance. That narrows the list even more - e.g. the only RDF format that has standardized affordances is RDFa (it sits on HTML).
And there's more. A common failing amongst current CMSs is their suboptimal use of HTTP. For example, it's common to find a common endpoint that's used for POSTing content and URLs of the form e.g. http://example.com/index.php?p=423
. These may be translated in use by mod_rewrite or whatever to "Pretty Permalinks" like http://example.com/2003/05/23/my-cheese-sandwich
. Seki will provide these pieces of functionality somewhat more natively: a resource will be called e.g. http://example.com/2003/05/23/my-cheese-sandwich
in the back-end store, it can be created/update/deleted using HTTP (GET/PUT/DELETE) requests directly on that URL. In other words, more RESTfully. This is also in line with some of the core parts of the Linked Data API. It's not a priority, but ultimately,for convenience, I'd like Seki to support more of that, particularly referencing resources using URI Templates.
Incidentally, I just came across the OASIS Content Management Interoperability Services spec. It does use Atom/APP as a base for one of it's bindings, but most of it is a big vocabulary of XML terms, and I reckon chances are the Interoperability will 99% be limited to other systems that speak CMIS. Haven't said it for a while, but I reckon it's called for here : should've used RDF.
The aim is to reflect the requirements above in Seki, so it could be seen as rather than Content Management System, a kind of Data Management System. There's still a huge amount of Blue Ocean around the Web of Data, and one aim I have with Seki is to use it as an experimental platform, exploring some new waters.
A key feature of Seki is that the backend database is a (SPARQL 1.1 capable) RDF triplestore. Like pretty much any other CMS, some way of mapping between the document format and DB schema is needed. Templating of one form or another is the usual approach, which ideally will be done in a way that helps keep separate the view aspect from model and control. Using an RDF store offers huge advantages over a traditional SQL RDBMS for Web applications in terms of flexibility, in that schemas don't have to be predefined. RDF triplestores also trump both SQL and most NoSQL DBs in terms of Web-friendliness because the URL (more generally, URI/URIRef/IRI) is treated as a first-class type, so everything can be intimately linked with the HTTP protocol.
There are loads of alternatives for templating engine, and Seki uses freemarker.js. Two main factors led to this choice: I've used it before (the original Java version, within Apache Stanbol) and it's simple (essentially just content replacement and includes, simple conditional display and lists). Many templating engines resemble full-blown programming languages (cf. PHP), I'd suggest that makes it too easy to overload the templating with extra functionality, breaking the separation of concerns.
To allow consistency and DRY, the same templating subsystem is used in Seki for formatting between the browser and application and between the application and store (SPARQL queries/updates are templated).
Regarding media types, having a triplestore backend means that the RDF formats are a no-brainer. For Web presentation, HTML is essential. While application/x-www-form-urlencoded is the default media type for HTML form POSTing, it doesn't really lend itself to reuse. A more versatile choice here is JSON. However, in itself, JSON doesn't have a notion of links. But JSON-LD does, with the bonus that it can also be considered a fully-fledged RDF format. Things like images, audio and video can all be handled in their own media types via REST. Atom (and Atom Protocol) are desirable for additional connectivity, see Social Connectivity below. WebDAV support would be nice to have, to allow the use of other clients, but it's not really a priority.
So in the near term I want Seki to be able to handle (in both directions):
Arbitrary media formats
Server-side : While HTML stays the main rendering format, I've started augmenting this with a bit of RDFa. This is easy with the templating system.
I've also been changing things to make JSON-LD the primary format for passing info to the server. There's still quite a way to go with this, mostly a matter of more general refactoring and code reorganization. I've got a libs at hand that help with this, notably node-rdf and json-ld (I held off incorporating an RDF lib into Seki as long as I could as for most things it would be overkill, but when it came to format conversion it seemed essential).
I've done a little coding towards 'baking' pages (turning them into static files), the original motivation being performance, though it does seem a generally a good idea, as Aaron has described. It's since occurred to me that this should be mighty handy for archiving and versioning. Simply bake older versions together with relevant metadata at a URI discoverable from the latest version of the page.
Client-side : I've run through a few editing UIs -
simple form textarea - functional but ugly/limited
TinyMCE - was pretty nice (and the RDFaCE RDFa + auto-annotation extension is very cool) but at the time I was running a separate HTTP server (in the Seki node.js code) on a different port to serve static files, and I ran into painful XSS issues (I've since reverted to serving static files on the same port, but later want to offer an external static server as an option)
Aloha - has a neat UI, but it's surprisingly big, and when I tried customizing it (with CSS etc. and Backbone.js) it seemed more trouble that it was worth. Around this point I remembered Create.js - a bit of embarrasing memory loss as I did a keynote at a workshop in Salzburg featuring it last year).
Create.js (see demo) - this is more or less a wrapper for an editor which makes it operate as a VIE component. It supports other editors (including Aloha) but the default is Hallo, my favourite editor to date.
I wanted to use jQuery and jQueryUI components in Seki's views, move to JSON-LD for client -> server messages and use Backbone.js to logically structure the messages. This is pretty much how Create.js operates, plus RDFa to indicate editable areas etc (a slight caveat is that it follows an earlier version of the JSON-LD spec, but it only took a few lines of code to massage messages into shape).
The general design approach of Create.js/VIE components is really good, they support a good degree of system decoupling.
Right now I've got Create.js basically integrated, though it still needs a fair bit of tweaking.
tweaking config of Create.js
making views for blog front page, archive links etc.
adding comment facility
What I've not got around to yet are the finer points of HTTP, notably the bits needed for caching (ETag & Last-Modified). But basic support for these is relatively straightforward.
I recently covered the requirements and design of this part of Seki in User Management via RDF. [Oops, I just overwrote the image over there with the latest version - oh well, it is the latest version...]
I've since tweaked the vocab a little to improve modelling of Spaces (sets of resources) and added direct linkage to FOAF profiles.
As it happens, the jQueryUI demo for a modal form dialogue happens to be the Users part of a user management UI, giving me a good starting point there.
I've put together a little bit of code to create a few default RDF graphs in the triplestore on startup (with the --init command-line option), including default instance data for admin and anonymous users in a graph (http://hyperdata.org/users
Additional I've set up a basic user registration form. For this (and similar forms) I plan to use the VIE form generator component. Unfortunately this currently only supports the JSON model used by schema.rdfs.org
. I've started a little bit of utility code to transform RDF vocabs into this format.
make admin UI to edit Users, Roles and Permissions
make views for individual resources in the users graph (Users, Roles and Permissions)
make ASK queries to check user's permissions for a given resource
hook into auth chain, providing appropriate representation rendering
The general direction of this is towards the blogosphere, Facebook, Google+, Twitter, Delicious, Flickr, Pinterest etc. Hooking into the blogosphere can in part be achieved by providing a syndication feed and a built-in aggregator. Adding knowledge of FOAF allows 'Friending'-like functionality. And all this can fit in with a distributed Web environment.
I have no intention of implementing anything as complex as Facebook, and am wary of walled gardens, but then again there's no reason to block such a thing as a potential future application. Per-user spaces (like individual blogs, G+ pages or Twitter streams) and threaded discussions are certainly within scope.
When it comes to things like Facebook's games and apps - it's a nice idea, however cheesy they tend to be. See pluggability below.
For linking into existing social net accounts, that can be achieved in some cases via relatively straightforward APIs, otherwise more complex apps that plug into the host system are potential options.
Blogosphere connectivity is fairly high up the priority list, mostly because I want to use it myself. Atom feeds are the first thing, and a built-in aggregator should be relatively straightforward later on (I've built several in the past, having a triplestore backend is really handy). There is a bit of trade-off judgement required with things like this. While snagging things like one person's bookmarks from Delicious can be relatively lightweight, it won't be desirable to mirror the whole of the blogosphere locally. I've not looked with an eye to hooking into Seki yet, but there are plenty of summarising and indexing/metadata-extracting tools out there, storing a little bunch of triples for pages of interest would be good.
Spidering of FOAF (and similar material) is on the list, some care (maybe limit to 1 hop) will be needed not to frighten the horses by appearing to invade privacy.
For the fun of it I might well add a Jabber chat client/server. Might even look into audio/video conferencing out of curiosity. Not really a priority right now though.
All I've tried along these lines is grabbing & converting a dump of my Delicious bookmark data, putting that in the store to experiment against.
FOAF profile builder (a la FOAF-a-matic)
Here I'll also include bits that didn't quite fit in the sections above. Most is Seki-specific so I'll roll the following together :
Requirements & Seki Design
System configuration : Some of the config is dependent on the host system and target SPARQL 1.1 store, those pieces must be configurable externally. Within Seki there's the need to load config info on startup and (ideally) allow its modification. At runtime there's a lot of potential for configuration and even system extension using declarative definitions in the RDF.
Data management : In the most general case, RDF editing can change any data. While crude, the query forms that are generally bundled with SPARQL servers (like Fuseki, the one used by default in Seki) offer this facility. A tool something like the (now stale) Tabulator generic data browser/editor would be very nice to have.
It's hoped that VIE will help with application-specific data management.
Pluggability : to act as a framework, it must be possible to plug in functional blocks or modules. Initially this will be limited to CMS-like themes (packaged up custom templates, bootstrap data and CSS etc).
Drawing back a little, blocks of functionality could be implemented by building them as (quasi-) self-contained services on HTTP, accessed via a RESTful API. Hooks could be inserted into the code at appropriate points at runtime to allow calling such services. I think it would be best to view such services as little agents (I did some work around this a few years back, which I presented at a Scripting for the Semantic Web meetup at ESWC2007 - must dig out the slides). If the (HTTP) messages are suitably self-descriptive, it should be straightforward to set up a simple Flow-Based Programming DSL, maybe layered on async. (See also: NoFlo). Of course, such services/agents could be completely external to the system.
I've put together a bit of code to create default graphs and populate them on startup (with the --init command-line option does bootstrapping).
By default Seki will serve static files on the same port as dynamically generated pages. It's now sing my own hacky server code. This only has minimal features so far (compared to say node-static), though for a bit of extra flexibility and to allow reuse I've implemented it as Connect middleware. My justification for DIY is that I want to know what's going on internally, better to provide hyperdata/hypermedia support.
integrate ACE source editor (just for HTML & Turtle for now)
experiment with VIE components, in particular the form generator
graph creation & data file loading form
set up Stanbol as an external service, dynamically hook Seki into it
figure out theme packaging definition & installation
explore Tabulator-like tools, consider implementing something similar
refactor the whole of Seki, setup up to allow npm packaging
tests & docs
There are a bunch of other idea bits and pieces floating around, especially using Seki to actually build custom applications. But I think the above covers most of the priorities for a good while. They may well change as I go along.
Call for Funding
My recent burst of activity was done in time when I really should have been doing work that would pay the bills. So if anyone has the ability to fund this or knows of any approaches I could try to funding, please let me know. While the dev path for Seki itself is pretty well determined for the near future, I will happily be influenced by $$$. In particular, if you have an idea for a web app that isn't already out there, fund me a bit (every little helps) and I'll see what I can do about implementing it. The core Seki framework will always be open source, but I wouldn't object to building closed source applications on top.
If you're tempted by this but want to wait until you've seen a demo, let me know and I'll ping you when I've got something live.
Seki lives on Github and has a G+ page (with some older docs). There are also rough TODO notes & refs. on Workflowy, and a handful of posts tagged 'Seki' on my blog. I'm @danja, firstname.lastname@example.org