Post has attachment
Semantics::Sentiments ran over 6K Gutenberg texts, based on 2K sentiment word classifications, visualized with gnuplot 5.x
Photo

Post has attachment
Light violet what "MetaFS" main components are . . .
Photo

Post has attachment
MetaFS 0.5.1 Update: major breakthrough (from my little coding world) after months of coding; IndexDB 0.1.21 bx is in the same speed range as MongoDB 3.0.2 with WiredTiger engine, while using a fraction (10%) of memory or even less. Just a few days back IndexDB backend performed at 10-20% speed of MongoDB (crawling slow), after some obvious optimizations in IndexDB and metafs-FUSE itself, IndexDB catched up; this took now 5-6 months work to get here: getting rid of MongoDB specific code with introduction of NoSQL abstract layer (I was hoping to get here where I am today), and having IndexDB speed-wise in the same "range" - remember, IndexDB is programmed in Perl with some storage engines (BerkeleyDB key/value, Sophia and UnQLite) which deal with the low level indexing, whereas MongoDB is coded in C++.

I am aware for other use cases IndexDB is still behind MongoDB, e.g. in-place update of key/value isn't supported yet in IndexDB 0.1.21 bx; I might support it soon(ish).

Creating 100K (empty) files into a single directory, IndexDB vs MongoDB:
- Creating: IndexDB performs 80-90% faster than MongoDB, quite a surprise.
- Delete: MongoDB performs 200-230% faster than IndexDB, partially explained that the index resides in the memory and not on the SSD disk, but there might be still room for improvement for IndexDB.

Algorithmically both backends are approached from the new NoSQL.pm layer, so there is no "special" treatment for IndexDB.
Photo
Photo
MetaFS Development
2 Photos - View album

Post has attachment
MetaFS 0.5.0 Update: I'm currently working on de-MongoDB my code and use NoSQL.pm (NoSQL perl layer) solely and I can say I'm nearly done (a lot of detail work and testing required), and make IndexDB as NoSQL-db solid enough to replace MongoDB.

Yesterday I did some benchmarks for IndexDB (with bx storage engine). A surprise, the i7 4780HQ in my laptop performs better than EC2 m4.4xlarge. I included the memory usage, ~30MB for IndexDB, whereas MongoDB easily acquires 1-2GB, depending how big a required index is for a query.
PhotoPhotoPhoto
MetaFS Development
3 Photos - View album

As of today I removed the github repository, as with the coming changes I won't be able to keep track for existing deployments (I haven't heard from anybody using MetaFS actually - so it won't affect anyone), and it would use up too much of my time to maintain some continuity - the next months I work on building a stable release, spring 2016 perhaps is a realistic goal, and a free community release likely will be released on the same or similar license (GPLv3 or MIT), and a commercial release with licensed add-ons (e.g. improved semantic interpretations).

I'm currently also looking for ways to secure the development financially long term, so I can focus on it 100%.

Post has attachment
Some brief screencast of a file browser for MetaFS using html5, trying to find an easy to use way to reduce query complexity.

Post has attachment
Brief screencast of the image query for MetaFS.

Post has attachment
MetaFS Web-GUI query/find/search interface tests, making the complexity useable. #MetaFS
Photo
Photo
MetaFS Development
2 Photos - View album

Post has attachment
More tests of #ImageQuery ... adding term input in combination with color selection and other options:
PhotoPhotoPhotoPhotoPhoto
MetaFS Development
5 Photos - View album

Post has attachment
Now that I enforced indexes on MongoDB for the most important keys, the query of images (out of 280K images) works as expected (~0.5-1.5s response time), quickly coded some simple interface to test it, will release web code later.
Photo
Photo
2015-04-29
2 Photos - View album
Wait while more posts are being loaded