Profile

Cover photo
Shalin Shekhar Mangar
Works at LucidWorks
Attended JSS Academy of Technical Education, Noida
Lives in Bangalore, India
233,618 views
AboutPostsPhotosVideos+1's

Stream

Shalin Shekhar Mangar

Shared publicly  - 
 
Nice and simple explanation of the US monetary system. This makes me wonder how India's monetary system differs.
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
This is an amazing machine! Watch as it progressively replaces a rail track.
1
David Smiley's profile photo
 
Amazing indeed.  I wonder why the track train is so long.  And what kind of prep ahead of the machine is needed (level ground).
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
Having grown up seeing these stunts every 'Magh Mela' in my hometown of Allahabad with all the excitement a young kid could have, I feel for for these stuntmen as their shows/careers gradually decline due to the rise of alternate entertainment avenues. I still remember the first time I saw  aMaruti car with these bikes in the well of death and being blown away that it could be possible. Of course, that was before I learnt high school physics :)
Brave stuntmen have long been riding the near-vertical walls of India's Well of Death—but the popular spectacle is on the decline
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
This is very cool!
 
Sub-second high-res PNG heatmaps on 8.5M geonames docs (512x256) on a Lucene/Solr index -- LUCENE-6191 & SOLR-7005   (note: the PNG is for machine consumption, not end-user viewing).
View original post
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
The folks at BloomBerg gave multiple presentations at Lucene Revolution on their very large scale Solr/SolrCloud deployments. They're also big on contributing back!

http://www.bloomberg.com/now/2015-01-15/open-source-bloomberg-solr-work-enhance-enterprise-search/
About the author: Steven Bower has worked for 15 years in the web/enterprise search industry - First as part of the R&D and Services teams at FAST Sear
5
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
 
If you've ever wondered how 2D spatial search algorithms work here's a simple animation, showing how the fast BKD tree addition to #Lucene  (https://issues.apache.org/jira/browse/LUCENE-6477) finds all points within the greater city boundary of London, UK.  The algorithm is described in https://www.cs.duke.edu/~pankaj/publications/papers/bkd-sstd.pdf.

First the bounding polygon (10K points!) is drawn in red, then the BKD tree intersection takes over, recursively visiting all previously indexed cells (shown in gray), testing each point in the cell to see if it's inside the polygon.  The cells were created during indexing  by recursively partitioning space, alternating latitude then longitude, until the leaf cell has between 512 and 1024 points.  Areas with a high point density result in very small cells.

It's interesting to me how many cells wind up "lopsided" as long slivers instead of being closer to squares; I didn't expect this, and it shows how important it is to visualize the things you work on.  Or maybe it's just a bug!

The animation also makes one limitation clear: the search recursion now visits all cells that overlap the enclosing bounding box of the polygon, but this is clearly wasteful as you see cells outside the polygon, but inside its outer bounding box.  To fix this, we need a fast way to check whether the shape overlaps an arbitrary axis-aligned rectangle.

The BKD approach differs from other space partitioning structures like quad trees (http://en.wikipedia.org/wiki/Quadtree) and geohash (http://en.wikipedia.org/wiki/Geohash) because it's data-driven, drawing lines depending on the data set, not static, drawing fixed lines in space regardless of what you are indexing.  It makes it a bit more costly at indexing time, but then at search time it's very fast: ~5.7X faster than #Lucene 's geohash implementation for various bounding-box searches around London.

It can only index points, which should be the common case for spatial search with #Lucene .

Many thanks to http://openstreetmap.org for providing the base map image, bounding polygon for London, and the full database of points and relations (I indexed a ~60 million subset for this animation).
4 comments on original post
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
I really liked the concept of the inflatable Ark Nova. Inflatable, mobile concert halls, how cool is that!
In this gallery of gorgeous photographs, architect Marc Kushner shares his selection of 10 of the most innovative buildings of today and tomorrow, including a reindeer viewing station, an inflatabl...
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
 
Such an innocent title for such a revolutionary improvement to #Lucene : now, BooleanQuery conjunctions (MUSTs) will break up queries like PhraseQuery into a "cheap" (all terms must be in the document) and "expensive" (check positions of all terms to see if the phrase really occurs) and then seek global agreement for all "cheap" parts before checking any expensive parts.

Over time we will fix lots of other queries/filters to break themselves into cheap and expensive too.  E.g. a distance filter can be a cheap bounding box or polygon check, plus an expensive per-hit distance calculation.

Exciting times for #Lucene .
Currently some scorers have to do a lot of per-document work to determine if a document is a match. The simplest example is a phrase scorer, but there are others (spans, sloppy phrase, geospatial, etc). Imagine a conjunction with two MUST clauses, one that is a term that matches all odd ...
View original post
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
Mesmerising!
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
 
Doing I/O properly is hard, and this is all C code... imagine trying to do it from Java instead?
2 comments on original post
1
Add a comment...

Shalin Shekhar Mangar

Shared publicly  - 
 
Solr 5 will release soon. Here are some highlights of the new release.
 
Hot off the heels of Lucene / Solr 4.10.3 and we are already nearing another major release for the Lucene / Solr project. More evolutionary rather than revolutionary as the 4.0 release was, there are still some very exciting improvements coming in the 5.0 release. The release process is likely ...
View original post
1
Add a comment...
Story
Introduction
Committer on Apache Lucene/Solr. Engineer at LucidWorks. Principal Engineer at AOL.
Education
  • JSS Academy of Technical Education, Noida
    Information Technology, 2003 - 2007
  • St. Joseph's College, Allahabad
    High School, 1988 - 2002
Basic Information
Gender
Male
Work
Occupation
Software Engineer
Employment
  • LucidWorks
    Engineer, 2012 - present
  • AOL
    Principal Software Engineer, 2007 - 2012
  • Onyomo.com
    Intern, 2006 - 2006
Places
Map of the places this user has livedMap of the places this user has livedMap of the places this user has lived
Currently
Bangalore, India
Previously
Allahabad, India - Noida, India
Shalin Shekhar Mangar's +1's are the things they like, agree with, or want to recommend.
These 23 Charts Prove That Stocks Are Heading For A Devastating Crash
www.forbes.com

If you've been doubting whether the stock market is experiencing a bubble, these 23 terrifying charts will put those doubts to rest.

Lettuce See the Future: Japanese Farmer Builds High-Tech Indoor Veggie F...
www.gereports.com

Humans have spent the last 10000 years mastering agriculture. But a freak summer storm or bad drought can still mar many a well-planted harv

The IPO is dying. Marc Andreessen explains why.
www.vox.com

Netscape cofounder and venture capitalist Marc Andreessen says the decline of the initial public offering is bad for ordinary investors. He

Book Excerpt: Exposing India’s Blood Farmers | Science | WIRED
www.wired.com

A few days before the Indian celebration of Holi, an emaciated man with graying skin, drooping eyes, and rows of purple needle marks on both

Inside the Mirrortocracy
carlos.bueno.org

There's a problem with Silicon Valley and the subcultures that imitate it. It's a design bug woven into people's identities and sense of sel

Here Is The Mystery, And Completely Indiscriminate, Buyer Of Stocks In T...
www.zerohedge.com

With the Fed having tapered its liquidity injections into the stock market from $85 billion to "only" $45 billion per month, retail investor

The State of In-Car UX
medium.com

No matter the price or the brand, the interfaces that adorn today’s vehicles are in a bad place. Thankfully, there’s hop…

Testing Lucene's index durability after crash or power loss
blog.mikemccandless.com

One of Lucene's useful transactional features is index durability which ensures that, once you successfully call IndexWriter.commit, even if

NoSQL Meets Bitcoin and Brings Down Two Exchanges: The Story of Flexcoin...
hackingdistributed.com

Flexcoin was a Bitcoin exchange that shut down on March 3rd, 2014, when someone allegedly hacked in and made off with 896 BTC in the hot wal

A veteran programmer explains how the stock market became “rigged”
www.washingtonpost.com

A conversation with Eric Scott Hunsader, a well-known critic of high-frequency trading.

Staking $1 Billion That Herbalife Will Fail, Then Lobbying to Bring It Down
www.nytimes.com

The activist hedge fund manager William A. Ackman bet a billion dollars on the collapse of the nutritional supplement company Herbalife, the

The Netflix Tech Blog: The Netflix Dynamic Scripting Platform
techblog.netflix.com

At the core of the redesign is a Dynamic Scripting Platform which provides us the ability to inject code into a running Java application at

Gold Fix Study Shows Signs of Decade of Bank Manipulation
www.bloomberg.com

The London gold fix, the benchmark used by miners, jewelers and central banks to value the metal, may have been manipulated for a decade by

Ken Shirriff's blog: Bitcoin mining the hard way: the algorithms, protoc...
www.righto.com

This article explains Bitcoin mining in details, right down to the hex data and network traffic. If you've ever wondered what really happens

Profitability of Public Transport Systems
pramaanik.wordpress.com

Taking the Bangalore Metropolitan Transport Corporation (BMTC) as an example, this post explains why it is important even for public transpo

Elite Technology On A Working Person's Budget
www.forbes.com

VideoBy Glen Martin The connectivity of everything isn’t just about objects talking to each other via the Internet. It’s also about the acce

Java Memory Model Under The Hood
gvsmirnov.ru

There are many sources where you can get an idea of what JMM is about, but most of them still leave you with lots of unanswered questions. H