Profile cover photo
Profile photo
Dejan Research
522 followers -
Advancing Search Education
Advancing Search Education

522 followers
About
Dejan Research's posts

A partitioning technique for improving the performance of PageRank on Hadoop

There are a lot of research results in large scale graph analysis on Hadoop. The performance of the graph analysis based on Hadoop is impacted by data partitioning. The effectiveness of data partitioning depends on how the data partitioning maintains data locality in each node of cluster, and this would be different from the problems faced with. One way of data partitioning known to be effective is partitioning data by domains.

For instance, this technique could be very useful in partitioning data by areas analyzing web graphs. But this kind of improvement from the data partitioning is limited to specific problems. In this paper, we propose a data partitioning technique based on semi-clustering for analyzing web graphs with PageRank algorithm on Hadoop.

With experiment, PageRank computation with our partitioning technique improves the performance, as the number of iterations increases. This method can be very effective in the case of large scale graph processing.

Authors
Hoon Choi 
Inf. & Software Res. Center, KISTI, Daejeon, South Korea 
Jungho Um ; Hwamook Yoon ; Minho Lee ; Yunsoo Choi ; Wongoo Lee ; Sakwang Song ; Hanmin Jung

Link: http://www.deepdyve.com/lp/institute-of-electrical-and-electronics-engineers/a-partitioning-technique-for-improving-the-performance-of-pagerank-on-j0zdfc0nOh

Post has attachment

Post has attachment
Wtf ("Who to Follow") is Twitter's user recommendation service, which is responsible for creating millions of connections daily between users based on shared interests, common connections, and other related factors. This paper provides an architectural overview and shares lessons we learned in building and running the service over the past few years.

Link: http://www.stanford.edu/~rezab/papers/wtf_overview.pdf
Photo

Post has attachment

Post has attachment
A conversation with scientist and inventor of the World Wide Web Tim Berners-Lee on what is wrong with social networking:
http://www.weforum.org/sessions/summary/insight-idea-tim-berners-lee
Photo

Post has attachment

Post has attachment

Post has attachment
"In some parts of this world the rains predict disease, and a hot, dry, dusty wind is the harbinger of a meningitis outbreak that is yet to come. Now, from where you sit, Google will soon predict the next great epidemic."

http://www.google.org/denguetrends/
http://mappyhealth.com/
http://www.who.int/topics/dengue/en/
http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//archive/papers/detecting-influenza-epidemics.pdf

Post has attachment

Post has attachment
Wait while more posts are being loaded