Shared publicly  - 
 
Impact of 3rd Party Google Comments on Rankings?

Google was granted patents this week for Google Sidewiki and Google Knol, and both describe reputation scores for commentors and ranking scores for the content they create.

Under an Agent Rank or Author Rank (+AJ Kohn) approach, the association of a comment with a Google Account could lead to the reputation scores of commentors influencing the rankings of pages in both social search and Web search at Google.

Which leaves me wondering how most bloggers and site owners might feel about that.
14
6
Eric Wagner's profile photoTrey Collier's profile photoFrancis Gary Viray's profile photoBill Slawski's profile photo
19 comments
 
I'll bet you may be right on this!
 
I'm viewing this as a transfer/furtherment of the the SiteAuthority system.

Google associates a site with specific terms and decides how much that site is trusted for content about those terms.

The profiles will have a similar setup.
Comments could act much in the same way as links.

If you have a site about X, Google will have assigned you a SiteAuthority for X, and a score for X.
If you then get Joe Blogs, a known expert on X commenting ... then it may act much like a link from Joes site to your page.

I think it is important to note that such things, though similar to Links, will likely carry much lower values ... you may be looking at 100 comments = 1 link etc.

What concerns me is - you still end up with the same issues as Links.
You still end up with a tiered system based on bias.
You may have the best content on the net - but if you don't get links, you won't rank.
You may have the best content on the net - but if you don't interact and butt kiss the upper echelons, you won't rank.

I still don't think G are going to get this right.
They are merely going to dilute things to the point that it will require much more effort to cheat/manipulate - without really identifying actual quality.
Instead, we may end up looking at a culture of back-scratching and payoffs ... G are liable to turn the net into real politics :(
(And lets face it, it already reached that point in some industries over 5 years ago.)
 
+Francis Gary Viray
Jsut about any system is open to abuse.
this one isn't "too bad" so long as G do dilute the values and limit the scope (which I think they will).

The problem is, it will simply further the divides that already exist.
Top tier sites already link to each other.
Top tier people already interact.
It will basically become twice as hard to break into the top for some terms/industries due to the "incest" and "closed doors" in those areas.

You've also got to be mindful of Local Influence.
I seriously doubt G will be able to fully monitor for paid bias ... so you will see more than a few local figureheads earning from visiting/commenting as well as linking to specific businesses etc.
(Not like you will be able to prove that JoePlumber paid LocalLeader $200 to come and comment+Link and that is why they are ranking for X word :( )
AJ Kohn
+
3
4
3
 
Very interesting +Bill Slawski. I'll have to read further but it certainly makes it more evident that Google wants to use activity and engagement as part of the AuthorRank calculation.

I'm not nearly as bothered by the gaming issues that seem to worry so many. I'm not saying it won't happen but I think you're going to find it vastly more difficult.

I'll take the other side in the debate +Lyndon NA. Have you looked at a stream of comments on, say, TechCrunch? A fair amount of dreck and from a number of people who are not anywhere near experts in their field. Two things there, first is that those people commenting might not have nearly as high an impact on the site/page because they have a low AuthorRank.

But if there's no one of real substance commenting. If the value of that dialog is not productive and adding value, then it is reducing the value of that page.

Lets face it, right now the sites that host powerful conversations do win in search results. That's already a powerful signal. So, all this does is better ensure that there is moderation, or that if you're simply copying and pasting comments from one place to another (oh yeah, I see this a lot) then those will count against you.

Now, being paid to come and comment and discuss is certainly something I can envision but it would have to be in that topic and you'd need to ensure those comments were productive. Each time you comment or author content you're putting yourself out there to be scored.

And any scalable abuse - commenting on 20 new blogs - will likely break established norms by a factor large enough to identify it as suspicious behavior.
 
I'll flip it a little.
Why are those sites at the top and winning?
Because they have great content?
In many cases - no.
It's because they are well marketed, and use social leverage to get links etc.

I'll take SEOMoz on this one.
I've seen numerous posts that are basically little more than drivel.
Yet they get acclaim because they are on SEOMoz.

I took an article apart from SEW/SEJ the other day (cannot remember which site).
There were numerous commenters saying "good job" "great post" "fantastic info".
Convincing the morons you are an expert is easy.
So you can reachc ertain heights easily.

The opposite side is even uglier in my eyes.
If you are Not socialy accepted - if you are not best mates with certain folk, if you are not in their pocket, kissing their backside or scratching their back ... then chances are slim that they will give you the leg-up that others who do those things will get.
Worse - upset those people ... and you face ostraciaztion.
You may never get teh rewards you actively deserve.

Social Media is NOT the right way to find quality.
No more so than Links.

So now sites are left with not 1 semi-useless system,
they face two.
They now have to garner links and cultivate acceptance.

Until Google can actually identify real quality content (a nigh unachievable task in many cases), we are left with sub-par methods that involve various negative scenarios.

We all know you can have the best site on the net about X ... and if you have no links, you will not rank.
It's no different here on G+.
If you are not heavily followed and/or interacted with, your posts won't get indexed.

Same flaws, different systems.

I don't see the comment approach being any different.
AJ Kohn
+
3
4
3
 
Ah, but all those butt-kissing comments will drop that commenter's AuthorRank. Those are the people that may ultimately reduce the value of a page! You see, it's that type of garbage I see AuthorRank and a better analysis of comments addressing.

Comments with no value that receive no replies will drag down that commenter's rank and when that page has more of this type of comment than real dialog, well, that's a bit of a signal that you've jumped the shark.

So if your comments are brown and your content is transmitted more often without context or annotations (i.e. - robo-syndication) then the site's rank could slowly erode. The ones at the top could be toppled.

As for it being difficult to get traction because you need people with high AuthorRank to take notice and engage, well, yeah, that's life, that's reality isn't it?

You have to engage with folks and connect with your community. It takes time (I should know). Your peers do matter within a niche in terms of trust and authority. You get a reputation.

Now, you could be the man on the hill with the great content. And at some point people with high AuthorRank might take notice and start linking to you and that's how you win. But if you're not responding to comments, not engaging, not being active, you might as well just write all of this in a word document eh?
 
Yes/No.

First - I'll clear up the the "butt kissing" angle.
I'm not refering to BK-Comments etc.
More the interaction.
you go around brown-nosing some high profilers - you get on their radar ... a % will get returns on those investments.

Basically - you get found by actively shoving yourself infront of people, and how you play it from there.
Most are not "discovered".

Next - how is G going to ascertain a naff comment frmo a good one?
They cannot do it on size/length.
They cannot do it on the number of links/references/citations.
They cannot even rely do it based on +'s/likes etc.

Next - why would G downgrade based on poor/low quality comments?
That's as stupid/dangerous as due to bad links.
You could easily bomb your competitors with autoposters with naff comments.
Not likely.
(Yes, I know - G may take a negative view if you have certain comment types already - but that's more to do with not maintaning/monitoring your site and the bad links out etc.)

Next ... reality?
What's that got to do with Relevancy, Accuracy etc.?
Google is meant to provide the "best", not the "most popular" or "the most manipulated".
What's the point of crafting good content, investing in editing etc.,
when you can simply chuck Mr.Popular 100 and get him and his cronies to vote you up?

Your "peers" and "betters" may not hold any real authority.
That's part of the problem I'm seeing.
The number of "pro's" I'm seeing that are only jsut catching up to things others have said for years ... and yet they are claiming credit/glory for it.
Reputtaion is built on perception - and that's jsut as manipulable as things like Links etc.
Worse - it leaves you at the mercy of social inequality.
You could watch your rankings dashed simply because you disagree with someone popular.
That's inexcusable.

Finally ... Engagement.
Yes and no.
Again - Accuracy and Relevancy and Worth and Value.
Those are what the rankings should be focused on.
Not whether you can invest 5 Hours a day to shoot the breeze, or chit chat with your peers.

Sorry +AJ Kohn - but it's flawed ... with the same issues as G has had for years.
Basing rankings on human actions means you are basing it on manipulatable factors that will include biases of various forms and to various degrees.

I'll point to a fine example.
Science.
How long did they lock Galileo up for?
Was it because he was wrong?
Was it because he was inaccurate?
Was it because they didn't like what he said?
 
+AJ Kohn Let us make it simple. How about if it is really a well-deserved comment applauding such an excellent content? Will that drop somebody's rank? =) And, if you are really someone who keeps on appreciating somebody else's work. Will that be harmful because you are already flagging Google with all these types of comment? =D

Kidding aside, I understand your point building about "reputation" using AuthorRank but just like what +Lyndon NA has been telling, there are ways to game this kind of ranking system.

Top dogs can game the system and close the doors to new ones in the industry. Manipulation and monopoly is a sickening system. Blackhats probably are now readying their cannons to kick some ass on this new ranking system. Will this be a cat and mouse again? I guess, yes.

I agree with +Lyndon NA -- "Basing rankings on human actions means you are basing it on manipulatable factors that will include biases of various forms and to various degrees."

I hope agreeing with +Lyndon NA will not lower my rank. =D
AJ Kohn
+
3
4
3
 
Promotion
Well, yes, that's how it works. You interact and engage with high-profile people in the community to get visibility. Once you get that visibility it's then how it's received.

Because getting discovered doesn't happen. The Internet is not A Field of Dreams.

Comments
Of course you can determine the quality of comments. You do it all the time and even I can think of some ways to determine quality, including spelling, grammar, punctuation, word choice, comment length, etc. etc.

And that +1 button on comments isn't there just for a laugh.

Popularity
I can think of plenty of ways to crack this problem. Speed of reshare might actually be a negative signal. How many ridiculously bland Seth Godin posts are retweeted without the person reading the actual content? I'd say quite a few. But all of that stuff is measurable. Time to view to Tweet. Number of roboTweets. The diversity of the people who share your content. Sheep mentality is discoverable.

Disagreement
There are two levels of this. First, having a debate with someone of high authority adds value so I don't think that would be a negative, it may actually be a positive. You've created a dialog, there's meat on those bones.

But being ostracized would clearly be a different story. Being so far out of the mainstream that you're written off. Yes, that could happen. But wouldn't it anyway?

If somebody were to post and say they have cracked the Google algorithm. They were able to understand every signal and how they interact through some crazy ass regression model we'd all probably say they were a crackpot. But what if that was Galileo? Hey, it happens. Sometimes the world isn't ready, but I don't see that as a Google problem, I see that as a 'human' problem that would be echoed by Google.

Part of the Whole
And remember, no one is saying this is the only metric that's being used. This is but one more way to use what you do and say to help form a better picture of what is meaningful and not on that topic.
AJ Kohn
+
2
3
2
 
Well, not every site will have a commenting system +Bryant Dunivan but commenting is a fairly typical and ubiquitous form of engagement on the Internet.

But sites that have posts about reggae - I'm sure there are raging debates about the best songs and what is truly reggae and the future of reggae. It might not be on wikipedia (who doesn't need any help) but on other sites and blogs.

And remember it's about the identity of those commenting and how that contributes to the whole idea of AuthorRank. What you contribute matters. It won't be the end all of signals but I believe it'll be more and more important, and soon.
 
I wonder - if this is implemented as conceptualized here, will "Comment Authority" be only a positive ranking factor, or both positive and negative?
 
"... I hope agreeing with +Lyndon NA will not lower my rank. =D ..."
ROFL

But - it does highlight a danger.
What if you happen to agree with someone "unpopular"?
That can have dire consequences based on how some people see this sort of thing working.
It can also have real-world (which affects online) relations, and potentially ranking etc.

Comments
Yes/No.
How are you to use a system to determine what we are currently discussing? We could just as easily be copying and pasting snippets and supplanting words.
As far as I know, syntact and semantic are still unstable enough to be unreliable.
As for things like +'s - too noisy by far. They are used to indicate agreement and trust, as well as bookmarking, lol'ing etc.

Popularity
Yes/No
Yes - blatant manipulation is easy to spot - no different than links.
Doesn't mean G have the resources to capture it all (like links) nor the incentive (like links). Further - it's easy enough to delay posting etc.
Sheeple/post lemmings etc. ... should they be devalued simply because they read and agree with a lot of things?
Should they be valued more because of it?
Should only the first X, or Z-Y intervaled interactions count?

Disagreement
Yes/No
Depends on those disagreeing, the type of disagreement, and possibly the subject and audiences ... and possibly even the location.
I've had more than a few run-ins ... the most marked one for me was at Cre8asite ... where it boiled down to my not starting my post with "I agree with what X said, and have this to add".
:sigh:
I've seen the same on numerous sites and communities.
It's pathetic.
Then you have the block culture.
They disagree - they block.
Fantastic.
Not because you are wrong, inaccurate or providing little value - simply because of their ego, saving their face etc.

It is a Google problem - as G are meant to furbish us with accurate stuff. Simply opting for Popular is what caused so much garbage in the first place.

Part of the whole
More than correct.
The problem is - it is likely to be a flawed implementation adding value to an existing and flawed implementation.
They are likely to compound the problems.
The vast majority will never know, nor care.
Most searchers are clueless imbeciles happy to accept what ever is fed to them (separate danger topic there :D).
They don't know about, and thus don't care about, the other 50 sites with perfect copy, fantastic content and that are 100% correct - as they are not being shown it.
The only people that would care are a few die hards (such as myself) and the site/content creators who aren't getting what is technically their due.

Engagement
I have simialr views to +Bryant Dunivan - I see holes in that all over the place.
I tried to get this across to some of the +SQA folk ... and used Tyres as an example ... and ran into a brick wall (poss. partial language barrier, or old G habbits dying slowly ;).
The response was that there are sites/communities for tyre enthusiasts.
Completely ignoring that 95% (guessing!) of peopel searchign for typres don't want to see forums, don't want to read comments, don't want to join a community - they want to find the best place that selss the best tyres (relative "best" to suit their needs/desires etc.).

Most companies simply do not have the time for "this".
Infact - many of us don't either - we make the time.
We aren't checking orders, finding new suppliers, running a front counter etc.
For the "tech" and "independant" crews ... this sort of thing can be done.
For Joe the plumber, for John the florist or Jane the trucky ... this is not a viable method.
Yet should their sites sink because they are busy working, where as their competitor has hired some barely-managing-english foreigner to run media/social campaigns?

Negative ranking?
+Eric Wagner - I honestly doubt G would apply negatives.
They aren't really keen on using negatives (notice the lack of -1 (though it could be useful :D).
Isntead, they prefer to score upwards and accrue/compound values, rather than apply negatives.
(That said - I often suspect that some actions may account for negatives ... and you also have their reactions that may be negative on the odd.)
 
And +Bill Slawski ... jump in :D
Don't leave us splashing in your pool without you (you'll miss all the fun) ;)

Come on sir - do you perceive negatives?
Did you spot anything in the patents to handle some of the holes ... or circumvent abuse?
 
Reading through the comments posted by everyone on this thread, I wish I had the ability to give out multiple plus 1s. :)

+Lyndon NA, you've all moved to the deep end of the pool already. :)

This passage from the Agent Rank patent seems to address some possible concerns about a reputation score getting too large:

Assuming that a given agent has a high reputational score, representing an established reputation for authoring valuable content, then additional content authored and signed by that agent will be promoted relative to unsigned content or content from less reputable agents in search results. Similarly, if the signer has a large reputational score due to the agent having an established reputation for providing accurate reviews, the rank of the referenced content can be raised accordingly.

A high reputational score need not give an agent the ability to manipulate web search rankings. In one implementation, reputational scores are relatively difficult to increase and relatively easy to decrease, creating a disincentive for an agent to place its reputation at risk by endorsing content inappropriately. Since the signatures of reputable agents can be used to promote the ranking of signed content in web search results, agents have a powerful incentive to establish and maintain a good reputational score.

By enabling contributions and interactions to add to a reputation score not only on a website that you might link to via authorship markup in your Google Account, but also in social networks like Google Plus, and in comments, and basing those scores on actual interesting and helpful contributions and meaningful interactions, a good part of those scores is based upon actually quality. You choose where to post, who to respond to, what to write, and which topics to cover. I think that gives an individual greater control over a reputation ranking than they might have over a PageRank.

One of my favorite academic papers is about ranking algorithms and how they need to be built in a way that makes it more expensive and more difficult to manipulate those systems. The paper was written in 2003, and the author held up PageRank as an example of a ranking algorithm that's difficult to game. I wonder if he still feels that way now. I would suspect not. The paper is:

Online Reputation Systems: The Cost of Attack of PageRank
http://u.cs.biu.ac.il/~herzbea/89-958%20seminar/papers/page-rank%20reputation.pdf

A snippet from the paper:

When strangers meet on the internet, reputation is most likely the only feasible way they can establish trust.

_For a reputation system to be useful for establishing trust, it must have a high cost of attack.
Google’s PageRank algorithm is promising because it weights all recommendations by the reputation of
the sources. I have examined PageRank and proved that under some limited conditions, PageRank has a
cost of attack equal to the cost of acquiring PageRank’s initial votes. However, this high cost of attack is
compromised by PageRank’s lack of support for complaints. Attackers can profit highly from buying a high
PageRank because deception goes unpunished._

_An interesting implication of understanding cost of attack is that reputation levels can be assigned market
values. My estimates of the cost of attack of Google are radically different from market valuation. The market
appears to significantly overvalue low PageRanks and marginally undervalue high PageRanks._

As for determining the quality of a comment, there are some suggestions about how Google might do that in the patent that I wrote about in the post:

How Google Might Rank User Generated Web Content in Google + and Other Social Networks
http://www.seobythesea.com/2011/07/how-google-might-rank-user-generated-web-content-in-google-and-other-social-networks/

These could involve things like

- How relevant a comment might be to something it is responding to
- How original it might be compared to other comments
- An inverse document frequency analysis that looks to see if the comment uses words that don't appear very frequently on the Web as a possible sign that the comment is either providing more specific analysis of a topic, or broadening it to provide relevant expansions upon it.
- Other signals such as length of the comment (which might not be a great signal, but it is probably harder to write a great comment that is somewhat short.

An author rank can also police for fake profiles, for authors that post at more than one place from more than one IP address at the same time, or who respond to hundreds or thousands of blog posts as if they were bought through Mechanical Turk. A credential, or reputation score can go up based upon interactions with others who might have high reputation scores, with high quality comments on high quality posts. It might go down by providing low quality comments on posts as well.

The Knol patent also provides an interesting take on Plagiarism in comments:

In addition, copied contributions may be tracked more easily in such a system. Known plagiarism identification techniques may be used to identify authors who have copied content from others. For example, content may be date stamped when it is first submitted to a system, and content that is very similar may be flagged as a potential problem. Such situations may then be pulled into a dispute resolution process, where the putative originating author and the suspected copyist may state their cases. Other users (such as authors who have been part of the system for a long time and have high reputations) may then decide who is correct. Users found to plagiarize or have other harmful conduct (e.g., flaming, inappropriate comments, etc.) may have their reputations lowered, thus resulting in lowered rankings for their work and less traffic by other users to their work.
 
+Bryant Dunivan Reggae discussions? :)

Why haven't Bob Marley's sons seen the same kind of success as their father?

When bands like the Police or the Clash (in Sandinistas) perform Reggae songs, is it really Reggae?

Are the best Reggae songs protest songs, like Peter Tosh's Downpressor Man?

How much of an influence did Mainland US Jazz stations have on the birth of Reggae?

Toots and the Maytals or Bob Marley?

Trying to find a way to get a social/attention/sharing graph of interactions amongst people to work effectively as a ranking system that a search engine can use to influence where pages show up in a social search or a web search is a pretty vast undertaking. I know I'm better off from reading the contrasting comments from +Lyndon NA and +AJ Kohn above about different aspects of how people might interact with one another, and how those signals might be used by Google. I also have more questions now than answers, but I've never seen that as a bad thing.
 
Ouch - when you hit the pool - you hit it, hard :D

First bit that jumped out at me ...
"... In one implementation, reputational scores are relatively difficult to increase and relatively easy to decrease, creating a disincentive for an agent to place its reputation at risk by endorsing content inappropriately. ..."
(Emphasis is mine)
Now that sounds interesting ... and vaguely like linking out to spam from your own site...

"... Since the signatures of reputable agents can be used to promote the ranking of signed content in web search results, agents have a powerful incentive to establish and maintain a good reputational score. ..."
(Emphasis is mine)
Does that read like Verified/Authorship-marked people can increase teh rankings of Authorship-marked content (and a lack of the reverse)?

.

Now - as far as I can see - G are going to be unable to identify real quality.
(I don't care what anyone says on that either - there are simply to many definitions of quality and too many variant situations that would alter the definition)

What they can do though is filter out the junk and the low value stuff.
Yet even that faces some pretty difficult issues.

Question 1 : Just because someone is dyslexic, or has typing issues ... does that mean what they say is any less correct or valuable?

Question 2 : Just because a comment doesn't fit the common pattern, nor fulful the standard criteria of signals ... does that mean it is really rubbish, of no value, to be ignored?

So that leaves us with Human Indication to identify such things.
+'s and citations and references and naming etc.

If you comment 5 times out of 10 comments, yet no one handles you ... does that mean your comments lack interest value?
If someone +'s your comment - does that mean agreement, or appreciation, or something else (51 meanings or such?)?
If some people neglect to + your name, will G still pick up on it, or does it carry less weight (AJ would be less than AJ Kohn, and that is less than +AJ Kohn?)

Even that lot has some holes in it.

So ... we are looking at lots of tiny signals.
Not a single signal itself will account for much of anything.
It will require numerous signals from numerous individuals to amount to much of anything.

Instead of it being a main Ranking signal - I see it being used more to reinforce existing ones (or upcoming ones as I think G are making more changes and focusing a bit more on on-page/on-site).

.

I also have some issue with the concept of negative social interactions.
Sorry - but who is it exactly that decides what is "acceptable" and what is not?
What I deem as correct and safe, you may not.

I'll really put my foot in it here as well.
I've personally experienced Googles concept of "nice".
It isn't.
It's a case of you play by their rules - or you are out.
You are not allowed to defend yourself.
You are not allowed to handle the situation at all.
You are expected to walk away.
Further - based on reactiosn of specific Googlers - you are not entitled to disagree with them without encoutnering backlash either.

At the end of the day - it goes to show what that there is a collective of geeks, nerds and wimps that are involved in making some of these decisions, and that many of them haven't got a clue about the real world that many others inhabit.
(Note: Not all of them fit that description! There are some specific ones that I've dealt with though that fit it perfectly)

If those same people are defining what trolling is, get to decide what is acceptable and what is not - then we may have some upcoming issues.
 
One of the reasons why I like to spend a lot of time with patents is to get a look at some of the assumptions that are being made by search engineers, and I'm not entirely convinced that those assumptions are quite on target all of the time.

If we look at some of the decisions that Google made with some of their past social initiatives like Orkut, with Google Q&A, and others, they've made some missteps. as well.

It's hard to say how strong of a signal any one piece in Google's potential social ranking signals might have, but it does look like they aren't trying to give any one signal too much weight.
 
I think that there is the key.
I think G are smoothing things a lot more ... almsot normalising across the board.
It should be interesting in about a years time, when people realise that they will need to acquire 2-3 times as many links, and have a few hundred pluses to get any real change :D
Add a comment...