General Tech Talk  - 
 
I'm thinking of trying something weird and would love the community's feedback... I have a client in penalty because of some crappy affiliate links. Like, really crappy. Major blackhat. We're trying to get the affiliates to cut it out, but it's like whack a mole and there are legal issues with just cutting them off. Here's what I want to do. All the affiliate links point to a certain page that is not the homepage (say /abc). What I want to do is make /abc still function, but as a custom 404 error page with all links on it as nofollows so that customers can still navigate into the site and make purchases. Our plan is to let Google know we are doing it in our reconsideration request - we're not trying to hide it, just send a strong signal that we don't want link credit from these rogue affiliates.1. Will this work, if all visits to the page respond with a 404? 2. Do you think this is shady? By the way, this is a different client than the last one I asked about... we do a lot of this type of work.
4
1
John Mueller's profile photoJenny Halasz's profile photoAndy Beard's profile photoAndrew Smith's profile photo
23 comments
 
by the way, long term, we are moving the affiliate program to a different domain entirely. But again, for legal reasons,we can't do that right now.
 
Whoa. Seems like a possible bad idea.

Why not require affiliates to use nofollow or link to an interim page that is blocked by bots/noindex in order to earn cash from the site?
 
Why just dnt u use the disavow file ? (Coz they are affiliate and they have sites that are known and sends traffic)
Or is there new sites popping up regularly ? 
 
I'd probably go with Ashley's idea for the long run. Wissam's will work too and will probably be faster/easier to implement.
 
Affiliates aren't blackhat unless they didn't specify rules for the affiliate program which was not smart. If the affiliate breaks the rules, most people drop the affiliate. Did they set no rules?

Of all the sample links I've seen google show, I've yet to see an affiliate link in the samples. Are you sure it's affiliates and not just another domain they own? My experience is if the affiliate does anything that could ruin the rep of the brand is booted and I've even seen them boot the visitors from the affiliate back to google
 
+Jenny Halasz :

> 1. Will this work, if all visits to the page respond with a 404?

Yes. A 404 (or, better, a 410 if the requesting client supports HTTP 1.1) is a very effective way to remove a resource from the link graph and, as a consequence, to remove also all the links pointing to it.

Source: http://www.seroundtable.com/404-links-google-15427.html


> 2. Do you think this is shady?

Absolutely no. Links pointing to resource returning 4XX status codes are voluntarily dropped from the link graph by Google. It's intended to work in that way.

Also, if you make the resource a 404, you don't need to nofollow its internal links, because from a semantic point of view you are declaring that the resource doesn't actually exist and Google will ignore its HTML contents.


+Ashley Berman Hale : asking affiliates to make their links nofollow is a good option, but the other methods that you cite, unfortunately, wouldn't work.

Neither a Disallow directive in the robots.txt file or a noindex directive in the page linked by the affiliates would prevent them from attributing PageRank&stuff to the destination URL.

The objective is not to show Google that you are not using the received PageRank&stuff, the objective is to prevent the attribution itself.

That's why a 404 status or nofollowing the backlinks or using the disavow tool to "make them nofollow" are all good solutions: all these methods actually modify the link graph, they "remove the roads" that lead to the URL of your website and nothing is attributed to it.
 
Serving a custom 404 here is an ingenious way of handling this but I'm not sure the right way. I actually really like the idea, but since we've heard nothing from Google about conditional 404s as a method to cut the connection (links) between your site and a bad site I'd say it's not the right thing. If you think it's that bad, contact all affiliates you want to keep. Build a new page, link to that in main nav. And actually 404 the old affiliate page. Notifying the "good" affiliates about the change. Then set the new page to index,nofollow and any future bad links here will not pass on to the rest of the site. However, as someone above said, I've also never seen an affiliate link in GWT.
 
+Andrew Smith : I think that Jenny is referring top a real straight 404 status code, visible to anyone, not to a conditional 404 status code.
Eric Wu
+
2
3
2
 
Interesting problem. While I agree with +Enrico Altavilla's assessment, and for this particular case where you already have an issue, I would follow through with the custom 404.

However, I'm not a fan of having my affiliates / partners knowingly link to a 404 page even though it's providing a particular experience. 

Normally, on my own sites, I create affiliate links by linking to an internal redirect service like /redirect/?aff_id=12321, and then robots.txt out that /redirect/ directory. This way the affiliates are not passing any sort of link value across domains.

However, since you're on the other side of the coin, in the future implementation, a separate domain sounds prudent, but also having affiliates link to a page that has noindex, nofollow like +Arsen Rabinovich recommended might be a good option.

Just note, that even a command of noindex, nofollow may not provide the same level of "separation" as a 404/410 might, since the page still exists within the overall link graph (see +Enrico Altavilla's excellent explanation of how PR flows http://goo.gl/XWIFhJ). There's a slight trade off there, but I don't really like the idea of making your partners link to a "dead page". 

Another possible solution is to have your affiliate link to a 302 redirect which then redirects to 404/410 which might be slightly cleaner ... but still feels icky to me. 

You definitely have a few options available to you, so I would just pick what you feel most comfortable with. Personally, I would do a 302 to a noindex, nofollow page ... but then again, I've never run my own affiliate program. So I'm not sure what the exact performance of that would be (both SEO and UX given a redirect).
 
Hi +Jenny Halasz 

"Affiliate URLs" are typically understood in the industry as being setup in a way so that PageRank is not passed.  For example like +Ashley Berman Hale  mentioned, URLs disallowed via robots.txt, nofollowed, 302 redirected and/or other.
https://support.google.com/webmasters/answer/66356?hl=en&rd=1

You can 410 pages if you would like but the problem is unnatural links tend to point to pages where this can't easily be implemented.  Google is pretty good at detecting spammy links to less important pages ;).
https://productforums.google.com/d/msg/webmasters/glDIZyRs5bs/66FpSvPC2QwJ

If you go the 410 route, be sure your custom error page is helpful to users.

I recommend helping your client setup actual affiliate links that do not pass pagerank and that align with industry best practices.  PageRank passing links to affiliates are not the same as "affiliate URLs."  401ing current pages isn't going to help prevent the issue from happening again in the future.

-Brian
 
+Enrico Altavilla : Yes true, custom or conditional I would kill the page regardless.

OP: forget the links and focus on new links dont try and pick peanuts from proverbial by trying to salvage some links to an affiliate page. The 302 will rank the old URL with the content from the new page. But it won't sever the connection like a 404/410. Given most affiliate links pass no positive value but could be hurting, you can always dig up the referral traffic and get those affiliates on board, hence contacting them first.
 
I would do a custom 410. This is not risky at all and is standard operating procedure. 
 
Wow, you guys are awesome. This forum is the greatest invention ever. First, please understand that this affiliate program was set up with no rules by a third party. My client has substantial legal and financial penalties for breaking the contract, so firing specific affiliates is not an option. We are in the process of making substantial changes, but they will take time to implement. And when I talk about blackhat, I'm talking the worst I have ever seen in my 14 years as an SEO. Hacking sites like children's hospitals to insert hidden links in the footer. To answer some of the questions:

1. +Ashley Berman Hale "Why not require affiliates to use nofollow" 
We do require affiliates to use nofollow. Many of them have implemented it, but not all.
"or link to an interim page that is blocked by bots/noindex"
As I understand it from +John Mueller, pages blocked by noindex and by robots.txt can still accrue PageRank. And since the destination URL is always something like http://www.site.com/1234, it's already on the domain.

2. +Wissam Dandan "Why just dnt u use the disavow file ?"
We did. Four times. And each recon request failed because there are 40-50 new signups or new links per day - and a lot of them are black hat as mentioned above. We are talking very high volume. Think an affiliate program on the scale of Amazon's.

3. +Jennifer M "Of all the sample links I've seen google show, I've yet to see an affiliate link in the samples."
All of the biggest links showing in Google belong to affiliates. Not sure why they don't understand these are affiliate links since they all follow a similar pattern, but they don't. And they do list them - hundreds of them.

4. +Andrew Smith "since we've heard nothing from Google about conditional 404s as a method to cut the connection (links) between your site and a bad site I'd say it's not the right thing. "
We are definitely not setting anything conditional. The 404 response would be for all visitors. The only thing I was concerned might be shady is showing the visitors essentially a copy of the home page so that the human visitors could continue to navigate.

5.+Eric Wu " I would do a 302 to a noindex, nofollow page"
We tried that first, and it did not work. I assume because the 302 can pass PR and because the noindex, nofollow can still accrue PR as well. Our long term plan is to take the affiliates to an entirely different domain, but we can't do it right away.

6. +Brian Ussery " the problem is unnatural links tend to point to pages where this can't easily be implemented. "
Fortunately in this case, ALL affiliate links point to this one page. So 410/404 is an option.

+Enrico Altavilla and +Russ Jones thank you for confirmation. I think we will go with this strategy. I will be sure to update the group if it works! Understand, everyone - our intent is not to deceive the search engines in any way, merely to disassociate ourselves from some particularly "talented" affiliates.

Thank you all so much, and if anyone else wants to comment, feel free. The thread isn't closed or anything. Happy Holidays!
 
One thing to keep in mind is that while 404'ing the landing page will make those links drop out of our graphs, it's an algorithmic process that takes place over time as we recrawl them. They won't immediately disappear, and with that, when the webspam team looks at it (for a reconsideration request), they won't immediately see what has happened (the linking pages will still have the link). In other words, you'll need to briefly explain what you did in your reconsideration request, and you'll definitely need to make sure that you're showing a good-faith effort (so that they trust you to not just make it a 200 again). In short, this is not a trivial, standard situation (as you can tell from this thread), so you'd want to be very clear & to the point in what you submit. 

Depending on the site, it's also important to remember that this manual action might just be a small part of the overall reason why the site isn't as visible in search anymore (assuming that's the case), so I'd consider this a small step in the right direction, but not as a stroll across the finishing line :). 

Good luck & let us know how it goes!
 
+John Mueller thanks for chiming in here. Generally speaking, which would be faster - removing 100% of the links and waiting for Google to discover they have been removed or 410'ing the page and waiting for Google to be convinced the page is never coming back?
 
+Russ Jones It depends ... but overall I'd assume that they're take a similar period of time (you'd see gradual changes anyway, it's not an "on/off" kind of thing when we recrawl the web). It's worth also keeping in mind that this only makes sense to do this for links that go to deeper URLs, you wouldn't want to 404 your homepage. 
 
+John Mueller thank you so much for your help, I really appreciate it. Can you confirm 2 things for me? 1. affiliate URLs are sometimes considered in the link graph (I think there's a generalized misconception out there that they are not and I would love to clarify this in a future SEL post) and 2. If we have a 3xx redirect already in place to the /abc page, will 404ing the /abc page still make the links eventually drop out of the link graph? In other words, the site currently functions like this - a link to www.site.com/1234 redirects to www.site.com/abc. The redirect is where the affiliate ID is recorded. We can currently only 404 /abc, not /1234. Is there a preference for the type of 3xx redirect to use?
 
1) yes, 2) mostly (the less it's clear what you're trying the do, the more likely the algorithms will have to make a judgement call of their own), 2b) I'd use a 301 if you don't want the redirecting URL indexed.
 
Good to see that confirmation on affiliate links from a Googler in b&w. However no algo is perfect so some have to pass value. Ahem amazon ahem.
 
Update: we did as +John Mueller suggested and resubmitted for reconsideration. The client's manual penalty was removed in early Feb 2014. This is the first step, but we still have a lot of work to do! Thank you all for your help.
 
A little late to the party but would like to add a couple of additional technical notes for when implementing various strategies

1. Parameters in URLs can cause issues. e.g. if you are serving a custom 404 page or doing some kind of redirect based on a parameter in the URL, make sure Google is forced to crawl the parameter in Webmaster tools
2. If you do a redirect based on a parameter then make sure it still works with a browser back button
3. There are some smart ways affiliates can use parameter ignoring so that Google doesn't crawl the links and potentially create dangling nodes
Add a comment...