Are computer backgammon players cheating?
In games which involve combinations of luck and skill, losers often believe they lost because of their opponents' luck, while winners believe they won because of their skill. +Matthew Leifer notices a humorous analogue in computer backgammon: the algorithms are very good, and as a result many people believe the algorithms are cheating.
In games which involve combinations of luck and skill, losers often believe they lost because of their opponents' luck, while winners believe they won because of their skill. +Matthew Leifer notices a humorous analogue in computer backgammon: the algorithms are very good, and as a result many people believe the algorithms are cheating.
Psychology of Backgammon Losers
We would all like to believe that if we win something it is because of our skill, but if we lose it is because of the good luck or cheating of our opponent. This is nowhere more apparent than in the game of backgammon. A good opponent will play in such a way that the dice rolls having highest probability will be "good rolls" for them on their next turn, enabling them to take the other player's pieces, consolidate their position, etc. However, since there is an element of randomness, when these rolls actually come up it is very difficult even for me to believe that my opponent's fortune is due to their skill rather than their luck. Whenever I am being consistently beaten by an opponent I have to fight very hard to make myself believe that there is actually something wrong with my play and figure out how to correct it.
This phenomenon is nowhere more apparent than in computer backgammon. Computer backgammon is essentially a solved problem. Unlike Chess or Go, the best neural network algorithms that run even on modest hardware will consistently beat most human players, and perhaps draw with the best in the world. In fact, we have learnt rather a lot from these algorithms. Many rules of thumb that were believed for years by most decent players have been overturned by computer simulations. Computers tend to play backgammon markedly more aggressively than is natural for a human, but we can show that they are nevertheless doing the right thing. If you are not an extremely high level player, playing against a good computer algorithm feels weird if you are used to playing against humans.
One of the most annoying results of this is that it is impossible to obtain a reliable user generated rating of a backgammon program. For example, GNU Backgammon is one of the two or three best algorithms in the world and an under-appreciated gem of the GNU project as a whole. A full 50% of the user reviews on the Ubuntu software centre accuse the program of cheating by fixing the dice rolls and give it a 1 or 2 star review. It is hilarious how many of these reviews include comments like they "know how to play" or have been "playing for a long time" as if this were evidence to support their claim. The rest of the reviews are by people who know better and they all give it 4 or 5 stars (4 is reasonable due to a few user interface quirks, but the algorithm itself is worth a 5). As a result, the averaged rating is only 3 stars, even though this is widely acknowledged as one of the three best backgammon programs in the world. It doesn't cheat because it doesn't have to, and it actually gives you mechanisms for checking that it is not cheating, e.g. entering dice rolls manually and a tutor mode that will correct your bad plays.
This phenomenon is repeated absolutely everywhere that user reviews of backgammon programs are available. On the Google Play store, I have seen this even on a program with an absolutely crappy algorithm that I can consistently beat on expert level as well as on decent ones that are backed by GNUbg. As a result, it is absolutely impossible to find out which are the good backgammon programs if you don't already know, since they all get a low average rating. It is especially on mobile where the good programs are generally less well known. It seems impossible to fix this problem, so I have a little favour to request from my developer friends:
Please could someone write a script that gets the user reviews and eliminates any that contain language about the computer cheating or fixing dice rolls and then recomputes the average based on the remaining reviews?
We would all like to believe that if we win something it is because of our skill, but if we lose it is because of the good luck or cheating of our opponent. This is nowhere more apparent than in the game of backgammon. A good opponent will play in such a way that the dice rolls having highest probability will be "good rolls" for them on their next turn, enabling them to take the other player's pieces, consolidate their position, etc. However, since there is an element of randomness, when these rolls actually come up it is very difficult even for me to believe that my opponent's fortune is due to their skill rather than their luck. Whenever I am being consistently beaten by an opponent I have to fight very hard to make myself believe that there is actually something wrong with my play and figure out how to correct it.
This phenomenon is nowhere more apparent than in computer backgammon. Computer backgammon is essentially a solved problem. Unlike Chess or Go, the best neural network algorithms that run even on modest hardware will consistently beat most human players, and perhaps draw with the best in the world. In fact, we have learnt rather a lot from these algorithms. Many rules of thumb that were believed for years by most decent players have been overturned by computer simulations. Computers tend to play backgammon markedly more aggressively than is natural for a human, but we can show that they are nevertheless doing the right thing. If you are not an extremely high level player, playing against a good computer algorithm feels weird if you are used to playing against humans.
One of the most annoying results of this is that it is impossible to obtain a reliable user generated rating of a backgammon program. For example, GNU Backgammon is one of the two or three best algorithms in the world and an under-appreciated gem of the GNU project as a whole. A full 50% of the user reviews on the Ubuntu software centre accuse the program of cheating by fixing the dice rolls and give it a 1 or 2 star review. It is hilarious how many of these reviews include comments like they "know how to play" or have been "playing for a long time" as if this were evidence to support their claim. The rest of the reviews are by people who know better and they all give it 4 or 5 stars (4 is reasonable due to a few user interface quirks, but the algorithm itself is worth a 5). As a result, the averaged rating is only 3 stars, even though this is widely acknowledged as one of the three best backgammon programs in the world. It doesn't cheat because it doesn't have to, and it actually gives you mechanisms for checking that it is not cheating, e.g. entering dice rolls manually and a tutor mode that will correct your bad plays.
This phenomenon is repeated absolutely everywhere that user reviews of backgammon programs are available. On the Google Play store, I have seen this even on a program with an absolutely crappy algorithm that I can consistently beat on expert level as well as on decent ones that are backed by GNUbg. As a result, it is absolutely impossible to find out which are the good backgammon programs if you don't already know, since they all get a low average rating. It is especially on mobile where the good programs are generally less well known. It seems impossible to fix this problem, so I have a little favour to request from my developer friends:
Please could someone write a script that gets the user reviews and eliminates any that contain language about the computer cheating or fixing dice rolls and then recomputes the average based on the remaining reviews?
Is it straightforward to share something someone's written on G+ somewhere other than G+?May 27, 2013
+Qiaochu Yuan Yes, clicking on the datestamp will give you a permalink.May 27, 2013- Herp derp I got here from your Facebook link so clearly you already figured that out. Oops.May 27, 2013
