如何设计掷硬币游戏的“机器学习”方法

Question

I am finishing up an algorithms class and the professor wants us to do an exercise in "machine learning" I pretty much have free reign in the project, but, it has to fall under the general umbrella of supervised learning. So, I decided to do a coin flip simulation with bets being made by a computer player. My idea is to make the coin biased and see if the computer can discover that bias when placing bets by using training data. I'm really not sure how to approach this..

An idea I had was to increment a win counter and pass that as a parameter into a method when determining the bets after a training round of say 100 bets. That way the computer player could make a decision to place more heads, if for example heads came up with 90% probability. After sufficient training I think the computer would discover the bias.

I have code here to increment a win counter depending on the outcome, but, I'm not really sure where to go from there. IE how to hold the corresponding data for if it was a win and if heads or tails came up and then use that information to place the bets in the next round. Any advice would be greatly appreciated. I am also open to using a different method if anyone thinks I am going about this the wrong way.

Please keep in mind that I am a second year undergrad and have very little knowledge of and exposure to actual machine learning techniques, hence the quotation marks in my question.

public class CoinFlip {

  public static void main(String[] args) {
    int bank = 500;
    int headsCount=0;
    int tailsCount=0;
    int winCounter=0;
    int lossCounter=0;
    String[] flipResults = new String[100];

    ComputerPlayer randomPlayer = new ComputerPlayer();
    Coin coin = new Coin();

    for (int i = 0; i < 100; i++) {
        randomPlayer.placeRandomBet();
        flipResults[i] = coin.flip();
        if(flipResults[i].equals("heads")){
            headsCount += 1;
        }
        else{
            tailsCount +=1;
        }
        if(flipResults[i].equals(randomPlayer.placeRandomBet())){
            bank += 50;
            winCounter+=1;
        }
        if(!flipResults[i].equals(randomPlayer.placeRandomBet())){
            bank -=50;
            lossCounter+=1;
    }
  }
}

public static class ComputerPlayer{

      double bet;
      String heads = "heads";
      String tails = "tails";

    public String placeRandomBet(){
        bet = Math.random();
        if(bet < .5){
            return heads;
        }
        else return tails;
    }

    public static String placeLearnedBet(int wins; int losses){
        //not sure where to start
    }

public static class Coin {

        double coin;
        String heads = "heads";
        String tails = "tails";

    public String flip() {
        if (Math.random() < .9) {
            return heads;
        } else {
            return tails;
        }
    }

}

Answer 1

If you know that coin is distributed uniformly with a bias then you should always go with the majority. If you do not know the distribution of the coin then you want to use an algorithm called randomized weighted majority ( Regret Minimization ). This algorithm basically states that you should choose heads with probability #heads/#flips. As far as the weights are concerned if you know that the coin is distributed uniformly then you can estimate the bias and calculate the variance of your estimate. As the number of flips goes up the variance goes down like the square root of the number of flips. Your wagers should go up appropriately. IMO the weights make the problem a little less clear. What is your goal? To optimize your total expected gain? If so then it is not clear whether you should wager at all until the last coin flip at which point your variance is lowest and wager all your money then. This strategy however does not minimize your variance because all your money lies on one wager. Also if the coin is unbiased is it better to not wager at all or to wager all your money. In both cases you have 0 expected gain but with very different variances.

Answer 2

As far as finding the bias, that's not really all that difficult. Basically, you would just guess the bias based on the number of heads out of the total number of plays and this ratio would give you the average bias. So for instance if you got 51 heads and 49 tails, then you would guess the bias towards heads is 51%.

Ultimately, I think Benjy is correct that you should bet (heads or tails) based on which is more probable (according to your bias). As you get more and more samples, your guessed bias will converge to the actual bias. As far as the betting goes, from an expectation stand point, you should bet as much as possible; but from a realistic standpoint, this is not a good strategy (since you will eventually lose everything if you always bet everything unless the coin is 100% biased one way or the other).

A possible betting strategy could be "forward looking", ie assume you are going to make the same bet for the next n rounds and calculate the probability that you will win vs. lose. You aren't directly trying to find the amount to bet rather you are trying to find the number n (the number of rounds to look forward) that give a certain probability of success. The riskier the betting scheme, the smaller the accepted probability of success (ie the riskier the scheme) and the more conservative the betting scheme, then the larger accepted probability of success. Once you find the number of rounds needed to guarantee you success (within your selected probability) you can then divide your pot by the number of expected rounds, that is bet the maximum amount such that if you lose every one of the next n rounds, you can still make a bet.

You can see that with the above, a conservative scheme would require more and more forward rounds and thus the bet would get smaller and smaller whereas with a risky scheme, the probability of success is small and thus would require fewer rounds and allow for a larger bet.

The last interesting thing is to decide how risky of a scheme you should choose. Based on what I did, the less biased the coin (ie the closer to 50% chance of heads or tails), the riskier you should be and the more biased the coin (ie say 90% chance of tails or 90% chance of heads) the less risky your betting should be. This sort of makes sense because if the coin is completely fair, then there isn't a betting scheme that will usually win so your best just making large bets and taking the 50% chance that you come out on top. On the other hand, if the coin is extremely biased then the game itself isn't very risky and thus your betting scheme doesn't need to be very risky. In the latter case, if you tend to make large wagers then you increase the chance that you lose everything vs. smaller wagers that all but guarantee a large payoff. Keep in mind that as your pot grows, you can make larger and larger wagers without increasing your risk of losing money and thus if the coin is very biased it's quite easy to essentially get exponential growth in your pot.