example19_get_payoff.zubr

What is the purpose of an optimizer? It attempts to maximize the expected value of the so-called payoff function. In this example we are finally implementing a method specifying the payoff function. First we have to tell zubr about it:

%option getpayoff own // method getPayoff

Then in the definition section we provide the implementation:


protected float getPayoff(VisibleState vs) {

    switch (vs.getVariableValue("alpha"))
    {
        case FALSE:
            return 0.0f;
       
        case TRUE:
            return 100.0f; // TRUE is "better" than FALSE
    }
    return 0.0f;
}

This way we make our optimizer to prefer alpha=TRUE and dislike alpha=FALSE. The example can be executed as usually, with zubr:

zubr example19_get_payoff.zubr > MyOptimizer.java

There are two possible decisions: MOVE and DONT_MOVE. The Perkun section in the zubr specification looks as follows:

values
{
    value FALSE, TRUE;
    value MOVE, DONT_MOVE;
}

variables
{
    input variable alpha:{FALSE, TRUE}; // alpha may have value FALSE or TRUE   
    output variable beta:{MOVE, DONT_MOVE};
}

You can execute the final program directly from my server:
http://www.pawelbiernacki.net/getPayoffOptimizer.jnlp

As is easy to anticipate the optimizer will do MOVE after FALSE, while he would do DONT_MOVE after TRUE. We still don't have the hidden variables here, but the example is sufficient to introduce the getPayoff method.

An interesting observation - if there are no hidden variables then the optimizer could be replaced by a simple function. Only then. The optimizers with hidden variables can be much better than any function mapping input to output, as was shown previously. 

The next example will be based on hidden variables, which is what makes zubr interesting.

example18_simple_automaton.zubr

In this example we define also the method "execute" which is performed once the optimizer finds the optimal decision:

%option execute own // method execute

In the definition section we provide the implementation:

protected void execute(Action a) {
    JOptionPane.showMessageDialog(frame, a.getVariableValue("beta").getName(), "Action", JOptionPane.INFORMATION_MESSAGE);
}

The model in this automaton requires you to enter FALSE, TRUE, FALSE, TRUE,... and so on if the optimizer chooses MOVE, and a constant signal if it chooses DONT_MOVE.

You can run the program from my server.

http://www.pawelbiernacki.net/simpleAutomaton.jnlp

The optimizer always chooses MOVE. What's wrong with it? We should define one more method that allows discriminating the input signals - getPayoff. Then the optimizer will choose such output signals that the expected value of payoff is maximal.

I would like to add some comments concerning the difference between perkun and zubr. Perkun creates all possible visible states in the very beginning, and then for each visible state it generates all possible states. A visible state contains a vector of the input variables values, while a state contains a vector of the hidden variable values. With the increasing amount of hidden variables its demand for memory grows exponentially. On the contrary zubr generates code that creates the game tree dynamically, like a chess playing program. The code written by zubr demands therefore much less memory than an equivalent program in Perkun, but is slower.

Zubr and perkun (and wlodkowic) support hidden variables. This is a killer feature - the hidden variables allow to compress our knowledge about history and are a natural way to do it. The examples discussed so far contained no hidden variables, but it will change.

The code generated by zubr contain my optimization algorithm (just like perkun and wlodkowic). This algorithm has not been documented yet, I tried to write some documents about it, but the result was not satisfactory for me. If you want to understand the algorithm please take a look at the source code (classes perkun::optimizer or wlodkowic::optimizer).

When you link libperkun or libwlodkowic to your own program you have to obey the rules of the GPL 3.0, but when you create a Java code with zubr you may use it just as you would use the outcome of yacc/bison http://www.gnu.org/software/bison/manual/html_node/Conditions.html. You are free to use the code generated by zubr in proprietary programs.

Download zubr (and perkun + wlodkowic) from: https://sourceforge.net/projects/perkun/.

Hidden variable based predictor vs. functions

I have created a minimalistic example demonstrating that hidden variables are beneficial. I have written a small program that compares a predictor created by zubr with the function predictors. You may take a look at my code (it is included in the JAR, license GPL 3.0):

http://www.pawelbiernacki.net/TestHiddenVariableBasedPredictors.jar

You may also run the program directly from my server:

http://www.pawelbiernacki.net/TestHiddenVariableBasedPredictors.jnlp

It calculates the scores (amount of correct guesses divided by amount of all guesses) for various lengths of the test sequence. The predictor with the id = -1 is a zubr generated optimizer, the other predictors are simply functions. There are all possible functions tested (there are 16 of them). They have the ids 0..15.





As you can see the hidden variable based predictor outperforms the functions (its score equals 0.92 for the test sequence of length 19 while the best function achieves only 0.70).

The difference between the function predictors and the hidden variable based predictor is that the functions are stateless, while the hidden variable based predictor does have a state. Its state is a belief - a probability distribution over the set of states (vectors of hidden variable values). My point is that the IF THEN construct is too weak to achieve the AI, because IF THEN takes into account the current state only, ignoring the history. The hidden variables are all about history - they are a natural way to compress our knowledge about history. Therefore the optimizers/predictors based on hidden variables are so much better than the stateless predictors.

id	1	2	3	4	5	6	7	8	9	10
-1	0	0.25	0.5	0.63	0.7	0.75	0.79	0.81	0.83	0.85
0	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45
1	0	0.25	0.42	0.44	0.45	0.46	0.46	0.47	0.47	0.47
2	0	0.25	0.33	0.31	0.3	0.29	0.29	0.28	0.28	0.28
3	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45
4	0	0.25	0.42	0.5	0.55	0.58	0.61	0.63	0.64	0.65
5	0	0.25	0.33	0.38	0.3	0.33	0.36	0.38	0.33	0.35
6	0	0.25	0.5	0.5	0.4	0.42	0.5	0.5	0.44	0.45
7	0	0.25	0.33	0.31	0.3	0.29	0.29	0.28	0.28	0.28
8	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45
9	0	0.25	0.5	0.5	0.4	0.42	0.5	0.5	0.44	0.45
10	0	0.25	0.33	0.38	0.3	0.33	0.36	0.38	0.33	0.35
11	0	0.25	0.42	0.44	0.45	0.46	0.46	0.47	0.47	0.47
12	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45
13	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45
14	0	0.25	0.42	0.5	0.55	0.58	0.61	0.63	0.64	0.65
15	0	0.25	0.33	0.38	0.4	0.42	0.43	0.44	0.44	0.45

Take a look at the numbers (the first row is the hidden variable based predictor).

You may wonder why I used the term "predictor" rather than "optimizer". Well, this has something to do with the nature of my example. The action performed by this optimizer is a bet - it tries to predict the next value of one of the input variables. I will discuss the example used here later.

Zubr can be downloaded from https://sourceforge.net/projects/perkun/.

example17_get_model_probability.zubr

In this example we tell zubr we will provide our own implementation of the method getModelProbability:

%option getmodelprobability own // method getModelProbability

Then in the definition section we provide the implementation:


protected float getModelProbability(VisibleState vs1, State s1, Action a, VisibleState vs2, State s2) {

    // here we can query the visible states for input variables
    // the states for hidden variables (there are none at present)
    // and the action for the output variables (also none at present)
   
    // vs1 and s1 represent the current visible state and state
    // a represents the action
    // vs2 and s2 represent the future visible state and state
   
    System.out.println("current alpha => " + vs1.getVariableValue("alpha").getName());
    System.out.println("future alpha => " + vs2.getVariableValue("alpha").getName());

    return 0.5f;
}

The VisibleState, State and Action are classes created by zubr. We can use their methods getVariableValue to query them for the input variable values, hidden variable values and output variable values, respectively. The method getVariableValue returns Value (an enum created by zubr).

As usual we can process our example with zubr and compile the result Java class:

zubr example17_get_model_probability.zubr > MyOptimizer.java

You can run the program directly from my server:
http://www.pawelbiernacki.net/getModelProbabilityOptimizer.jnlp

However it only consumes the input, no output is shown. In order to see what the optimizer decisions are we will have to redefine the method "execute".

In order to cancel the execution press "Cancel" button on the getInput dialog box.

example16_on_error_in_populate_belief_for_consequence.zubr

In this example we will add an error handling method. First we have to tell zubr we will do it. In the declaration section add:

%option onerrorinpopulatebeliefforconsequence own 
// with this option we tell zubr we will provide our own implementation of the method 
// onErrorInPopulateBeliefForConsequence

Then in the definition section we provide its implementation:


// this is the implementation of the method we promised to provide:

protected void onErrorInPopulateBeliefForConsequence(Belief formerBelief, Action formerAction, VisibleState vs) {
    JOptionPane.showMessageDialog(frame, "error in populate belief for consequence", "Error", JOptionPane.ERROR_MESSAGE);
    System.exit(0);
}

As you can see the method takes following arguments:

• Belief
• Action
• VisibleState

They are all classes defined by zubr. The error happens when given the belief we attempt to perform an action and obtain as a result the visible state that was unexpected. Our implementation ignores them and simply displays a dialog message, then exits.

If you download zubr (https://sourceforge.net/projects/perkun/) and process the file example16_on_error_in_populate_belief_for_consequence.zubr from the "examples" folder with it then you obtain the optimizer Java code:

zubr example16_on_error_in_populate_belief_for_consequence.zubr > MyOptimizer.java

This Java code can be compiled and executed. You can run it from my server:

http://www.pawelbiernacki.net/optimizerWithErrorHandling.jnlp

This program just allows you to input the alpha (twice) and then exits displaying our error message. Why? The problem is we did not provide any model, i.e. whatever happens will be unexpected for our optimizer. In the next example we will provide the model and things will become more interesting.

example15_get_input.zubr

An optimizer created by zubr communicates with the outer world via two methods:

• getInput
• execute

What is an optimizer? From our point of view the optimizer runs in a loop:

    while (loopIsRunning):
        getInput(...)
        ....
        execute(...)


In this example we tell zubr that the method getInput will be provided by us. The other one, "execute" will be a default one. We will provide an implementation of getInput which will enable the user to set one input variable at one of two values it may have.

// This is an example zubr specification.

package optimizer;

%option class MyOptimizer    // here we modify the target class name, it is optional
%option getinput own    // with this option we tell zubr we will provide our own implementation of the method getInput

// We can import Java packages here.
import javax.swing.JFrame;
import javax.swing.JOptionPane;
import java.awt.Dimension;

In the middle section we provide the Perkun code declaring one input variable, alpha, which may have one of the two values - FALSE or TRUE.

// here we put the Perkun code (values and variables)

values
{
    value FALSE, TRUE;
}

variables
{
    input variable alpha:{FALSE, TRUE};   
}

The method getInput looks as follows:
// this is the implementation of the method we promised to provide:
protected void getInput(Map<Variable, Value> m) {

    Object[] possibilities = {"FALSE", "TRUE"};
   
    String n = (String)JOptionPane.showInputDialog(frame, "Alpha=?", "Continue?", JOptionPane.PLAIN_MESSAGE,
         null, possibilities, "FALSE");
      if (n == null) {
        loopIsRunning = false; // here we tell the optimizer we want to break the loop
      }
      else
      if (n == "FALSE") {
         m.put(mapNameToVariable.get("alpha"), Value.FALSE);
      }
      else
      if (n == "TRUE") {
         m.put(mapNameToVariable.get("alpha"), Value.TRUE);
     
}
It is filling the map Map<Variable,Value> named "m" with values for all input variables, depending on the decision made by the user. We are free to use the map Map<String,Variable> named "mapNameToVariable" to access the variable "alpha". The Value is an enum created by zubr.

In more complex cases we will possibly have more input variables, then we will provide a different dialog.

When we run this example with zubr we will obtain a Java code:

zubr example15_get_input.zubr > MyOptimizer.java

After compiling and running it a dialog will be shown to set the variable "alpha" either with FALSE or TRUE. After you enter the alpha value twice (and press OK) an error will occur. Moreover - this error will not be reported, and we would like to have a Swing dialog shown when it happens. How to achieve this? We must provide an important error handling method, which will be discussed in the next example. The default version of this method does nothing, and the error should be handled otherwise.

example14_my_optimizer.zubr

Next example - example14_my_optimizer.zubr. In the declaration section we have the following code:


// This is an example zubr specification.

package optimizer;

%option class MyOptimizer    // here we modify the target class name, it is optional

// We can import Java packages here.
import javax.swing.JFrame;
import java.awt.Dimension;

The instruction %option class MyOptimizer is not a Java code, it is a zubr option. It tells zubr we want to change the default target class name. Nothing complicated. We must take it into account and modify the class OptimizerThread accordingly:


protected static class OptimizerThread extends Thread {
    private MyOptimizer optimizer;

    public void run() {
        optimizer = new MyOptimizer();
        optimizer.loop(1);
      }
}

Now it creates MyOptimizer rather than Optimizer. Also note that we added in the declaration section "package optimizer;" - this means that we have to place the result Java code in a package "optimizer" (you have to create the package). As usual we can execute this example with zubr:

zubr example14_my_optimizer.zubr > MyOptimizer.zubr

The result code should be compiled with a java compiler and executed. But it still only instantiates an optimizer and runs the loop without any means to communicate with the outer world. How to make the communication? Be patient, this will be covered in next examples.

For the Windows users - I have created an installer: http://www.pawelbiernacki.net/perkun.msi.

Download zubr from https://sourceforge.net/projects/perkun/.