Saturday, 18 November 2017

Using TensorFlow™ with Delphi

- or how to use a TStack<T> to simulate a RPN calculator.



This post is a very simple example on how to use "Google's" TensorFlow - which is an open source Machine Learning library. And this is also a tribute to the old HP reverse polish notation calculators I never had 😞.

Update: Hartmut David was so kind to now put his code on GitHub (https://github.com/hartmutdavid/TensorFlow4Delphi) - thanks!. And I did a pull request for the missing OpSub. The repository also includes the newest .dll so you might skip the renaming hassle described below.


One of my best friends in school had an HP-12C while I had a TI-30 - and we had this endless battle on what notation was best - I guess you mock what you do not understand - but then again I never saw him do anything but addition and subtraction on his cool HP-12C 😁

The Delphi wrappers are done by Hartmut David (http://www.hadv.de/de/main/index.html), and based on the structure of the C# porting of the TensorFlow C-API by Miguel Deicaza (https://github.com/migueldeicaza/TensorFlowSharp).

Hartmut has a couple of German articles on his work on the wrappers that also include test units. The wrappers can also be downloaded from his site - and I have not included then here. But might include a link to my git repository, where I have the files including a missing subtract function. I will reach out to Hartmut, and see if we could get his code placed on something like GitHub - which could be beneficial since a bit of syntactical sugar could be added, by maybe adding the contributions of others.

Installation

One way to get hold of the tensorflow.dll is to install it via Pythons pip install as described on the TensorFlow.org webpage here, and then just grab and rename _pywrap_tensorflow_internal.pyd to tensorflow.dll from <UserDir>\AppData\Local\Programs\Python\Python36\Lib\site-packages\tensorflow\python\. While we are at it also grab python36.dll from <UserDir>\AppData\Local\Programs\Python\Python36\ - a bit scary that it seems we have to jump these hoops, when support for others languages is supported. Put these two together with the application we build or in the path.
Also download the .zip from Hartmut site: http://www.hadv.de/de/2017/TensorFlow20171003.zip

The very basics

I am by no means an expert on TensorFlow or Machine Learning, so I will just give a bit of basic in the current context.

TensorFlow is based on a computational graph, that first is build and the run. The graph consists of nodes, that can be connected by zero or more "tensors" as input, and produce a "tensor" as output. We will in this small example use two types of nodes - a constant and a operation.

The code

The form just consists of a TLabel and a TImage populated with an image of a calculator. For the font of the label I used https://www.dafont.com/display-free-tfb.font, which was close enough - even though I did plan to include some code for an extended 7 segmented control - that does behave like the HP display - but that must be another time.

I wanted to make this example as simple as possible, so I do only handle keyboard input by setting the Forms.KeyPreview to true.

We want to use a generic, TStack<Double> by using the System.Generics.Collections, and we also include TensorFlow.DApi and TensorFlow.DApiOperations from Hartmut's wrapper in our uses clause, and create a TensorFlow session and graph variable.

TForm1 = class(TForm)
    Image1: TImage;
    Label1: TLabel;
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure FormKeyPress(Sender: TObject; var Key: Char);
    procedure FormKeyDown(Sender: TObject; var Key: Word; Shift: TShiftState);
  private
    { Private declarations }
    Stack: TStack<Double>;
    TFSess: TFSession;
    TFGraf: TFGraph;

As can be seen above did I include four methods - the constructor and the destructor:

procedure TForm1.FormCreate(Sender: TObject);
begin
  Stack := TStack<Double>.Create;
  Label1.Caption := '';
  TFSess := TFSession.Create;
  TFGraf := TFSess.Graph;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
  Stack.Free;
  TFSess.Free;
end;

The graph is created as part of the session, I just wanted to illustrate that I am working on the graph by have an instance refering to it.

Whenever I press Enter, I want to push the "value" of the label to my stack, and clear the "display":

procedure TForm1.FormKeyDown(Sender: TObject; var Key: Word;
  Shift: TShiftState);
begin
  if Key=VK_RETURN then
  begin
    Stack.Push(StrToFloat(Label1.Caption));
    Label1.Caption := '';
  end;
end;

Finally I want to handle the key-presses to see whether a numeric key or one of the 4 basic operators have been pressed:

procedure TForm1.FormKeyPress(Sender: TObject; var Key: Char);
var
  j : integer;
  a, b, c : TFOutput;
begin
  j := Pos(Key, '0123456789.');
  if j>0 then
    Label1.Caption := label1.Caption+Key;

  j := Pos(Key, '+-/*');
  if (j>0) and (Stack.Count>0) then
  begin
    // Pop from stack into a
    a := TFGraf.OpConst(_TDouble(Stack.Pop));
    // Put Label into b
    b := TFGraf.OpConst(_TDouble(StrToFloat(Label1.Caption)));
    // Set tensorflow operation
    case Key of
      '+': c := TFGraf.OpAdd(a, b);
      '-': c := TFGraf.OpSub(a, b);
      '/': c := TFGraf.OpDiv(a, b);
      '*': c := TFGraf.OpMul(a, b);
    end;
    // call runner on c
    Label1.Caption := TFSess.GetRunner().Run(c).Value.ToString;
  end;
  Key := #0;
end;

So when an operator key has been pressed, the last added value from the Stack is fetched, together with the current value from the "display", the operation node is set and the session with the graph is run, returning the result.

The OpSub function was for some reason missing in the version of the TensorFlow.DApiOperations unit I downloaded - but I added it and have included it here:

function TFGraphHelper.OpSub(x, y: TFOutput; operName: TFString): TFOutput;
var
 l_oDesc: TFOperationDesc;
 l_sBuf1, l_sBuf2: TFString;
begin
 l_oDesc := TFOperationDesc.Create(self, _PTFChar('Sub',l_sBuf1), MakeName('Sub', operName, l_sBuf2));
 l_oDesc.AddInput(x);
 l_oDesc.AddInput(y);
 Result := TFOutput.Create(l_oDesc.FinishOperation());
end;

It would be nice if we eventually could add some syntactical sugar and write something like TFSession.Run(a+b).

One funny note - try to look at the digits in the displayed picture - when you turn the picture upside down - they spell Ole Olsen - who won the Speedway World Championship three times back in the days of "Stranger Things" :D

Contribute and enjoy