Knime External Tool – OCR of structures

Knime is a fantastic tool to create automation of handling data without the need for programming. Although, the more complicated the data becomes the better it is to have or to acquire knowledge with regards to Java, XML, SQL, etc.

Some limitations do exist with Knime, again depending on your data and the end-result you require. In certain situations the need to use an external program might arise.

Now, there is an “External Tool” node in Knime, though officially it is designed for usage in a Unix type environment. But, with some tweaking it does work under Windows!

The example I will give here will include chemical structure recognition stemming from pictures, i.e. OCR.

As it often as is with Knime (or programming in general), there are always multiple ways to solve a problem. This here is simply one solution.   

What this workflow does:

Read a directory of PNG (preferred) picture files of structures, even reactions, and converts them to smiles files. The workflow creates a DOS-batch file with the osra commands which gets executed by the external tool node.

To obtain best quality and to know about the recognition limitations, you should read about the OSRA tool via the links below.

What you will need:

  • OSRA – Optical Structure Recognition Application. I use V2.0.0. This app is freeware if you can compile yourself, otherwise you can purchase (and support the programmers) for a modest fee the compiled version.
    https://sourceforge.net/p/osra/wiki/Home/ (Latest is V2.1.0)
    https://cactus.nci.nih.gov/osra/ (though I think this page isn’t updated anymore?)
  • Knime, preferably latest Version >= 3.3.x (though it should run with any V3; originally it was developed in V2.x, so it should run there as well, though I can’t test it anymore at this point).
    https://www.knime.org/
  • Nodes in Knime: Standard installation, including:
    NGS tools [I like using their “Wait” node]
    Erlwood Nodes [used for chemistry part]

In order for this workflow to work properly, you will require following files in following places – (this has to do with the fact that some nodes, as e.g. the external node tool, can’t be opened/executed for testing if certain entered data isn’t available). Thus it’s easier if you copy enclosed, alt. create (empty notepad files would be sufficient) as described:

  • OSRA in following location:
    C:\osra\V2.0.0\osra-bin.exe [I don’t make usage of %PATH% and the batch file included in OSRA distribution]
  • Additional files/folders:
    C:\osra\donotdelete\extToollGreen.txt [can be emtpy; used for giving the “clear” sign when node is done]
    C:\osra\donotdelete\ignore_me.txt [emtpy]
    C:\osra\donotdelete\ignore_me1.txt [emtpy]
    C:\osra\donotdelete\ignore_me2.txt [can be emtpy; echos the cmd line output, can be ignored, potentially parsed, I don’t]
  • Input/Output location:
    C:\test\ [currently; may be anywhere else, it includes your images and the resulting structures]

The first metanode reads the name of the picture files and creates an executable batch file called by the external tool node. [open the picture in a separate window to view full size]

The tricky part is the external tool node, should you do a full reset and not have all the necessary files in place, you won’t be able to open it and do a comparison of the set-up.

And here the flow-variables:

The remaining portions are less tricky and it is a matter of taste what you want to do with the obtained files. In my second metanode, I make a list of all the smiles files (OSRA creates one output file per one input file) and combine it with the original inputfile (resp. filename). [open the picture in a separate window to view full size]

Finally, in the third node, the structure is drawn out to have a visual comparison to the picture input. [open the picture in a separate window to view full size]

After that, it is up to you what you want to do with the results.

A zip-file, containing the workflow and mentioned text files and folder structure may be downloaded from this link. Some examples of varying quality graphics are included.