How to Experiment with NumPy in Flux?

Introduction

NumPy is a Python module widely used in scientific computing area and could improve the capacity of what Flux users could achieve when they write PyFlux scripts.

However, NumPy is a native C-Python module whereas PyFlux scripts are executed by Jython, a Java Python interpreter, which is not able to execute C Python modules thus preventing NumPy usage with Flux.

Nowadays a third-party project named JyNI (Jython Native Interface) provides an experimental solution to execute native C Python with the Jython interpreter, starting with Jython 2.7.1

Since Flux 2018.1 release, Flux has been upgraded to use Jython 2.7.1 and it becomes possible to use the JyNI library to experiment with native C Python modules like NumPy.

Disclaimer

The JyNI library is a third-party library which is still in alpha version and NumPy support with Jython remains experimental. This means that Altair is not responsible for maintaining or fixing bugs related to NumPy integration with JyNI and Jython.

Besides, usage of NumPy with Flux has not been qualified in the current release and nothing has been made to allow a smooth and seamless integration of NumPy with Flux. On the contrary there are known limitations and issues: See the Troubleshooting section for more details.

Prerequisite

Using NumPy with Flux requires the users to download the following third-party libraries:

CPython 2.7.1: https://www.python.org/download/releases/2.7.1/

This is the only CPython version for which the NumPy experimentation was tested. Some issues have been observed when other versions were installed instead of or next to version 2.7.1
JyNI 2.7 alpha 5: https://github.com/Stewori/JyNI/releases/tag/v2.7-alpha.5
Download both
- The generic JyNI.jar
- The platform-specific archive corresponding to your environment. For example on Windows 64 bits use: JyNI-2.7-alpha.5-bin-win-amd64.zip
NumPy 1.13.3: https://pypi.org/project/numpy/1.13.3/
Download the numpy-1.13.3-cp27-xxxxx.whl archive corresponding to your platform

Step-by-step

Follow the steps below to install and configure NumPy for Flux:

Ensure that no other CPython or NumPy module is already installed on the machine
Install CPython 2.7.1
Create a UserDLL directory to store the additional DLLs that will be required to use NumPy with Flux. For example: %INSTALLFLUX%\Third_party\jython-2.7.1\userdll
Create a UserLib directory to store the additional Python module required to use NumPy. For example: %INSTALLFLUX%\Third_party\jython-2.7.1\userlib
Install JyNI
1. Copy JyNI.jar into: %INSTALLFLUX%\Third_party\jython-2.7.1\javalib
2. Extract the JyNI.dll folder from the platform-specific archive into your UserDLL directory. Be careful to copy the JyNI.dll folder ifself (not only its content) inside the UserDLL directory
Install NumPy
1. Unzip the numpy-13.3.3-cp27-xxx.whl archive into your UserLib folder
2. Look for the NumPy DLL inside: <UserLib>/numpy/core
  
  and copy it to your UserDLL folder
  
  The name of the DLL to copy depends on the platform-specific archive you downloaded. For example for windows 64 it is named:
  
  libopenblas_v0.2.20_mingwpy.dll
Starts Flux Supervisor
Configure the Flux Supervisor options to use your UserDLL and UserLib folders for all Flux modules (2D, 3D, Skew, PEEC)
1. Open the Options dialog by clicking on the Options button in the bottom left
2. In the dialog’s navigation tree select the Access Paths > Python node
3. Enter the User Python scripts directory by typing the path to your UserLib folder
4. Enter the User libraries directory by typing the path to your UserDLL folder
5. Apply the changes and close the dialog by clicking OK
Start Flux from the Supervisor
In Flux, test you can use NumPy functions
1. In the PyFlux shell enter the following Python code to define a 3x5 matrix and print it in the console:
2. Execute the code by clicking on the red arrow on the left of the PyFlux shell
  You should see the following output in the Flux console:
  
  [[ 0 1 2 3 4]
  
  [ 5 6 7 8 9]
  
  [10 11 12 13 14]]

Troubleshooting

Flux does not start any more after I configured NumPy for Flux
- What should I do?
  Look at Flux Supervisor Options and ensure that you have configured the User libraries directory for the module (2D,3D, Skew, PEEC) you are trying to use, even if you do not want to use NumPy for this specific module.
- Why do I have to do this?
  After you have applied step 5a and copied the JyNI.jar in
  
  %INSTALLFLUX%\Third_party\jython-2.7.1\javalib
  
  then this jar is loaded automatically whenever you start Flux regardless of the module you choose. Because this jar requires access to the JyNI.dll then you must configure the User libraries directory for all modules otherwise the jar will fail to be loaded and Flux won’t start.
Flux Server does not start any more after I configured NumPy
- What should I do?
  To fix Flux Server, you need to remove the JyNI.jar from the directory:
  
  %INSTALLFLUX%\Third_party\jython-2.7.1\javalib
  
  But then you won’t be able to use NumPy in Flux any more. This is a known limitation in this release and it means that currently you must choose between using NumPy or using Flux Server, but you cannot have both at the same time.