Native functions¶

Native functions are a special feature of LocalSolver that allows you to create your own operators for your model. They have two main interests:

To create expressions that cannot be represented easily with the available catalog of operators of LocalSolver. For example, LocalSolver does not have a special operator for inverse trigonometric functions (such as arctan, arcsin or arccos). With native functions, you can create them as soon as your coding language has it. In fact, you can create almost any operator or function you want as soon as it returns a valid floating-point number.
To reduce the size of your model. If you have many equivalent expressions, or a recurring pattern, you can replace it with a native function and reduce the number of expressions in the model and thus, reduce the memory footprint of LocalSolver and improve the global search performance.

Caution

Even if native functions are a really powerful and simple to use feature, you have to take care of the few pitfalls listed at the end of the document, especially the thread-safety issue.

Principles¶

To use native functions with your model, you have to proceed in 3 steps:

Create and code your function. The exact way depends on the targeting language (implementing an interface, using a delegate, ...), but the main principle is the same: a list of arguments is provided to your code as input and it must return a floating point number as output.
Instantiate your function and transform it to an LSExpression than can be made available to other expressions. At the end of this step, you will get an LSExpression of type O_NativeFunction, that does not have a value by itself, but than can be used in O_Call expressions.
Pass arguments to your function and call it. For that, you have to create expressions of type O_Call. The first operand will be your native function created in step 2, and the other operands will be other LSExpressions. The values of these operands will be passed to your code/function during the search.

You can use native functions with the modeling language and all the languages supported by the API of LocalSolver (Python, C++, Java, .NET).

Example¶

In this example, we expose the inverse cosine function (also called arccosine) to our model, and we try to minimize a simple expression based on this new operator.

LSP
Python
C++
C#
Java

In LSP, the virtual machine does most of the job for you. Indeed, any function can be turned into a new operator with the special function nativeFunction as soon as it returns a double.

Furthermore, the arguments provided in the O_Call expressions are simply exposed as arguments of the LSP function. Thus, in the example below, the value of the argument x + y is simply passed to the lsAcos function:

use math;

function model() {
    x <- float(-1, 1);
    y <- float(-1, 1);
    func <- nativeFunction(lsAcos);
    maximize call(func, x + y);
    constraint x + 3*y >= 0.75;
}

function lsAcos(val) {
    // math.acos is a function provided in the math module.
    return math.acos(val);
}

In Python, any function can be turned into a new operator as soon as it takes a list of numbers and it returns a double. To create a new native function, use the create_native_function() method.

Then, you can use your function in CALL expressions. To create CALL expressions, you can use the generic method create_expression(), use the shortcut call() or use the specific overloaded __call__() operator on LSExpressions. Values of the arguments of your native function will be exposed through a specific object call a “native context” (see LSNativeContext) that can be seen as a read-only list:

import localsolver
import math

def lsAcos(context):
    if context[0] < -1 or context[0] > 1: return float('nan')
    return math.acos(context[0])

...
ls = localsolver.LocalSolver()
m = ls.model
func = m.create_native_function(lsAcos)
x = m.float(-1, 1)
y = m.float(-1, 1)
m.maximize(func(x + y))
m.constraint(x + 3*y >= 0.75)
...

In CPP, you have to extend the LSNativeFunction class and specifically the call() method to implement your native function (step 1).

Then (step 2), you instantiate your function and turn it into an LSExpression with the createNativeFunction() method.

Finally (step 3), you can use your function in O_Call expressions. To create O_Call expressions, you can use the generic method createExpression(), the shortcut call() or use the specific overloaded operator() on LSExpressions. Values of the arguments of your native function will be exposed through a specific object call a “native context” (see LSNativeContext):

#include <cmath>
#include <localsolver.h>
...

// Step 1 : implement the native function
class LSArcCos : public LSNativeFunction {
public:
    lsdouble call(const LSNativeContext& context) {
        return std::acos(context.getDoubleValue(0));
    }
}

LocalSolver ls;
LSModel m = ls.getModel();
LSArcCos acosCode;
// Step 2 : Turn the native code into an LSExpression
LSExpression acosFunc = m.createNativeFunction(&acosCode);
LSExpression x = m.floatVar(-1.0, 1.0);
LSExpression y = m.floatVar(-1.0, 1.0);
// Step 3 : Call the function
m.maximize(acosFunc(x + y));
m.constraint(x + 3*y >= 0.75);
...

In C#, the signature of native functions must conform to the LSNativeFunction delegate that takes a LSNativeContext instance and returns a double.

Then, you can use your function in LSOperator.Call expressions. To create LSOperator.Call expressions, you can use the generic method LSModel.CreateExpression or use the shortcut LSModel.Call. Values of the arguments of your native function will be exposed through a specific object call a “native context”:

using System.Math;
using localsolver;

...

double Acos(LSNativeContext context)
{
    return Math.Acos(context.GetDoubleValue(0));
}


void TestNativeFunction()
{
    LocalSolver ls = new LocalSolver();
    LSModel m = ls.GetModel();
    LSExpression func = m.CreateNativeFunction(Acos);
    LSExpression x = m.Float(-1.0, 1.0);
    LSExpression y = m.Float(-1.0, 1.0);
    m.Maximize(m.Call(func, x + y));
    m.Constraint(x + 3*y >= 0.75);
    ...
}

In Java, you have to implement the LSNativeFunction interface and specifically call() method to implement your native function (step 1).

Then (step 2), you instantiate your function and turn it into an LSExpression with the LSModel.createNativeFunction method.

Finally (step 3), you can use your function in LSOperator.Call expressions. To create LSOperator.Call expressions, you can use the generic method LSModel.createExpression or the shortcut LSModel.call. Values of the arguments of your native function will be exposed through a specific object call a LSNativeContext:

import java.lang.Math;
import localsolver.*;

...

void TestNativeFunction()
{
    LocalSolver ls = new LocalSolver();
    LSModel m = ls.getModel();
    LSExpression func = m.createNativeFunction(new LSNativeFunction() {
        double call(LSNativeContext context) {
            return Math.acos(context.getDoubleValue(0));
        }
    });

    // Users of Java 8 can simplify the code above by using a lambda:
    // LSExpression func = m.createNativeFunction(
    //    ctext -> Math.acos(ctext.getDoubleValue(0))
    // );


    LSExpression x = m.floatVar(-1.0, 1.0);
    LSExpression y = m.floatVar(-1.0, 1.0);
    m.maximize(m.call(func, m.sum(x, y)));
    m.constraint(m.geq(m.sum(x, m.prod(3, y)), 0.75));
    ...
}

Pitfalls¶

Solver status & cinematic¶

Most of the time your native function will be called when the solver is in state Running. Do not attempt to call any method of the solver (to retrieve statistics, values of LSExpressions or whatever) in that state or an exception will be thrown. The only accessible function is LocalSolver::stop().

Thread-safety¶

The search strategy of LocalSolver is multi-threaded by default. Thus, multiple threads can call your native functions and your code at the same time . This is not a problem as soon as your native function does not have any side effect. In other cases, it is your responsability to ensure the thread-safety of your code by using mutexes, critical sections or any other locking mechanism you want. You can also limit the number of threads to 1 with the nbThreads parameter if you don’t want to deal with multi-threading issues.

Note

This recommendation applies only for C#, C++ and Java. Python (CPython) and LSP interpreters use a Global Interpreter Lock (also called GIL) that synchronizes the access of their underlying virtual machine, so that only one thread can execute at a time. If this special property of CPython and LSP simplifies the use of native functions, it can also have a huge performance impact (see Performance issues).

Performance issues¶

Even if we designed native functions to be as fast as possible, sometimes you will be faced with performance issues. There are two kinds of performance issues that can occur with native functions:

The final result of the search is not as good as it can be expected
The speed of the search is degraded compare to a model without native functions.

The first issue is due to the nature of the feature itself. Indeed, LocalSolver doesn’t know anything about the new operator you add. It doesn’t even know if your operator is deterministic or not. Thus it will not be able to target the search or explore the solution space as it does with operators defined in its catalog. So if you observe feasibility troubles or if you can easily improve the solution returned by LocalSolver, try to reformulate most of your native function with the operators of LocalSolver.

The speed issue is a totally different problem. Calling a native external function is a bit more costly for LocalSolver than calling one of its internal operator, but it is negligible. However, you can encounter severe thread contention issues in Python and LSP. As explained a bit earlier, Python and LSP virtual machines use a Global Interpreter Lock that prevents 2 threads to access the managed code at the same time. Because of this locking mechanism, putting more than one thread for the LocalSolver search with the nbThreads parameter will not increase the number of moves performed by the search, not even keep this number equivalent, but will severely decrease it compared to a single-threaded search.

Warning

If you use native functions in Python or LSP, we strongly recommend you to disable the multi-threading of the search.

Memory management¶

In C++, you have to free the memory of the native functions you created. LocalSolver does not manage memory of objects created outside of its environment. This recommendation does not apply for managed languages (LSP, C#, Java, Python).