MidPoint Expression Language (MEL) Design Notes

Last modified 15 Jul 2026 07:37 UTC

MidPoint Expression Language (MEL) is based on Common Expression Language (CEL) by Google et al.

Language Features

The expression language is designed as safe (secure) language:

  • It allows access to functions that are explicitly allowed (and implemented) as language extensions. No generic access to JVM, Java libraries or operating system (files) is allowed.

  • The language is not Turing-complete by design, making it hard for an attacker to abuse it.

  • Care is taken to avoid possibility of infinite loops or even complex computation in the expressions, significantly reducing opportunity for resource depletion and DoS.

CEL has a "functional" character. It is an expression language, not a programming language. It does not have branches or loops. However, this is not a major obstacle. Iterations can be done using list processing (filter, map). Basic branching can be done with ternary operator (? :). CEL should be sufficient for vast majority of midPoint mappings, autoassign expression and similar common uses.

Overall, the language seems to be suitable to be used by low-privilege midPoint administrators and power users. However, it is probably not suitable for use by ordinary end users.

Examples

Simple condition:

jobCode == 'A1234'

Username generator:

focus.givenName.norm.substring(0,1) + focus.familyName.norm.substring(0,7) + iterationToken

CEL with midPoint extensions (MEL) works nicely with polystrings as well (did not work in Groovy):

focus.givenName == 'Jack'

Getting list of all OIDs from all `targetRef`s in assignments:

focus.assignment.filter(a, has(a.targetRef)).map(a, a.targetRef.oid)

CEL or MEL

Decided to call the language "MidPoint Expression Language" (MEL)

We are going to extend standard CEL with a lot of functions, for ease of use, convenience, but also to provide essential functionality (e.g. prism objects). The code will not be backwards compatible with CEL.

MEL and Groovy

The ambition is to make CEL/MEL default scripting language for midPoint. CEL/MEL may even be the only scripting language enabled by default, which will make midPoint secure by default (Filter expression are not considered to be scripting language, these will be enabled).

However, CEL/MEL is unlikely to completely replace Groovy in very complex scenarios. Therefore, we would like to keep possibility to enable Groovy even for future deployments, as a tool of "last instance" for heavy customizations. There is no plan to remove Groovy support in foreseeable future.

Implementation

Implementation in midPoint 4.11 is based on cel-java by Google. The cel-java project is not well documented, but we can work with the code.

CEL-Java allows definition of custom types and functions (although it is quite cumbersome), which we are going to use heavily.

Proto (protocol buffer) types are not supported in midPoint, at lease not now. Prism schema does not have easy mapping to protocol buffers schema (e.g. missing persistent item identifiers). This can be done later. For now the prism types are dynamic (dyn), which means that their interpretation will be postponed to runtime. CEL compiler does not deal with Prism schema, it is not be able to check the types in scripts. We can live with that, at least for now.

PolyString, ItemPath, QName, deltas and similar "hardcoded" Prism types are implemented as CEL types as well.

Built-in MidPoint Libraries

Built-in midPoint libraries such as basic and midpoint do not make much sense here. These libraries are designed for Java/Groovy to give user the flexibility and ease of use (relative to Java difficulty). They are not meant to be secure, and they are heavily riddled with Java concepts (e.g. java typing system).

There are several difficulties using such libraries in CEL:

  • CEL is very not like Java. It is not Turing-complete object-oriented environment such as Groovy or Python. Adapting Java libraries to CEL is far from being straightforward. E.g. CEL does not have sufficiently powerful type system or type-based overloading, making translation of heavily-overloaded Java functions in our libraries difficult.

  • Security. CEL is designed to be constrained and safe, which we need to maintain. We must make sure that our extensions and libraries that we provide are secure. Exposing existing libraries may provide too much unrestrained functionality.

  • We would like to have custom language extension (MEL) rather than discrete libraries. We would prefer ease of use and understanding. We want the expressions to look natural.

Therefore, a better approach seems to be to re-work existing libraries in a CEL-compatible ways, to provide the functionality in a manner that is compatible with CEL spirit. This requires manual maintenance of the extensions when the "Groovy-like" libraries change. However, as this naturally provides a barrier against exposing any random and possible insecure method to CEL environment, this may be in a fact a good thing.

We need to think 10 years ahead, not 10 years back.

Google vs Project Nessie

There are two Java implementations of CEL:

  • Google cel-java: Original implementation from Google. It is somehow incomplete and immature, yet it seems to be a reasonably good fit. The project seems to be active. However, it seems to be mostly work of one person, with several minor contributors.

    Problems with this project were discovered later (perhaps too late) during implmentation. See below for details.

  • Project Nessie cel-java: It has some features that would make mapping of Java objects and types to CEL easier. However, we have decided to not map our Java libraries to CEL directly anyway. Seems to be even less mature and has lower code change intensity (single maintainer?). Most commits are made by a bot (renovate).

When implementinf in midPoint 4.11, Google cel-java implementation seemed to be a better fit for us, although it still leaves much to be desired.

Problems with Google cel-java Project

While Google cel-java project seemed to be usable at first (also confirmed by prototypes), serious problems started to appear during implementation:

  1. CEL engine cannot handle null. Even worse, it is not designed to handle null (see comment, issue, and several topics in cel-java-discuss mailing list). The recommended way to handle non-present values in cel-java is to use optionals, which are non-intuitive and difficult to use for non-expert users. It looks like this is an overall pathos in CEL community (see issue on cel-spec), therefore we will need to swim against the stream here.

    Given the discussions linked above, it is very unlikely that cel-java project would change direction, even if we make significant contribution.

  2. Design of cel-java engine seems to heavily rely on static data typing. While this may be a benefit for simple expressions, it is a major problem when working with dynamic data structures (which are omnipresent in midPoint).

  3. null has its own data type (!!!).

  4. Conditional operator (?:) is statically typed too, requiring the same data types in both branches. Which is a problem for expressions such as cond ? string : polystring or even cond ? string : null (as null has its own data type).

  5. The project is using bazel build system, which seems to rely on Google infrastructure. It does not have good integration into Maven ecosystem. The project cannot be built out-of-the-box. Significant modification of the build configuration was necessary to even build it. There is absolutely no documentation regarding the build, not even simple instructions how to build the project.

  6. The build system seems to be vastly over-complicated. There are micro-dependencies at the class (!!!) level. This makes the code is very rigid, resisting modifications.

  7. The project is stuck at Java 8 level (!!!).

  8. Android (and possible other) dependencies are complicating matters, making the project difficult to maintain.

  9. Contribution to the project requires signing of Google CLA (copyright assignment, no guarantee of attribution, patent grant including retaliation, legal indemnification, potentially some form of exclusivity or limitation of contributor’s rights, unilateral changes by Google, etc.), which may be unacceptable risk. Although not completely analyzed yet, there are too many red flags.

Therefore, it is extremely unlikely that we can move forward by contributing to the Google cel-java project.

The interim solution is to maintain a fork. The fork has modified bazel configuration to be able to build the project and publish Maven artifacts. There are necessary modifications in the fork to support nullability, at least at the level that we need. However, some modifications are ugly, more-or-less workarounds, as cel-java was not designed to handle null values. This approach seems to be a dead end.

We need to decide how to move forward. Perhaps switching to Nessie implementation and contributing to that project would be a good path. This needs to be further investigated.

What Needs to be Done?

  • Good handling of deltas may be particularly hard nut to crack (e.g. for audit reports).

  • More tests. Switch some (many?) integration tests from Groovy to CEL/MEL.

  • Figure out the caching (see below), check performance.

  • Finish documentation

    • Reference documentation for CEL/MEL, documenting at least our extensions (material for LLMs).

    • Tutorial - very important, as CEL tutorial from Google is not exactly the best thing a world has ever seen.

    • Examples: common midPoint use cases.

Open Questions

  • How much do we need to expose midPoint schema to CEL? It looks like the DYN CEL type can be sufficient.

    ANSWER: We are good with DYN for now, at least until we figure out how to proceed in the future.

  • Script caching. CEL-Java compilation relies on knowledge of types of variables. Current script cache in midPoint considers only script source code as cache key, not the variables.

  • Performance. Will it be acceptable? With or without pre-compilation/caching?

  • String functions lc and uc are supposed to work on ASCII chars only. We want them to work on international chars as well, which may not be possible and/or break the CEL lang spec.

    ANSWER: Yes, by all means. CEL language spec seems to be more what you call "guidelines" than actual rules, anyway.

  • JSON support? Do we want/need it? Probably not.

  • Would LLMs be able to create good code, even including custom midPoint extensions to CEL?

  • Consider use of macros, especially for vararg cases and lambdas.

Limitations

  • CEL-Java implementation seems to be somehow incomplete and less mature, at least when compared to Rust implementation. However, there are ways to proceed. Maybe we should consider contributing to cel-java project later?

  • CEL-Java seems not to support vararg functions. Arrays/lists need to be used instead (e.g. f([a,b,c]) instead of f(a,b,c)). This may not be a bad thing, given the functional character of CEL. As a workaround, macros may be used to provide illusion of vararg functions (not prototyped yet). This can be added later (post 4.11).

Implementation Notes

Nulls vs Optionals

CEL has two ways to express no value: null and empty optional. This is very non-intuitive, as foo == null does not work for optionals. Any attempts to fix this (operator overload, wrapping all values into Java Optionals, making all variables DYN, etc.) failed miserably, making it all even worse.

Introduction of isNil() and isPresent() methods as well as nil variable was a best solution. These are checking for null as well as empty optionals.

Overall, CEL is not really built to work well with optional/null values. E.g. the provided "Optional extension" to CEL does not work as expected. Following code might look nice, but it does not work:

optional.of(fullName).orValue('John Doe')

Therefore, default() function was created instead.

It would be also nice to have a coalesce operator:

fullName ?? 'John Doe'

However, this is not supported neither in CEL spec nor in cel-java, and there is no easy way to add new operator in cel-java.

Due to unpleasant nature of null handling in CEL, we are introducing common string functions (e.g. substring(), trim()) as global functions. Global functions are much better at handling null values.

See Null Values in MidPoint Expression Language.

Overload of Conditional

CEL conditional operator (?:) needs to have the same data type in both branches. This is usually fine, but it is somehow inconvenient when strings and polystrings are mixed in the two branches. The ?: operator can be overloaded to support combination of string/polystring in branches. However, this turned out to be a very bad idea. Overloading the operator caused short circuit function of the operator to stop. The ?: operator always executed both branches, which was very inconvenient, especially due to handling of null values.

Functions string() vs str()

CEL has built-in string() function which is supposed to convert data to string. This would be nice for explicit conversion of polystrings to strings. However, it fails, as the stock string() function is not built to work with null values.

Hence the str() function.

Mysteries

  • The optional select operator is .? but in source code it is OPTIONAL_SELECT("?.").

  • Strange %% operator is sometimes mentioned in error messages. It is just a typo, or is there some UFO?

Unsolved Problems

What to do with these? They do not have easy solution. Could we live with these? Or do we contribute/fork cel-java? Any other way?

null type

null has its own type in CEL, which is a major pain. The nil mechanism was implemented as a workaround, but it is not a nice solution. See Null Values in MidPoint Expression Language.

See Also

Was this page helpful?
YES NO
Thanks for your feedback