Architecture #14870
Updated by François ARMAND almost 5 years ago
Historically, we used `liftweb` `Box` type to handle result which may fail in Rudder. This was good and quite on the edge 10 years ago, and is good, because `Box` usage brings two majors features: - a) `Box` clearly split appart the nominal path (`Full[A]`) from errors (`Failure`). In that regard, `Box` is the same than `Either` or any modern `IO` monad (but remember that at the time, scala `Either` wasn't right-biased...) - b) (`Failure`), and more importantly, errors come with the possiblity possibliity to iteratively build up explanation, giving more context depending where the error happen and chaining user-oriented message. On that second point, it means that you can "chain" explanation, so that each layer can set the relevant context for the error, typically from very technical for lower layer, and more user-oriented in higher one. These stack of messages can then be formatted and displayed depending of the targetted audience. For example, imagine that your UI did an ajax request to get some details on something, and the database is down at that moment. The user oriented message then can be read as: "There was a problem with data access, please retry or contact your administrator" while the log display: <pre> [.....] There was a problem with data access, please retry or contact your administrator <- JSON request to URL .... returned an error <- Impossible to get details for configuration with ID xxxxx <- Connexion to database error: (technical details of why)" </pre> This is a very good, and extremely important feature for a project like Rudder. So we use it pervasively in Rudder code (> 120k usage in Rudder 5.0). Nonetheless, `Box` has three non-the less some major problems: - 1/ it's a lift dependency, and lift is web-framework oriented, and not much used beside that case. We would prefer to have an effect/error lib for that (and bridge it with `Box` in the web part). This one is not a breaking one, though. - `Box` is tri-stated: `Full[A]`, `Empty`, and `Failure`, and 10 years of usage lead us to believe that even with the stricter discipline, `Empty` semantic is impossible to maintain in time. We never know if it's a non-explained error, or if it is a expected empty case. The semantic of `flatMap` let us believe it's the first interpretation, but actually, we never want to have not contextualised errors. So that state is more of a burden than an help (we still have to deal with it even if we banned it), help, and contribute to massive puzzling for new commers. comers. - and more specifically, `Box` is not at all a principled effect management library, with verified applicative/monad laws, and helpful combinators. `ZIO` answer the three draw backs of `Box`, because obviously it is a strict, principled effect library, developped with 10 years of R&D evolution on the topic, subject, and is even pushing the state of the art on it. the subject. It goes even way further than what provided `Box`, further, bringing a whole new world of async and concurrent programing, `Schedule`, `Software Transactionnal Memory`, efficient purelly functionnal queues, etc etc. We were forced to programing that was not possible with only `Box` (and so, we used some other libraries third party library like `monix` for these topics (like `monix`), that, which multiplies multiply the concept and dependencies. to handle that subject). But `ZIO` doesn't help directly for the contextualisation of errors. At least, it helps a lot for the developper, with an extremelly powerful tracing framework: https://www.slideshare.net/jdegoes/error-management-future-vs-zio But (obviously) it does not help for what are `Result` in the context of `Rudder`. This PR introduces 3 things: - the concept of `RudderError` which is the main type of errors in Rudder. It Rudder, and which provides common error cases and combinators. Notably, the defaults errors are: "system" ("system" which encapsulate exception, "unexpected" for value that should not happens for example when building a config, and "unconsistant" for things that should not happen from a business perspective like "that entity should really be in the base because I just checked"; a `Chained` error case which checked") and most importantly, provides a ".chainError" combinator to (yes, suspens) chain an error with a new contextualised message; And an `Accumute` error for the Applicative accumulation of errors. All Rudder errors implement a "msg" method that allows for user rendering of the method, with the correct logic for Chained / Accumulated ones. These basic cases come with combinators like ".notOptionnal", ".chainError", ".accumulate". message. - the concept of `Result`, with a pure and an effect variant. The pure variant is an alias to `Either[RudderError, A]` and is dedicated to non effectful code (ie: no execption, no I/O, no random, no "System time millis" etc) but with perhaps a potential business error. errors. The effect variant is (TADAAAA) dedicated to effectful computation (java bridge that can throw error, I/O, etc). - and of course, ZIO as our effect library. It also provides combinators to go from `Box` to/from `Result`, and port a substantial part of Rudder lower level to `ZIO`. The main set-up is available in: https://github.com/fanf/rudder/blob/arch_14870/use_zio_for_effect_management_in_rudder/webapp/sources/utils/src/main/scala/com/normation/ZioCommons.scala That basic framework also allows to have specialized domain error for a dedicated part of the application. I did it for LDAP connection module (also: technique parsing; entity mapping), as we want to manage LDAP errors with a more precise semantic than what provides the base errors described above. This can be seen here: https://github.com/fanf/rudder/blob/arch_14870/use_zio_for_effect_management_in_rudder/webapp/sources/scala-ldap/src/main/scala/com/normation/ldap/sdk/LDAPIOResult.scala