Scala is a really interesting and exciting programming language because it is a language where…
- you can solve the same problem in many ways.
- you have a lot of powerful tools which can make your life as a developer much easier or much harder.
- everything implicit can be explicit at the same time.
Recent Article: 7.5 IntelliJ IDEA Productivity Tips for Software Developers
I am a Team Lead at BigCommerce. Like many Scala developers, I came to Scala from the Java world a couple of years ago. Scala looked challenging for me and attractive because of its functional programming.
And I don’t regret my choice.
People in our company have various experiences working with Scala. Some of them are novices and just in progress coming from other languages, like PHP. Some of them are developers with many years of experience. So taking into account the most popular problem of Scala, the learning curve, it is not an easy task to build the company ecosystem.
So I will share our experience on how to find the right balance here.
Immutable variables make our lives less complicated. You don’t need to keep in mind information about the state and how it can evolve. You can safely reuse the same object in multiple scenarios. It is especially useful in the world of concurrency and parallelism. No object cloning, no synchronization of the access, no unpredictable values, etc. Operations on immutable objects usually return new immutable objects as a result without “side effecting.” This means that you can chain multiple operations together.
However, there are cases when immutability is not the best option we can choose. The most crucial one is performance. For example, if you need to fetch a large amount of data, modify it, and then aggregate in a new data structure, immutability might become a bottleneck. Memory and garbage collector pressure can quickly eliminate all the benefits. In this case, it would be better to go with mutable data structures, or in some cases, to pay more attention to algorithms’ knowledge.
Recent Article: BigCommerce Launches a Public Bug Bounty Program
Usually it is possible to isolate usage of mutable structures in these performance sensitive parts of the logic and use immutability outside of these parts. For example, there is a method responsible for building a tree. You encapsulate all the “mutable” logic inside the method, but from the outer world perspective, it accepts as arguments and returns as the result only immutable data structures. So all the complexity with mutability is encapsulated in one place and doesn’t have an impact somewhere else in your program. This can be a balanced solution depending upon the needs.
To sum up, depending upon the “performance sensitivity,” immutability can improve the development and review processes by eliminating efforts needed, decreasing chances for human fault possibility.
2. Pure Functions
When a function is pure, it doesn’t have side effects. But also when a function is pure, its signature is more meaningful. In strongly typed languages like Scala, if you see some return type in a pure function, it means that there are no other options, no unpredictable results, only values of this type should be returned. Pure functions make our code more readable and testing more easy. Of course, the ideal case would be to use it everywhere, but in reality, it is almost impossible. It is hard to imagine the real production code without depending upon some 3rd party libraries, so the only thing left here is to make sure that you handled all the edge cases where you deal with such libraries.
A lot of Scala people come from Java where throwing exceptions is a common case.
Throwing exceptions in Scala is a bad practice.
There are several reasons for that.
Let's dive into the Java world a bit. There are two types of exceptions: checked and unchecked. Java compiler forces you to handle all checked exceptions and will definitely inform you in case you forgot to handle some of them. Unchecked exceptions cannot be checked by the compiler, so it is the responsibility of the developer to handle it properly.
Scala uses unchecked exceptions only, so the compiler won't help you in this case. When developers use some function and they don't know anything about its implementation, usually they expect it to be total (for each input value of function exists the proper output result). But throwing exceptions inside this function causes problems, especially when they are undocumented. I think that so-called "exception-driven logic" really complicates the development process. Code becomes less readable and hard to debug. What is more, throwing an exception is quite an expensive operation on the Java Virtual Machine.
So what are the alternatives? We can use the flexibility of the Scala type system.
1) Option. When you want to indicate that it is a successful result or failure and don't care about failure details, just use the Option type. If it is None, that means some failure happened, and you don’t need more details here.
2) Either. When you want to differentiate failures and have additional details about it, use Either. This is a pretty common practice in Scala to use Either for error handling, and by convention the Left value is an error, while the Right is the actual successful result. Pattern matching will force developers to handle both cases.
3) Try. Sometimes, it is impossible to escape from exceptions. A good example of this is if you are using some Java libraries where exceptions are very common. Then just wrap the method call you are using in Try (and then convert it to Either maybe).
4) Future. You can use both Future.successful and Future.failed for handling the errors. In this case, you should stick to the usual Exception hierarchy for errors. If you don’t want to, then you can combine it with previous options. For example, Future[Either[E, R]] (and maybe benefit from monad transformer EitherT from Cats library).
If you think you have an exceptional situation, usually, you don't. Parsing errors, entity not found, database query errors, or similar are usually expected errors, and you can handle this with a standard Scala type system.
Exceptional situations are instances when there is no reason to continue application run and no way to deal with it. This includes out of memory error, virtual machine error, etc. So use non-exception types for expected errors and exceptions for unexpected errors.
Monad is a word which is a complete nightmare for newcomers, and it scares a lot of people from learning functional programming in Scala. But in practice, everything seems pretty simple because you don’t need to know category theory and all of the monad laws in order to use it. You may not even know that something is a monad. For example, Scala collections, Lists, Vectors, Options, they are monads. But what is important is that they have very powerful APIs, they are composable and you can use it for-comprehensions. People really love it for convenience and the amount of the code needed to implement some solution.
Implicits is probably the most controversial topic. And I think it is one of the most sufficient problems in the Scala learning curve problem. The general advice would be, “ Try not to use it,” because the benefits usually turn into drawbacks in this situation.
Scala implicits are a very powerful tool that is misused in most cases. It eliminates boilerplate code and makes it easier to reason about. Everything seems like magic. And at the same time, it is a common case that code can become really unreadable. Debugging can become a nightmare. It is especially hard for beginners. Hopefully, Scala 3 will improve the situation.
Still, I believe that moderating implicits usage and having some standards can help avoid misuse. For example, using implicits to reduce the number of arguments or implicits conversions is not the best practice case. On the other hand, extension methods, type classes are really useful and powerful. Just use it judiciously.
That is my favorite one. Functional effect systems are cool. ZIO, Cats – they use powerful and effective non-blocking solutions in their execution models. Are there alternatives? Scala’s Future might seem like an outdated and forgotten decision, but in reality, things can be different. I won’t deep dive but will try to cover the most important part from my perspective.
Using green threads or fibers is one of the major benefits of effect systems, in my opinion. Smarter resource usage, less thread shifting, less implicits for ExecutionContext. Scala’s Future has nothing to do with that. Fibers might really improve the performance of the application.
Another benefit of effects is referential transparency. Unlike Future, the effect is just a declaration or blueprint of how to do some task. It doesn’t execute the task immediately; the actual execution is a separate step. It makes your code easy to reason about.
Last, but not least, effects have a very powerful and rich API. It is much easier and more convenient to write code with effects than with Future. A lot of boilerplate can be avoided.
If the effects are so good, why then even compare with Future? The reality is that theory and practice can be different things.
First of all, it depends on the logic. If the flow is to fetch some data from a storage and return it to the client, then there is “not much space” for fibers to bring you some performance win. There is a little place for a potential win from thread shifting. On the other hand, if there are a lot of transformations and data aggregation, the performance improvement can be significant.
Secondly, it is common that application depends upon different 3rd party libraries that more often have implementations for Future only or some different Java alternatives in case it is a Java library or is just a synchronous solution. That means that you will need to wrap/unwrap the data into/from effect (or convert into/from something) every time you deal with that library. As a result, a lot of such wrapping or conversions minimize the performance benefits and require more efforts for adapting 3rd party libraries to usage.
Finally, people usually learn Future first because it is from the standard Scala library. And they need additional time and effort to get used to the effects. Besides, there is a project Loom, which brings native green threads support to Java Virtual Machine. What is the fate of effects and Future after its release? Time will tell.
So depending on the conditions, the question “whether it is worth it to use effects and how much commercial benefit they can bring” can have different answers.
To sum up, it is a big challenge to find the balance between all the power and flexibility that Scala provides versus simplicity and real value. Sometimes, Scala can shine, and sometimes it is better to stick to a more traditional way of writing the code. The principle of least power can help here a lot.
If you have any questions, reach out to me on LinkedIn.